smconsole not launching in xwindow display

[Duc Tran]

We use ReflectionX for xwindow display to launch the smconsole running on our Linux VM where the r12.52 SP1 CR05 policy server is installed.  We have had our fair share of issues and tweaks that are needed here and there to get xwindow display to work properly but the latest issue we are seeing is that once in a while the xwindow will not launch the smconsole on a particular policy server.  The following error message will display when trying to launch the smconsole:

 

"Error: performing semget() on ColdFusion Registry semaphore:Permission denied"

 

At first I thought the issue is due to our ReflectionX client but ruled this out by launching xclock and it successfully exported the xclock display, so this error must be specific to the SiteMinder component for the smconsole.  Rebooting the policy server will a resolve this issue, but the problem appears to come back again later.  I think this problem first manifested when I was changing the directory permissions for the SiteMinder installation directory, but at one point I rebooted the Linux server and this problem went away and I do not recall making any more directory permission changes afterward, but later seeing this issue again.

 

Just hoping that someone out there in the community might have seen this before and can point me in the right direction to figure this out.

 

Much thanks!

[Ujwol]

Hi Duc,

 

Have a look at this :Policy Server Upgrade along with user permission change from root to smuser

 

This seems related to your issue.

Regards,

 

 

Ujwol

[Ujwol]

Did that help Duc ?

[Duc Tran]

Hi Ujwol,

 

My apology for such a late response.  I found that thread earlier and was discouraged because it was specific to that starting up of the policy server, but with your recommendation I revisited that thread and gave it a try by deleting the following files in the /tmp directory but it did not resolve my issue with being able to launch the smconsole with xwindow:

 

  /tmp/GCL-SiteMinder-A.pipe

  /tmp/GCL-SiteMinder-B.pipe

  /tmp/GCL-SiteMinder.sem

  /tmp/snrrpni{{pip

 

What I actually learned about this issue is that it appears that the smconsole fails to launch in xwindow ONLY if the policy server is running.  If I shut the policy server down then try to launch the smconsole then it works, but if the policy server starts up then smconsole will not launch.

[Duc Tran]

Hi Ujwol,

 

I think I must have unintentionally marked this thread as "answered".  I am still having issue with this.  One thing new that I learned from this particular issue is that the problem must be related to some process that the policy server open/creates when the policy server is started which needs to be read/write by the smconsole and therefore, when the policy server is started the smconsole will not launch and the following error is display while trying to launch the smconsole via xwindows:

 

Error: performing semget() on ColdFusion Registry semaphore:Permission denied

 

My operating system is REHL 6.  I use either putty and ReflectionX to export the display to my desktop for the smconsole.  This is still all part of our SiteMinder r12.52 upgrade process from our current version r12.0.  We successfully completed the r12.52 environment parallel upgrade in our DEV and STAGING environment but this is the first time that we're seeing this behavior with the smconsole in our new r12.52 PROD policy servers.  Our PROD environment has 3 policy servers and this is the first policy server installation so far and we're encountering this issue with the smconsole.

 

I did delete the following files in the /tmp directory and restarted the policy server but still same behavior:

GCL-SiteMinder-A.pipe

GCL-SiteMinder-B.pipe

GCL-SiteMinder.sem

hsperfdata_smuser

snrrpni?{{?pip

 

I have not seen this issue with our other policy servers in the DEV and STAGING environment so perhaps I should just delete everything and start all over again fresh and may not run into this again, but figuring out what causing this would be a great victory

[Ujwol]

Hi Duc,

 

I haven't come across this issue myself yet.

Have you tried running strace against the smconsole to see if that provides any additional clue?

 

strace -Ff -t -i -v -o strace.log -s 16384 <command to launch smconsole>

 

Tagging our unix expert markodonohue Patrick-Dussault

[Duc Tran]

Hi Ujwol,

 

No I have not tried the smconsole tracing yet, but I just learned something new about this issue which is quite interesting.  So with my r12.52 upgrade, I am doing the parallel upgrade method so I am building out brand new servers/VMs and installing the r12.52 and configuring the PS SSO between the new r12.52 PS with the existing r12.0 PS.  So far I completed the DEV environment which as only a single policy server.  Next I completed the QA environment, which has two policy servers.  I am now starting on the production environment which has three policy servers.

 

This issue with the smconsole not launching while the policy server is running only appeared on the first of the three new PROD r12.52 server.  As of this morning I finally decided to skip this first prod server and move to the next server feeling confident that this smconsole issue/mess is specific only to that first PROD server but I guess I am wrong.  I completed the second PROD server installation and tested the smconsole behavior and see the same exact issue on this new second PROD server too.  There must be something slightly different within the server OS/libraries for these three new PROD servers compared to the DEV and QA servers because this problem only exist on these three new servers.

 

Here is the really interesting new discovery.  I know that if I shut down the policy server then the smconsole will launch just fine but once I start up the policy server then go into the [smhome]/bin and launch smconsole then I get that error message, BUT if I shut down the policy server then launch the smconsole and while the smconsole is open, I then start up the policy server then from that point on I can stop the smconsole and launch it back again anytime I want just as long as the policy server does not stop and then start up again while the smconsole is NOT running.  Basically the smconsole needs to launch first before the policy server start up so that the smconsole can register its file somewhere before the policy server starts up to overwrite it.

 

This is quite strange, but I am hoping that based on this description, folks could chime in with thoughts on possible causes

 

Regards,

[Duc Tran]

Hi Ujwol,

 

I want to thank you for helping me out with this.  I new it had to relate to a permission issue and eventually figured it out.  My SiteMinder installation direction is owned by "smuser" and the group is "smgrp".  We use ReflectionX as our XWindows client and this software is configured to use my user ID to SSH to the server and then executes the Xwindow application using my user ID.  The servers in our lower environment was previously setup by our Unix admins and they went ahead and added both the "smuser" and my user ID to the "SMGRP" group but they forgot to do so for the PROD servers.

 

To resolve this I modified the "SMGRP" group and added my user ID that the ReflectionX client uses and I am able to launch the smconsole.

 

Once again, thank you for putting in the time to help me.

 

Regards.

[Ujwol]

Hi Duc,

 

Glad I was able to give you some pointer

 

Thank you for sharing your detailed analysis. I am sure the knowledge you shared will be be helpful for many others having the same issue in the future.

 

Regards,

Ujwol