Server has restarted error message?

[Anon Anon]

Has anyone experienced this problem with SDM r12.6?  Analyst are actively working in a ticket and get a pop up 'Server has restarted' and lose the window they are actively working in.  This seems to be one of the unwanted features we inherited with r12.6.  I have work with support in the past and the issue was never fully resolved.  We seem to be getting more occurrances of this behavior,  My environment consist of a primary application and two web servers running on Windows 2008 R2. We are using CISCO load balancers for the volume and running a SQL database (Windows 2008 R2).

[SteveTroy]

My guess is that either the Domserver or Web Engine process is terminating unexpectedly.  Please open an issue with support so they can properly diagnose the reason why. 

[Jon_Israel]

This could also be true - if the domsrvr or webengine is crashing/dying and restarting on its own - this may also throw the same message, however, it is not as commonly seen as the session persistence cause. 

@Carla - if you run a pdm status the next time this occurs, check the start date/time for webengine and domsrvr on each of the servers.  See if it is more recent then the rest of the processes on the server.

Thanks,

Jon Israel

[Anon Anon]

Ok thanks Jon.  I am pretty sure I will have the option tomorrow.

[Anon Anon]

Jon

Wouldn’t the log file show if the domsrvr is crashing and restarting?

[Raghu Rudraraju]

Correct.

 

pdm_status,  stdlogs and/or windows application event viewer would show if something like that did crash.

 

_R

[Anon Anon]

thanks

[Jon_Israel]

Hi Carla,

This message is many times caused using a load balancer without having session persistence turned on or enabled.  What happens without session persistence sometimes is that when a user has an active session on one server, and then performs an action in service desk, the load balancer will try to bounce that request to another server, but that user's session does not exist there - so it will just throw a message about the server being restarted (as the session id doesnt exist on that server) and then will automatically re-login the user on the server if you have single-sign-on enabled.  With session persistence enabled, when a user has an active session on one server, that session will stay on that server until it is ended.  This way the load balancer wont bounce sessions to other servers and the problem will not occur.   Can you confirm that "Sticky Sessions" or "Session Persistence" is enabled?

Let me know,

Jon Israel

[Anon Anon]

I had a couple of folks to connect directly to the webservers and they still got the message.  They were using the direct url not hitting the load balancers.

[Raghu Rudraraju]

Do you have a Web Director  too or just a hardware load balancer?  

 

If someone connects to a secondary server directly  and continue to stick to the same, there's really no reason why an error like that should popup

1) unless there was a real crash of an SDM program  OR

2) the connection was still intercepted by a firewall (consider secondaries in DMZ) of some sort and closed the connection after a few seconds of idle time.

 

There maybe other scenarios though, where in a web page or something was coded in such a way that some entry still looks for your load balanced SDM alias  as opposed to just the local server based entries.  That could result in your request being sent to the load balancer behind the scenes again.

 

If you were to open a browser on the secondary server itself and point to the local SDM web server,  are you seeing a similar error? 

 

 

_R

[Anon Anon]

No Web Director

[Anon Anon]

I received this from the Network Engineer this morning.  Not sure if this is helpful.

 

 

Carla,

 

Here is the current setup in our Citrix Netscaler load balancer for ServiceDesk.  It may help the CA tech to know our settings.

 

 

 

VIP IP: 10.128.20.22/80

 

Primary Persistence: CookieInsert (120 minute timeout) Secondary Persistence: SourceIP (120 minute timeout) Server 1: 10.128.24.100 Server 2: 10.128.24.101 Client Idle Timeout: 3600 secs (default 120) Server Idle Timeout: 3600 secs (default 360) Client Source NATing is turned off (default is ON)

[Jon_Israel]

Hi Carla - thanks for providing that information for us.  In this case, I would say that having the session persistence set to 120 mins, only 2 hours, is a bit short for a session time out on a load balancer.  In many cases we have found that having the sessions stick to that server for at least 12-18 hours is best to prevent this type of issue.  The way this will work is that their session will at least stick to the same server for the entire work day and will expire sometime after the end of their day.  Then the next day when everyone logs in again, the load balancer will distribute the load evenly again among the servers.   Whats occurring now that is causing the problem here is that if the user is logged in and working, after 120 mins or two hours, the load balancer can bounce their session to another server when they click something in service desk - and thats what is causing the problem to occur, or at least a good part of the occurrences.   NOW, as you had mentioned earlier, you had users connect directly to a given secondary outside of the load balancer, and they had the problem as well.  That would be the piece that could actually be caused by a process on whatever server they are on (such as webengine or domsrvr) dying and restarting.  So for that side of it, its best to look at a pdm status output to see what the start date/time is on all processes and see if any of them were started more recently than the rest.

Let us know - hope this helps!

Thanks,

Jon I.

[Jon_Israel]

Carla, As we discussed on the phone today, the problem with the Server Restarted message is most definitely being caused by the load balancer settings, as you confirmed that the folks going directly to the secondary are not having that message but rather are having more performance related issues - such as hourglassing.  As David noted, it appears to be some database performance causing the product performance issues.   With that I will mark this discussion as resolved at this time. Please feel free to let us know if you have further problems related specifically to the Server Restarted message.

Thanks,

Jon I.