vCenter

 View Only
  • 1.  VirtualCenter service stops during SQL2005 cluster failover

    Posted Aug 07, 2009 08:24 PM

    I've noticed this on a few occasions, where the VirtualCenter server shuts down during a failover of the SQL2005 cluster that hosts the VC database. The following events are logged:

    Event Type: Error

    Event Source: VMware VirtualCenter Server

    Event Category: None

    Event ID: 1000

    Date: 8/7/2009

    Time: 1:15:30 PM

    User: N/A

    Computer:

    Description:

    The description for Event ID ( 1000 ) in Source ( VMware VirtualCenter Server ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Check database connectivity before restarting. Error: Error[VdbODBCError] (-1) "ODBC error: (42000) - [SQL Native Client][SQL Server]SHUTDOWN is in progress." is returned when executing SQL statement "UPDATE VPX_ENTITY SET NAME = ? , TYPE_ID = ? , PARENT_ID = ? WHERE ID = ?".

    ...and then:

    Event Type: Information

    Event Source: VMware VirtualCenter Server

    Event Category: None

    Event ID: 1000

    Date: 8/7/2009

    Time: 1:15:30 PM

    User: N/A

    Computer:

    Description:

    The description for Event ID ( 1000 ) in Source ( VMware VirtualCenter Server ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: Forcing shutdown of VMware VirtualCenter now.

    The service stays down until it is manually restarted. The service is configured to restart upon a failure, but if the service is being shutdown (technically not a failure, right?) then this does little to help. Ideas or suggestions would be greatly appreciated.

    Thanks



  • 2.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Aug 07, 2009 09:13 PM

    You may want to try these troubleshooting steps to see if it helps

    If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

    Regards,

    Stefan Nguyen

    VMware vExpert 2009

    iGeek Systems Inc.

    VMware, Citrix, Microsoft Consultant



  • 3.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Aug 09, 2009 02:33 PM

    With old version of VC 2.x a minimal timeout in SQL service will stop the entire VC service.

    Are you using VC 2.5U4 or U5?

    How does it take a SQL failover?

    Andre



  • 4.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Aug 10, 2009 01:50 PM

    I am using VC 2.5U4. I believe the SQL failover takes appoximately 30 seconds for the database to become available again on the new active node. Is there a way to increase the timeout value?



  • 5.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Aug 11, 2009 04:35 PM

    Is there a way to increase the timeout value?

    Sorry, no idea.

    Maybe there is some ODBC configuration, but in VC I do not know how to change the DB timeout value.

    Andre



  • 6.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Sep 01, 2009 01:35 AM

    I have the exact same problem. Any direction to solving this would be appreciated.



  • 7.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Oct 16, 2009 08:02 PM

    I finally contacted VMware Technical Support regarding this issue. I was told that the VirtualCenter application is shutting down the VPXD service after the connection to the database is lost. There is no way to modify the timeout value from VirtualCenter. Because the service is stopped, and is technically not a service failure or crash, the recovery actions that are configured for the service are not executed. In other words, if you have the service set to automatically restart after a failure or crash, a restart will not be attempted as the service is being shutdown by the application itself.

    Right now, I am exploring having SCOM 2007 monitor this service, and have a recovery action setup to attempt to restart it. Additionally, I will explore using a script to monitor the service or set up an event trigger.



  • 8.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Feb 09, 2010 10:50 PM

    I am testing a way around this by triggering a powershell script based on EventID 1000 from source VMWare VirtualCenter Server in the Event Log. When an event from vCenter with ID 1000 is logged, I fire off this powershell script that looks for the "ODBC error" and "SHUTDOWN is in progress" phrases, and restarts vpxd after 30 seconds if it finds them (this way Info event 1000's won't do anything).

    Here's the code if it's useful to anyone:

    -


    $logentry = Get-EventLog -LogName Application | Where {$_.EventId -eq "1000"} | Where {$_.Source -eq "VMWare VirtualCenter Server"} | Select -First 1

    if ($logentry.Message -match "ODBC error")

    {

    if ($logentry.Message -match "SHUTDOWN is in progress")

    {

    Start-Sleep -s 30

    Start-Service vpxd

    }

    }

    -




  • 9.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Apr 13, 2011 01:00 PM

    Exact same problem here.  I have searched high and low for a solution to this in VI3....since we are moving to vsphere 4.1, I am wondering if anyone knows if there are any DB timeout values configurable in that infrastructure?



  • 10.  RE: VirtualCenter service stops during SQL2005 cluster failover

    Posted Apr 15, 2011 05:33 PM

    Just wanted to add that we see the same type behavior (VC service initiates shutdown during remote SQL cluster fail-over) and have verified with VMware that there is no way to adjust the timeout value on VC (as of 4.0, U1).  Similar to other solutions mentioned in the post we have addressed the problem with an automated restart process integrated into VC monitoring/alert steps.