VMware vSphere

 View Only

  • 1.  Get VCenter Host last reboot time

    Posted May 22, 2012 05:31 AM

    Hi,

    We are trying to find when a specific host was last rebooted?

    $vmhost.ExtensionData.Runtime.BootTime

    Is this the most reliable way to get this or:

    1. There could be sceanrios where this won't be set?
    2. Is the time returned by the above always the UTC time?
    3. Could this change depending on time zone settings on the host?


  • 2.  RE: Get VCenter Host last reboot time

    Posted May 22, 2012 05:44 AM

    ESX

    To determine why a ESX host was abruptly shut down or rebooted:
    1. If the host is currently turned off, turn the host back on.
    2. Ensure that there are no hardware lights that may indicate a hardware issue. For more information, engage the hardware vendor.
    3. Log into the host at the console as the root user.
    4. Type the command:

      # cat /var/log/vmksummary

    5. Determine if the ESX host was deliberately rebooted. When a user or script reboots a VMware ESX host, it generates a series events similar to:

      Mar 9 10:01:49 localhost logger: (1265803308) hb: vmk loaded, 1746.98, 1745.148, 0, 208167, 208167, 0, vmware-h-59580, sfcbd-7660, sfcbd-3524
      Mar 9 10:24:42 localhost vmkhalt: (1268148282) Rebooting system...
      Mar 9 10:26:13 localhost vmkhalt: (1268148374) Starting system...
      Mar 9 10:26:47 localhost logger: (1268148407) loaded VMkernel

      If your ESX host has been deliberately restarted, review the vCenter Server logs to identify any recent tasks that may have told the ESX host to reboot.The following are a list of other resources that help determine why an ESX host was rebooted:

    6. Determine if the VMware ESX host was deliberately shut down. When a user or script shuts down a VMware ESX host, it generates a series events similar to:

      Mar 9 10:01:49 localhost logger: (1265803308) hb: vmk loaded, 1746.98, 1745.148, 0, 208167, 208167, 0, vmware-h-59580, sfcbd-7660, sfcbd-3524
      Mar 9 10:42:34 localhost vmkhalt: (1268149354) Halting system...
      Mar 9 10:44:46 localhost vmkhalt: (1268149486) Starting system...
      Mar 9 10:45:40 localhost logger: (1268149540) loaded VMkernel

      If your VMware ESX host has been deliberately shut down, review the vCenter Server logs to identify any recent tasks that may have told the VMware ESX host to reboot. The following are a list of other resources that may help determine why an VMware ESX host was shut down:

      • For information on tracking user login and activities, see Tracking ESX host user logins and activities (1010026).
      • Third party products that reside in the service console or using the VMware vSphere API may are able to manipulate the functionality of the VMware ESX host. For more information about third party software in the service console, see Third Party Software in the Service console.
      • If a server hardware watchdog timer is enabled, it may automatically reboot the ESX host if it detects that the operating system is unresponsive. For more details about your server hardware watchdog timer, consult the applicable software documentation and support. For more information about Hewlett Packard's server hardware watchdog, see HP Automatic Server Recovery in a VMware ESX Environment (1010842) and if necessary engage Hewlett Packard documentation and support.
      • Sometime virtual power shutdown/restart usingan iLOon HP server can be the reason if the ESX server rebooted or shutdown.

    7. Determine if the ESX host experienced a kernel error. When an ESX host experiences a kernel error, it generates a series of events similar to:

      Sep 1 02:01:09 vsphere5 logger: (1251788469) hb: vmk loaded, 3597562.98, 3597450.113, 13, 164009, 164009, 356, vmware-h-79976, vpxa-54148, sfcbd-12600
      Sep 1 04:26:35 vsphere5 vmkhalt: (1251797195) Starting system...
      Sep 1 04:26:46 vsphere5 logger: (1251797206) VMkernel error
      Sep 1 04:27:41 vsphere5 logger: (1251797261) loaded VMkernel

      If your ESX host has experienced a kernel error, see Interpreting an ESX host purple diagnostic screen (1004250).

    8. Determine if the VMware ESX host hardware abruptly rebooted. When the VMware ESX host hardware abruptly reboots, it generates a series of events similar to:

      Mar 9 10:01:49 localhost logger: (1265803308) hb: vmk loaded, 1746.98, 1745.148, 0, 208167, 208167, 0, vmware-h-59580, sfcbd-7660, sfcbd-3524
      Mar 9 10:44:46 localhost vmkhalt: (1268149486) Starting system...
      Mar 9 10:45:40 localhost logger: (1268149540) loaded VMkernel

      If your VMware ESX host has experienced an outage and it was not the result of a kernel error, deliberate reboot, or shut down, then the physical hardware may have abruptly restarted on its own. Hardware has been known to reboot abruptly due to power outages, faulty components, and heating issues. To investigate further, engage the hardware vendor.

    9. Alternatively, the outage may have been deliberately triggered by an administrator by physically pressing the power button to turn off the hardware or using the hardware tools such as iLO/DRAC/RAS etc. This occurrence may generate this event in the /var/log/vmkernel log of the ESX host:

      VMKAcpi: 1865: In PowerButton Helper

    10. If your VMware ESX host experiences an outage that is not the result of a kernel error, deliberate reboot, or shut down, then the physical hardware may have abruptly restarted on its own. Hardware may reboot abruptly due to power outages, faulty components, and heating issues. To investigate further, engage the hardware vendor.

      Alternatively, if an administrator has physically turned off or restarted the physical hardware due to the fact that the console is not responding to user interaction, see Determining why an ESX/ESXi host does not respond to user interaction at the console (1017135).

    Notes:

    • This message is alsois logged when the server is powered down through the System Management Interface (such as HP iLO).
    • If the server is powered off by pressing the power button and the button is held for more than 10 seconds, this event is not logged.


    If an administrator has physically turned off or restarted the physical hardware because the console was not responding to user interaction, see Determining why an ESX/ESXi host does not respond to user interaction at the console (1017135).

    ESXi

    To determine why a VMware ESXi host was abruptly shut down or rebooted:

    Note: By default, VMware ESXi logs do not persist upon a reboot. If a VMware ESXi host experiences an abrupt reboot due to reasons other than a VMkernel error, the logs do not persist and you do not have access to the logs prior to the reboot to determine the cause. The steps in this section assume that the VMware ESXi host has been configured to re-direct the logs to a location where the logs persist. For more information on how to configure a VMware ESXi host to redirect the logs to an alternate location, see Configure Syslog on ESXi Hosts in the Basic Administration Guide for your version of ESXi.

    1. If the ESXi host is currently turned off, turn the host back on.
    2. Ensure that there are no hardware lights that may indicate a hardware issue. For more information, engage the hardware vendor.
    3. Determine where the logs are being redirected to:

      1. Open vSphere Client.
      2. Connect to the ESXi host or vCenter Server managing the ESXi host.
      3. Provide the credentials of an administrative user.
      4. Select the ESXi host in the Inventory.
      5. Click the Configuration tab.
      6. Click Advanced Settings.
      7. In the Advanced Settings dialog, verify the location where the log files are being redirected:

        Note: If either of the following settings are not properly configured, then logs do not persist upon a reboot and may limit the amount of information that can be gathered for troubleshooting.

        • Syslog > Local > Syslog.Local.DatastorePath contains the location of the logs if they have been redirected to a VMFS volume.
        • Syslog > Remote > Syslog.Remote.Hostname contains the IP address or hostname of the syslog server that houses the logs for this host.

    4. Navigate to the location of the log files, and based on the modified date of the files, open the log file using your preferred editor.
    5. Determine if the VMware ESXi host was deliberately restarted. If an ESXi host was restarted deliberately, the/var/log/messages file will contain events similar to the following:

      • Hostd: [2010-03-16 12:51:54.284 27D13B90 info 'TaskManager'] Task Created : haTask-ha-host-vim.HostSystem.reboot-50

        Or

      • DCUI: reboot
      If your VMware ESX host has been deliberately shut down, review the vCenter Server logs to identify any recent tasks that may have told the ESX host to restart.

    6. Determine if the ESXi host was deliberately shut down. If an ESXi server was shut down deliberately, it contains an event similar to:

      • Hostd: [2010-03-16 13:01:00.550 2FEDEB90 info 'TaskManager'] Task Created : haTask-ha-host-vim.HostSystem.shutdown-78

        Or

      • DCUI: poweroff
      If your VMware ESX host has been deliberately shut down, review the vCenter Server logs to identify any recent tasks that may have instructed the ESX host to power off.

    7. Verify whether the virtual machine or ESXi host has generated a core dump:
      1. Log into Tech Support mode. For more information, see Tech Support Mode for Emergency Support (1003677).
      2. ESXi hosts do not automatically collect the core dumps. To collect the core dump, manually run theesxcfg-dumppart command. For more information, see Manually regenerating core dump files in VMware ESX and ESXi (1002769).

        Note: Not configuring a core dump partition could interfere with the analysis of the abrupt reboots. For information on setting up a core dump partition, see Configuring an ESX/ESXi host to capture a VMkernel coredump from a purple diagnostic screen (1000328).

      3. If your VMware ESXi host has experienced a kernel error, see Interpreting an ESX host purple diagnostic screen (1004250).

    8. If your VMware ESXi host experiences an outage that is not the result of a kernel error, deliberate reboot, or shut down, then the physical hardware may have abruptly restarted on its own. Hardware may reboot abruptly due to power outages, faulty components, and heating issues. To investigate further, engage the hardware vendor.

      Alternatively, if an administrator has physically turned off or restarted the physical hardware due to the fact that the console is not responding to user interaction, see Determining why an ESX/ESXi host does not respond to user interaction at the console (1017135).

    9. The ESXi 5.0 log file /var/log/vmksummary.log contains information regarding ESXi host startup and shutdown, and an hourly heartbeat with uptime and other metrics. For more/related information, see Format of the ESXi 5.0 vmksummary log file (2004566).


  • 3.  RE: Get VCenter Host last reboot time

    Broadcom Employee
    Posted May 22, 2012 11:02 AM

    I would personally prefer to use PowerCLI.Check this thread http://communities.vmware.com/thread/314423