How to trigger NTP client to sync time?

[Horinius]

I have an internal NTP server in the LAN (so there's no need to configure any firewall for this matter) but my ESXi 4.1U1 servers are totally out of time (3 to 5 minutes).

I've tried to restart the NTP client n times but the time is still not synch'ed.

Of course, I could manually change the time but what's the point to have the NTP client if we have to do it manually?

Is there any command (through SSH) to trigger the time-sync?

TIA

[VTsukanov]

Have you opened necessary port at ESXi firewall?

Also take a look at vmware kb Troubleshooting NTP on ESX and ESXi

[Horinius]

What ESXi firewall?

I've followed that article to trouble-shoot the problem:

1) network connectivity is absolutely OK: ping to and from the ESXi have replies

2) NTP query utility gives a message similar to the example: no "request timed out" or other error

3) Reading ntpd log: I don't have any message similar to

      ntpd[263140]: synchronized to <ntp.server.ip.address>, stratum <X>

    This is all I have when the NTP client is restarted:

     Mar 30 09:38:10 root: ntpd Starting ntpd
     Mar 30 09:38:10 ntpd[20617]: ntpd 4.2.4p6@1.1495 Wed Sep 22 02:33:15 UTC 2010 (1)
     Mar 30 09:38:10 ntpd[20618]: precision = 1.000 usec
     Mar 30 09:38:10 ntpd[20618]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled
     Mar 30 09:38:10 ntpd[20618]: Listening on interface #1 lo0, 127.0.0.1#123 Enabled
     Mar 30 09:38:10 ntpd[20618]: Listening on interface #2 vmk0, 10.1.1.23#123 Enabled
     Mar 30 09:38:10 ntpd[20618]: kernel time sync status 2040
     Mar 30 09:38:10 ntpd[20618]: frequency initialized 0.000 PPM from /etc/ntp.drift
     Mar 30 09:38:10 ntpd[20618]: using drift file "/etc/ntp.drift" instead of "/etc/ntp.drift"

    (A question of curiosity about this part: I just want the NTP client inside ESXi to sync time to another NTP server.  But this part talks about ntpd inside ESXi which implies NTP server, not NTP client.  Are they the same?  Is this part relevant?)

4) Capturing network traffic:  I have the messages similar to the examples.

     10:43:42.555074 IP 10.1.1.23.123 > 10.1.1.1.123: NTPv4, Client, length 48
     10:43:42.555386 IP 10.1.1.1.123 > 10.1.1.23.123: NTPv3, Server, length 48

     10.1.1.23 is IP address of ESXi host while 10.1.1.1 is IP address of NTP server.

I don't know what went wrong....

[VTsukanov]

1) If you see the exchange between your esxi host and external ntp server over tcpdump-uw, then the problem is not in your firewall

2) You can check state ntpd service by command

/etc/init.d/ntpd status

or by VIClient - Host - Configuration - Time Configuration

3) Also you can use ntpq - standard NTP query program, usually useful output command

ntpq -pn

(+)

4) Check the contents of your files /etc/ntpd.conf & /etc/ntpd.drift

[Horinius]

ntpd service is started in ESXi, NTP server is working correctly (because other NTP clients are having the correct time) and it's not a problem of exchange between NTP client and server.  All these lead to one plausible conclusion: NTP client in ESXi server is damned broken!

All these don't answer directly to my question: how to trigger NTP client to sync time?

I've just checked one of my ESXi server and it's late by 4 minutes.  I've restarted 5, 6 times the damned service but nothing is changed!  And I have to correct the time by hand!  Useless thing.

[VTsukanov]

Post output command

watch "ntpq -p <esxi_ip_address>" 

?

[Horinius]

Like this

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
ntp-server.mydo 88.190.227.30    3 u 4971 1024    0    0.000    0.000   0.000

[VTsukanov]

reach 0 - means  failure in contacting the configured NTP server

Do you have other external ntp servers?

Post the output command "tail -f /var/log/messages" during the execution of commands "/etc/init.d/ntpd restart"

(+)

Compare the output "hwclock" with the correct time - if differences time more then 1000 sec, you should be set true time by hwclock

[Horinius]

failure in contacting the configured NTP server??

But in the article you suggested, it's written that

If you receive the message "No association ID's returned", the ESX/ESXi host cannot reach the configured NTP server. If you receive the message "***Request timed out", the ntpq command did not receive a response from the ESX/ESXi host's NTP daemon.

And I'm in neither of the mentioned cases above!

I've just done a test with my internal NTP server as well as its upstream external NTP server (fr.pool.ntp.org) but I did it in Win7.  This is how I proceeded:

1. I changed the clock to 4 minutes slower

2. I entered the internal NTP server

3. I pressed the [Update now] button to sync time and wait for a few seconds

I repeat the same steps with fr.pool.ntp.org in step #2.

And the result: both are working!

On the other hand, an hour ago I've changed ESXi server clock to 4 minutes slower and inputted fr.pool.ntp.org (the external NTP server as you suggested) as the ONLY NTP server.  I've restarted the service several times --> the time wasn't changed!  I've waited for an hour and the time is still wrong!

I think we have to admit it: NTP client is broken in ESXi 4.1U1.

PS: One more time: How to trigger NTP client to sync time?  Is the answer "No, VMware engineer forgot to implement this part and thus it's impossible"?

[Horinius]

I've just run the "watch ntpq...." command again (NTP client pointing to internal NTP server) and this time "reach" is 1, then 3, then 7.  BUT the clock is always wrong!

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
ntp-server.mydo 88.190.25.158    3 u 4971   64    7    0.218  267009.  10.029

[VTsukanov]

1) offset 267009 is very big (true value near 0), try  change it by set true time by hwclock & date  (Now ntpd looks as working)

2) > " ... forgot to implement this part and thus it's impossible"? :smileyhappy:

I do not know any other method except service ntpd restart (But this does not mean that this method does not exist :smileyhappy: )

[Horinius]

1) offset 267009 is very big (true value near 0), try  change it by set true time by hwclock & date  (Now ntpd looks as working)

If I understand it correctly, offset is the time difference, in millisecond, between the client and the server.

267009 / 1000 / 60 = 4.45

Don't forget I already said that I had changed ESXi 4 minutes slower. So the big offset seems correct to me.

2) > " ... forgot to implement this part and thus it's impossible"? :smileyhappy:

I do not know any other method except service ntpd restart (But this does not mean that this method does not exist :smileyhappy: )

I had done this as well many many many times but without any success.

Then few hours later, the clock is correct again.  I then repeated the same test the other day, but this time the clock wasn't readjusted afterwards.

I have the feeling that NTP client only synch's time once every n days (where n probably is larger than 5).  If that's the case, VMware has not totally implemented NTP client.

[msandrini]

Hi Horinius

We had your same problem and wanted exactly to know how to trigger off the time sync on the hosts. We are using a LAN internal server(out of domain) as NTP server and syncronizing this one with external ntp servers. Then pointed the hosts to use it as ntp server.

We were struggling because no time adjustment was happening on the hosts clock and, apprently, there is no way to kick it manually.

As last resort, we tried to put the internal ntp server on the same vlan as the hosts and updated the ntp server ip address in the Time settings.

It works.

The hosts looks to be syncronizing the time about once per hour. We tried to bring back the clock of the hosts and they seem to adjust within that amount of time

The point is... we still haven't answered your question.

Regards

Marco

[Horinius]

Hi Marco, thanks for your update.  For me, I don't see any way out, unfortunately.

My internal NTP server is a physical server and I cannot put it in the "vlan".  By "vlan", you mean the "vlan" inside ESXi?

When you tried to bring back the clock, how much time did you bring it back?  I tried 5 minutes and the clock was adjusted.

[msandrini]

Hi Horinius

As a vlan i mean the IP address of the NTP server is in the same class of the hosts.

Example:

NOT SAME VLAN

internal ntp server: 192.168.1.50 / 255.255.255.0

ESX host: 192.168.2.100 / 255.255.255.0

SAME VLAN

internal ntp server: 192.168.1.50 / 255.255.255.0

ESX host: 192.168.1.100 / 255.255.255.0

The ntp server can be either phisical or virtual.

i tried 5 minutes too and it worked.

[Horinius]

I see.  You are actually talking about different networks (some people would call them subnets, in reference to subnet mask), not vlan.