VMware vSphere

Hi - After upgrading my host to ESXi 6.5 and VCSA to 6.5 I have found that one of my VM's shows all memory being utilized in the summary screen and when checking the performance chart for percentage used it displays 100% (This is not the case when checking task manager in Windows)

Consequently the vSphere web client complains about memory usage for this VM.

I have 20+ other VM's on 2012 R2 that are not experiencing this problem. The only difference I can see is that this particular VM has passthrough enabled for the LSI card.

The VM has no reservation set and the limit is set to 'unlimited'. If I set a reservation to match the total memory size the problem still occurs.

The VM in question runs Server 2012 R2 with an LSI 9260-8i passthrough.

Latest VMware Tools is installed.

I have rebooted the VCSA, the host and the VM.

This issue was not present on ESXi 6.

Try to restart management agents on your ESXi server.

I have rebooted the host already.

I have the same problem but my was doing with 6.0 as well. Let me know if you find a solution.

Interesting that you had the same issue on 6.0. 

I'm guessing this is a bug with the alerts system for memory usage in vCenter.   Turning off the alert for that particular VM is the only workaround at present

I'm seeing something similar after upgrading to 6.5.   Memory active for a guest has gone up on a lot of my VM's (various O/S versions including Linux).   What is interesting is if I vMotion to another host it appears to drop back to its pre 6.5 levels but then starts to creep back up.

Did you ever find out the cause for yours?

See attached.  6.5 upgrade on 2/9.   Increases memory on 2/13 in an attempt to silence the alarm.   vMotioned on 2/21.

Brett

Someone might have already chimed in on this, but since we performed an in-place upgrade from ESXi 6.0 to 6.5, we started seeing VmMemoryUsageAlarm on quite a few VMs.  While we usually don't rush to upgrade VMware Tools on the VMs, I did see that a number of VMs with version 10249 (10.0.9) were those noted with the alarm, so I upgraded them to latest version 10272 (10.1.0) and saw a dramatic reduction in vRAM utilization.  This seemed most prominent in our older, lighter VMs with 2GB or less vRAM assigned.  Has anyone seen anything similar in their environment?

I've attached the chart for one VM that shows "normal utilization" up to our 6.5 upgrade (03/06), then usage spiked to about 95%.  I upgraded VMware Tools yesterday and usage dropped back down to "normal utilization".

We are in the same boat. We migrated to vSphere 6.5 by clean installing the vCenter server and then the ESXi hosts one by one.

Some VMs with lower allocated RAM immediately started spitting out errors in the web client regarding active memory, which went over 75 to 98% and oscillated. One particular VM, which has a PCI device passed through it (this in turn reserves all the RAM allocated to the VM from the start), has an active memory counter to 100% all the time. On vSphere 6.0 we've never seen this behavior. Only allocated and reserved to 100% but active memory was ok.

Issue is still present in VCSA 6.5b with latest ESXi 6.5 patch.

Screenshot is from the HTML client (flash client is the same)

I'm seeing the same problem with VMs that are @ tools version 10249 after upgrading to 6.5 on my Hosts. I will try to schedule a tools update on the VMs.

I'm seeing the same issue after an upgrade from 5.5. VM tools have been updated to 10272, but still the same. It's only happening on a few VMs.

Same issue as well.  vCenter was a brand new 6.5 appliance deployment.  Hosts were upgraded from 6 and then rejoined to the new vCenter cluster.

Random VM's showing high memory usage in vCenter but windows/Linux OS shows nothing out of the ordinary.

VMware tools at latest.  Host builds at 6.5.0 4882521 so not the latest.  vCenter also a few patches behind at 6.5.0.5300 5178943.   We have 2 vCenters with separate clusters using a shared PSC and we see the fault in both DC's on either vCenter.  I'd be more inclined to think this was a host issue like the one that existing back on Version 4 with incorrect memory usage alerts.

Same behavior here.

This occurs when memory is heavily used on a VM. For example, when there is a full scan of mcafee on a VM, the memory does not seem to be released correctly by the ESXi server afterwards. If I do not do anything, the memory usage by the ESXi will only increase. If I make a Vmotion of the VM, the whole thing becomes normal again.

Someone opened an incident? It's harder for me, I speak French.

I just came across this KB Article for everybody's viewing pleasure.

VM Memory Usage heuristic over-reporting on ESXi 6.5

Snippit from the KB Article so people don't have to click on the link..

VM Memory Usage heuristic over-reporting on ESXi 6.5

Thanks for the link to the article - it certainly has the hallmarks of the problem.

I can narrow the problem down to VM's configured for PCI Pass through - and reserving all memory.

At that point - the Active memory graph tracks the Granted memory and doesnt ever fall like it does on 6.0.

I wish the KB had highlighted PCi Passthrough, or rather 'Reserve all Guest Memory' in the 'more prevelent' list.

Also - the solution provided (unless I am missing something) is not as easily possible as suggested.

The rule which is triggered - is at the VC root level (by default) - and unless this is removed, and then individually assigned at child container levels, it can not be simply removed from the VM which is triggering it.

Please let me know if there is an easier way to remove the alarm defenition from a VM other than removing from the root parant - and subsiquently preventing all other VM's from triggering this alarm.

Rich

I am seeing the same thing with a 2008 R2 Server with LSI passthrough card for attached storage.

I am also seeing this same behavior on 2 of our VM's.  Both happen to be configured with a PCI/PCIe passthrough device.

No performance impact, but highly annyoning.

Keith

I'm having the same issue on a single VM that is running on a VMware ESXi, 6.5.0, 7388607 host.

This is the only VM with a PCI/PCIe passthrough device.

Physical PCI/PCIe device: 0000:07:00.0 | LSI Logic / Symbios Logic Dell 6Gbps SAS HBA Adapter

Yep - The VM in question has a LSI 9260-8i in passthrough mode.

Bug with LSI drivers and 6.5?

This is still present in VCSA 6.5u1g..

Any ETA on a fix from VMware barring disabling memory usage alerts?

good morning folks,

any news on this? seems to take place on vm with 100% memory reservation (all lock), as ours doesn't have a pci device passed through.

kind regards fritz