Kernel panic -not syning: Fatal exception

[Avaakaford]

Hi Team,

I'm facing Kernel panic error on vmmark & it's cloned VM's on following hardware .

Hypervisor: VMware ESXi, 7.0.3, 20842708
Model: ProLiant DL365 Gen11
Processor Type: AMD EPYC 9354 32-Core Processor

same environment is working fine on Following hardware

Hypervisor: VMware ESXi, 7.0.3, 20842708
Model: ProLiant DL360 Gen11
Processor Type: Intel(R) Xeon(R) Gold 6430

Can you please suggest for resolution for this

 

Thanks & Regards

Ajay Vaaka

[James Zubb]

I have verified that VMmark3 VMs boot fine on our DL385 Gen 11 with EPYC 9354 using ESXi build 20842708.

Have you made any modifications to the ESXi install?

What version of the system ROM do you have on the host?  I am currently running v1.20.

[Avaakaford]

thanks James for responding to my request .

I'm running v1.30 ROM. I will try downgrade to v1.20 from my and see .

 

Thanks & Regards

Ajay Vaaka

[Avaakaford]

regards VMware image ,we are using HPE VMware-ESXi-7.0.3-20842708-HPE-703.0.0.11.2.0.9-Jan2023-depot with NetAppNasPlugin2.0.1 and our approved version of VMWare Tools with SPP 2023.02.00.00

[Avaakaford]

, which HPE SPP did you used ?

 

Thanks & Regards

Ajay Vaaka

[James Zubb]

I verified the VM boots OK with the April HPe custom image.  I do not have any Netapp storage, so I cannot test with that installed.

What version of tools are you trying to use?  Did you update the tools inside the template?

[Avaakaford]

VMware tools : 9.10.2.48224 ( build-2822639) is running on it . will update it now.

 

i tried BIOS to v1.2 , kernel issue is persistent

[James Zubb]

That is the default tools version which should work.

Are you running from netapp storage?  Can you move the VM to another storage to see if it still occurs?  Otherwise I can only think the host has some hardware issues such as a bad DIMM.  

[Avaakaford]

we use NFS storage, storage is not the issue here. I moved the same template to Intel hardware and hosted it on the same datastore. it is up and running.

Moved to local VMFS, still the issue is persistent.

I have two servers with AMD hardware, same issue on both. I don't see any DIMM issues as well. 

 

Thanks & Regards

Ajay Vaaka

 

 

[James Zubb]

Have you made any BIOS changes in the system?

I cannot reproduce this on our system, so I am not sure what other help I can give you.

[Avaakaford]

can you please share your receipe . i will reproduce on my end .

 

Here the BIOS settings i'm using

 

 

Attachment(s)

Copy of New-BIOS2.xlsx   14 KB 1 version

[James Zubb]

There is no real recipe.  I just installed the 7.0 HPe image on the system and booted the Vmmark3 template.  

Are other VMs booting OK on the system?  These are production CPUs (not beta or preproduction)?

[Avaakaford]

these are beta servers . WIndows vm's running fine on it

[James Zubb]

If these are beta CPUs I suggest contacting HPe to update the systems to production CPUs.  This kernel panic does not happen on any production Zen4 system I or our other partners have run on.

[Avaakaford]

all the components are retails SKUs. Can i get exact recipe that you are using , i will try same from my end .

 

HPE : SSP/SPP (Firmware )

ESXi OS : ESXi Image & source path

 

Thanks & Regards

Ajay vaaka

[David Morse]

Hi Ajay,

Earlier in the thread you mentioned "these are beta servers".  Can you clarify exactly which component(s) in the servers are beta?

Thanks,
David

[Avaakaford]

i meant to say test servers , server components are retails SKUs confirmed from my colleague

[David Morse]

---

all the components are retails SKUs. Can i get exact recipe that you are using , i will try same from my end .

HPE : SSP/SPP (Firmware )

ESXi OS : ESXi Image & source path

---

Ajay,

As mentioned above, there isn't a recipe per se.

HPE BIOS/System ROM: 1.20
ESXi OS: ESXi 7.0 U3 Apr 2023 

Also, please try using an unmodified VMmark3 template (in other words, do not install a specific version of VMware Tools), just to remove that as a potential variable.

Thanks,
David

[Avaakaford]

issue resolved after VM hardware version increased from 11 to 19

[fork_it]

We are experiencing the same issue with Linux VMs and appliances.

[flube]

hardware version?

[kassiopeia_G]

Are there any news here for that case, we see PODs on our latest systems too.

Hypervisor: VMware ESXi, 7.0.3, 22348816
Model: ProLiant DL385 Gen11
Processor Type: AMD EPYC 9354 32-Core Processor

[Avaakaford]

, my issue is resolved after VM hardware version upgraded from version 11 to 19 in my case. please try it