PowerEdge R740 ESXi Management Network Issues

[kzapater]

Hi all, this may have been added to the wrong board so apologies in advance...it's the first time using these forums!

I'm having an issue with one of our ESXi hosts, it's a PowerEdge R740. It's having an issue whereby you cannot ping the management address or access it, we've check the switch configuration and it's fine. All of the other hosts configured in the same way work without issues.

I'm starting to think it may be an issue with the Broadcom Adv. Dual 10GBASE-T Ethernet  (BCM57416) add-on card that's currently sitting in PCI-E Slot 2. I've tried upgrading/downgrading the firmware, the first time of upgrading it everything started to work however it started playing up once it was added to our vCenter.

After removing it everything stopped working again, I've tried a few downgrades/upgrades using the drivers from Dell's website. What's on is that the adapter is showing as connected with a link status of being up. There were no errors showing at the switch side, I've asked a colleague to bounce the ports which had no effect either.

I've re-installed ESXi from scratch and entered all of the correct settings but it still isn't working, unfortunately it's a server in a remote datacentre so wondered if anyone had any ideas before I have to make my way there and arrange a Dell engineer to replace the add-on card which I suspect has a faulty firmware module.

Thanks in advance.

[a_p_]

Moderator: Moved to ESXi Discussions

[E_Espinel]

Hello.
Do you have access to the IDRAC port of your server ?
The IDRAC is for remote access to your server, do you have it configured and have you worked with it ?

As it is a remote DC, when you are there you can directly connect a portable to the port where the ESXi management network is and perform a continuous ping as a connection test.

 

[kzapater]

Hi  

Thanks for your quick reply, I had forgotten to say that yes I do have access to the iDrac and in turn the remote console.

I've been using the remote console to run various esxcli network commands and pings to try and find out what the issue might be.

When looking at the NIC status I can see that the relevant ports are showing as connected, however they remain uncontactable.

I'm really beginning to think it's a fault in the add-on card firmware.

Thanks

[a_p_]

Some questions,and thoughts:

• which ESXi version, and build is installed?
• which firmware version is installed (and/or did you try yet)?
• how is the network - physical as well as virtual - configured (tagged/untagged/channels)? In case of a channel, try disabling one port on the physical switch?
• can you confirm that the correct network cables are in use (CAT5e, 6, or 7)? Maybe someone onsite can temporarily try other cable(s)?

André

[kzapater]

Hi  

Here you go:

which ESXi version, and build is installed? VMware ESXi 7.0.2 (VMKernel Release Build 17630552)
which firmware version is installed (and/or did you try yet)? Currently it is the latest at 21.80.16.95, I have tried about 4 other versions though.
how is the network - physical as well as virtual - configured (tagged/untagged/channels)? In case of a channel, try disabling one port on the physical switch? It's at a remote DataCentre, Our Network engineers have checked the ports and they're working, they even tried a shutdown/no shutdown
can you confirm that the correct network cables are in use (CAT5e, 6, or 7)? Maybe someone onsite can temporarily try other cable(s)? Yes, the correct cables are in use.

When I first upgraded the firmware I got it working, communication started working and it was pingable. However once added into our vCenter it started playing up when configuring additional networking components and had to be removed. After a configuration reset it all stopped working again.

Thanks

[a_p_]

Would it be an option for you to test the network connectivity by temporarily installing another OS on the server?

André

[kzapater]

 

It's the one thing I haven't tried yet...I'm in contact with Dell currently about a replacement part, I'll see what they say before I go ahead with a different OS.

Thank you

[kzapater]

So, I've just installed Server 2019 Std and configured the relevant adapter with a static address to match the VLAN it's connected to. I also ensured the adapter had the VLAN ID assigned in the hardware configuration.

It shows as enabled but unfortunately still doesn't ping it's default gateway, neither can I ping it's IP from my PC.

Definitely thinking there's a fault on the card's firmware!

[a_p_]

Just one more question, because you mention VLANs.
Can you confirm that you did not set a VLAN-ID on the virtual port group for the physical switch ports' native/default VLAN? The native/default VLAN must not be tagged on the virtual port group (or the OS side in case of Windows).

André