Resetting how ESXi discovers the vmnics

[JDMils_Interact]

We are using Cisco UCS servers where we can programmatically setup the vNICs on the hardware which would then be available as vmnics in ESXi.

We have a number of servers which were setup and are connected to vSwitch0 for management (2x UCS vNICs as uplinks) and the dvSwitch (another 2x UCS vNICs). Servers are ESXi 7.0u3g.

We have modified the vNIC placement in UCS Intersight so that all derived servers from this one template will have the vNICs in a specific order. Thus vNIC0 will be discovered by ESXi first and will become vmnic0, then vNIC1 will become vmnic1, etc. For new servers this is OK because when ESXi is installed, it will discover the vNICs and give them the correct and corresponding vmnics.

However for existing servers, where the vNICs have already been given vmnic labels by ESXi, after changing the Cisco vNIC order and re-applying it to the physical ESXi server, the vmnics are now connected to the wrong dvSwitches.

 

This is what an existing server's networking looks like:

Cisco vNIC Name	Cisco Placement order	ESXi vmnic Label	dvSwitch
vnic-cms-mgt-a	Auto	vmnic2	vSwitch0
vnic-cms-mgt-b	Auto	vmnic3	vSwitch0
vnic-cms-esxi-a	Auto	vmnic0	dvResource
vnic-cms-esxi-b	Auto	vmnic1	dvResource

 

When I changed the vNIC placement in Intersight to manual so that I could put specific vNICs with specific vmnics, I get this on the server:

Cisco vNIC Name	Cisco Placement order	ESXi vmnic Label	dvSwitch
vnic-cms-mgt-a	0	vmnic0	dvResource
vnic-cms-mgt-b	1	vmnic1	dvResource
vnic-cms-esxi-a	2	vmnic2	vSwitch0
vnic-cms-esxi-b	3	vmnic3	vSwitch0

 

One way to re-arrange the vmnics on an existing ESXi install is to re-install ESXi overwriting the local disk- bit much!

The other way is to manually re-arrange the vmnic labels (https://kb.vmware.com/s/article/2091560)- bit much!

 

My question is, can I instruct ESXi to do a rediscovery of the physical vNICs and based on the placement order of the vNICs, replace the vmnic labels with the correct ones.

[Nathan Savolskis]

OK, I see where you're going with this...but my first question is: is there a specific reason why you want to associate the Cisco vNIC with the ESXi vmNIC?  On the logical side, the networking from the ESXi (vmnic to dvswitch) remains, so unless there is a specific reason (say the Cisco vNIC mgt-a is a 1G and the esxi-a is a 10G....then you're really haggling over configuration documentation. 

Short answer is: unless you are continually doing this type of reconfiguration, I would either document the new change and move on, or run through the KB work.  You could re-discover networking through ESXi rescan, but you're introducting more network interruption to those resources.  Is that worth the effort?

That is where I would start - why do you need to have that specifically documented, and then how much interruption to ops are you willing to make in order to align with (assuming) older config documentation?

[JDMils_Interact]

There is a good reason to map the Cisco vNICs to the vmnics- NSX-T! With NSX-T, we have a Transport Node Profile which is configured so that specific host vmnics/uplinks are used for the NVDS distributed switch. In our case, we have configured vmnic2 & vmnic5 for the NVDS dvswitch so we need to keep these free in ESXi so that when hosts are added to the cluster, these vmnics are not automatically allocated to the vSwitch0. This is with hosts which have 6 uplinks- 2x management, 2x data & 2x NVDS/Geneve networks.

We are testing the new UCS-X infrastructure in our test environment where we have around 10 ESXi servers up & running which we initially manually set the vmnic labels on each vNIC discovered by ESXi.

I've managed to get the UCS-X vNIC Placement working really well such that vnic-nvds-a will be mapped to vmnic2 & vnic-nvds-b will be mapped to vmnic5. I now want to update each of my server profiles in UCS-X which will cause each server to present its vNICs in a different order and thus jumble up the vNIC to vmnic mappings currently setup. One way to fix this is to re-install ESXi overwriting the local disk.

I thus would like to re-apply the Cisco UCSX server profile with the new vNIC placement to the existing servers all running ESXi 7, and force ESXi on each one to re-map their vmnics to match the Cisco vNIC placements.