Moving vmk0 from vDS to vSS fails

[StuDuncanHPE]

So, I'm trying to automate removing a host from vcenter, and need to move the vmks off the vDS to a vSS.  Started with William Lam's very good script here:
https://williamlam.com/2013/11/automate-reverse-migrating-from-vsphere.html
But it was failing.  So I broke out the migration of each vmk, one step at a time.  vmk1 & 2 migrate just fine. vmk0 migrates just fine in the gui. But fails every time via PowerCLI - network change, can't talk to host, rolled back change.

My version of William's script:

#####rollback to std switch so host can be removed

#add std switch

$vss = New-VirtualSwitch -VMHost $vmhost -Name "vSwitch0"

$vmnic0 = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmnic0" -Physical

$vmnic1 = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmnic1" -Physical

$pnic_array = @($vmnic0,$vmnic1)

#add std portgroups

$mgmt_pg = New-VirtualPortGroup -VirtualSwitch $vss -Name "Management Network"

$vmotion_pg = New-VirtualPortGroup -VirtualSwitch $vss -Name "vMotion"

$backup_pg = New-VirtualPortGroup -VirtualSwitch $vss -Name "Backup"

$pg_array = @($mgmt_pg,$vmotion_pg,$backup_pg)

#add vmk interfaces

$mgmt_vmk = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmk0"

$vmotion_vmk = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmk1"

$backup_vmk = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmk2"

$vmk_array = @($mgmt_vmk,$vmotion_vmk,$backup_vmk)

#move vmnic1 and vmk0,1,2 to std switch

# maybe not needed?? Remove-VDSwitchPhysicalNetworkAdapter -VMHostNetworkAdapter $vmnic1 -Confirm:$false

Add-VirtualSwitchPhysicalNetworkAdapter -VirtualSwitch $vss -VMHostPhysicalNic $vmnic1 -VMHostVirtualNic $backup_vmk -VirtualNicPortgroup $backup_pg -Confirm:$false

Add-VirtualSwitchPhysicalNetworkAdapter -VirtualSwitch $vss -VMHostPhysicalNic $vmnic1 -VMHostVirtualNic $vmotion_vmk -VirtualNicPortgroup $vmotion_pg -Confirm:$false

#this fails

Add-VirtualSwitchPhysicalNetworkAdapter -VirtualSwitch $vss -VMHostPhysicalNic $vmnic1 -VMHostVirtualNic $mgmt_vmk -VirtualNicPortgroup $mgmt_pg -Confirm:$false

#this fails

Add-VirtualSwitchPhysicalNetworkAdapter -VirtualSwitch $vss -VMHostPhysicalNic $pnic_array -VMHostVirtualNic $vmk_array -VirtualNicPortgroup $pg_array -Confirm:$false

#Remove VDSs

foreach ($vdswitch in $vdswitches) {

Remove-VDSwitchVMHost -VDSwitch $vdswitch -VMHost $vmHost -confirm:$false

}

remove-vmhost $fullhostname -Confirm:$false

Every time vmk0 is included in the migration, it fails.

So I then tried the network spec version, based on this discussion:
https://community.broadcom.com/vmware-cloud-foundation/discussion/migrate-vmkernel-from-dvs-to-standard-switch-and-vice-versa

$networkid = $VMHost.ExtenSionData.Configmanager.NetworkSystem

$interface = "vmk0"

$nic = New-Object VMware.Vim.HostVirtualNicSpec

$nic.portgroup = "Management Network"

$_this = Get-View -Id $networkid

$_this.UpdateVirtualNic($Interface, $nic)

And it fails exactly the same way - "An error occurred while communicating with the remote host. Network configuration change disconnected the host 'hostname.localdom.com' from vCenter server and has been rolled back"

Feels like I'm missing something simple, but I've been beating my head against it for 2 days now & can't see anything.

Appreciate any pointers where I might be missing something.

[StephenMoll]

Here is my guess:

The script is connected to vCenter.

Everything works except any commands involving the migration if the management kernel port.

If vCenter is executing and coordinating the task, then I can imagine that once the management kernel point tries to move vCenter loses contact and the task fails.

I think I would first try, connect-viserver to the host itself. The thinking here is that the task can maintain contact with the host, because it is running internally. Re-establishment of your web-client and PowerCLI sessions will be required, as I would imagine once the management kernel port has moved, it will effectively have a different identity.

Alternatively, create an additional management kernel port on a portgroup that won't move, connect to that, and move the original kernel port whilst it isn't in use.

[StuDuncanHPE]

Interesting thought.  I'll try it.  But every other example of doing this doesn't have to make that step.  Which is why I don't have it in my above scripts.

I wonder if this is how the gui handles it.

[StuDuncanHPE]

So this method didn't work. It caused the host to go completely offline & never come back.

[ChrisLeblanc]

I've done something similar but from VSS to VDS. 

My scripts are designed for automating a host build and to get rid of the VSS after install.

In our case, we have 2 nics that connect to the VSS.  I was getting the same problem as you.

What helped me was to add a start-sleep 30 second delay between moving the first nic and the second.

In testing, I was always able to ping the host doing it this way.

Might be something you want to integrate in your script if possible

[StuDuncanHPE]

So you're on the right track with what I'm doing.  This is for building out hosts using pxe.  No problem with building the vDS & moving everything to it - works fast and well. 

But this specific part is for undoing the builds.  Both for my testing of the build process, but also when things are decommed or moved.  We have ~2000 blades, so this isn't really a one-off thing either.  Annual refreshes mean annual decoms :-)