How to change management network VLAN painlessly

[vitaprimo]

Some time ago I had a VLAN for each type of traffic/usergroup and the infrastructure was on VLAN0(1-untagged). Now everything (with a few exceptions for transit VLANs) is on VLAN0.

I want to move the infrastructure traffic to another VLAN with minimal downtime. From an overview to the network on vCenter it seems a pretty straightforward move: just go to Manage Host Networking under the dSwitch and reassign the VMkernel Network Adapter to another dPort.

However...

Just doing so would mean that I'd end up with the management IP on a competing VLAN away from the subnet while all of the servers are still on the main subnet-VLAN combination--I'd have the same subnet in two VLANs and that screams trouble. For starters, hypervisors would lose contact with domain controllers. I can counter this by drafting a custom hosts file and pushing it to the hosts. What would be more worrisome is that vCenter itself would be left on another VLAN unable to contact the hypervisors.

As we know, Distributed-anything is uneditable, unassignable, uneverything from ESXi for who-knows what reason and once the dSwitch is broken someway, it's like a day job putting it back together, going back and forth to the physical servers, resetting the network, having enough resources on the servers to take the load while one or more are in maintenance mode so dSwitch changes won't fail and so on. It escalates pretty quickly into a full-fledged nightmare. Anything of the fancy things done on vCenter are locked away in ESXi, even ESXi obviously knows about them.

Another way to go about it, I think, is to add temporary management-enabled VMkernels with temporary-subnet addresses then merge the old subnet into the new VLAN and finally just remove the temporary subnet from the new VLAN. But, that's as far as I got. I don't know if I'm missing steps that might trigger the nightmare. I am using networked storage, both NFS and iSCSI, but I just ordered a bunch of SSDs to go local (and maybe go vSAN after all is in place) and minimize the complexity and try to continue online as much as possible.

I do not want to edit the physical switches, specifically, changing switch management networks. It just seems too risky since I'm using different brands that have different names for the same thing. The main router that would allow me inter-VLAN communication is virtualized. I could do it on switch (L3) but it's not as featured.

Any suggestion/link/advise/criticism is welcome. Thanks !

[hussainbte]

what is the versions of ESXi hosts.

do you have multiple physical nics on the ESXi hosts.. may be one in spare to use in standard switch?

if it is not an issues I would suggest you to specify vLANs you currently ate and the vLANs you want to move to.. we dont want the discussion to get confusing.

[vitaprimo]

First, thanks for answering,

The main subnet is 10.0.0.0/22 on VLAN0. I don't have a VLAN in mind but I'm thinking about VLAN1000 to more-or-less have a resemblance to the subnet.

All hosts are on vSphere 6.7. One of them has a slightly different build number, older than the rest but nevertheless it's 6.7. Except for one, they all have multiple NICs, I have spare SFP+ NICs but it can't be modified; it's an [otherwise unused] iMac I'm using for its GPU. It's most sitting there idle though so I'm fine wiping its settings or removing it completely.

Are you by chance saying using a standard switch would be the best approach??

Thanks again!

[hussainbte]

Yes I am thinking about setting up a standard switch for a secondary Management network.

here is the plan I have in mind.

1) Disable HA and DRS in the cluster to avoid any issues.

2) Setup a standard switch on the ESXi hosts with the new management network. vLAN1000 and a new vmkernel port enabled with management traffic.

3) as we need to put our vCenter also on the same management network setup a VM port-group on the new standard switch for vLAN1000(you are OK to do this just for the host on which the vCenter is running on).

4) Once the setup is ready access the host directly and swap the vCEnter network from old vLAN to vLAN1000.

5) if you have everything configured with host-names using DNS this can actually be helpful. swap the IP in DNS server for the ESXi and vCenter hostname.

6) Once your vCenter is up with the new IP in new vLAN with all the services working fine, you are almost there.

7) Set the vCenter managed IP in vCenter to the new IP.

8) remove all hosts from vcenter one by one and add them again with the new IP..

9) if the migration goes smooth all you you have to do later ois discard the management from the old DVS switch and migrate the networking from standard to DVS

the only think I am concerned about is how these individual vCenter services work on changing the vCenter IP.

if you a vCenter appliance or a vCenter with all services running in a single machine well and good..

best of luck !!!

[NathanosBlightc]

As I understand from your description, I think you are very worried about losing vCenter and managing capability of dvSwitches. However, as the primary point of virtual infrastructure management, you should always keep the vCenter Server in the top-level of availability by any possible solution (HA, VCHA, FT, Replication and so on). Although you should note the management plane of VDS is controlled by vCenter but data plane still remains in the ESXi host.

Execution of a safe VMKernel migration depends on many factors such as:

1. The current configuration of exist VMkernel port

2. Number of uplinks for the switch

3. VLAN configuration on the physical switch side

I believe the safest way is as you mentioned: Add another VMKernel port for the host management on the VSS or VDS based on the virtual networking structure you need, then start working on this new design network (subnet/VLAN) and connect them to the vCenter server.  Whenever you are ensured about all aspects of connectivity, you can remove old VMkernel port, Anyway, you can migrate all of these phases (Uplink, VMkernel, VM Traffic) by running a single migration wizard if you have enough physical uplink to provide full physical redundancy for each dvPortGroup traffic