Is this happening when configuring the management interfaces in the DCUI? Do you know if the switchport(s) on the other end is an Access port or a Trunk Port?
I strongly suspect it is configured as an Access Port. In which case the switch is expecting "untagged" traffic, but you are sending it "tagged" traffic, which the switch doesn't recognize, and therefore a loss of connectivity occurs.
Assuming it IS an access port, and configured for the correct (native) VLAN on the switch end, then you don't need to set (ie tag) the VLAN on the ESXi end, just leave it blank.
HTH