I tried again, got the partition warning, but after a couple of minutes it went away,
I'm removing the witness traffic from the interface, not the whole vmk:
esxcli vsan network ip remove -i vmk0
I will now post the output of the commands, that I captured before the removal.