Someone added a virtual switch to ESXi - Vulnerability!!!

[EloHHH]

Hi,

We are running almost 30 ESXi nodes and today we saw a high CPU(100%) usage on a few of the servers and when we checked, there was a log for:

Network connectivity restored on virtual switch vSwitch0, portgroups: LAN_Network.10", "VM Network", "Management Network. Physical NIC vmnic0 is up

We do not have any clue about this, but there was a new vSwitch "LAN_Network.10" and new port "10.2.2.1" which was not created by us!

As soon as we removed this port, the CPU came back to normal! (sadly can not remove the vSwitch, it says "The resource 'LAN_Network.10' is in use.")

During this process, we also saw a warning that "SSH is enabled", but when we checked services, the SSH was Stopped! we had to start and stop the SSH service to make sure it is off completely!

 

Anyway, we afraid that there is a new vulnerability on ESXi!

Is there anyone else that experienced this lately?

 

 

 

[Duncan Epping]

I have not seen or heard about this, however, if someone or something created this then the logfiles must also show by what or whom it has been created. make sure to also check the events in vcenter server to see if anything shows up there. I would doubt it is a new vulnerability which is exposing this, it could be you are running an unpatched version, and chances are much bigger that someone ran a script against vcenter without realizing, or someone made a random mistake at some point. If there is a vulnerability being exploited right now then you have a bigger problem, as the creation of network ports would not be the primary target. The target would be the VMFS volumes or the disks of all the guests VMs, they would be looking to encrypt those.

So the first question to answer would be:

1) is your vcenter server of esxi hosts exposed to the internet in some shape or form?

2) who has access to your root / administrator accounts?

3) are you frequently changing the passwords of the above accounts? If no, and you are worried, change them now!

4) do you see any strange files on your datastores that do not belong there?

5) which version are of vSphere are you running? 

6) any VMs which you don't recognize?

I would recommend contacting a security consultant who has expertise in this space.

[Nathan Watts]

could be this: https://www.zdnet.com/google-amp/article/necro-python-bot-revamped-with-new-vmware-smb-exploits/ 

[Bmercier]

Hello,

Exactly same problem this morning.

ESX version : 6.7.0 Update 3 (Build 14320388)
ESX listening on public IP, ssh deactivated. No vCenter server (standalone, using web ui)

This morning, new vmkernel 10.2.2.1/24, new network "LAN_Network.10" and SSH activated.
CPU is OK (5/10%).

I found theses logs :

2021-06-03T06:47:16.031Z cpu23:2098610)MigrateNet: vm 2098610: 3263: Accepted connection from <::ffff:167.248.133.54>
2021-06-03T06:47:16.031Z cpu23:2098610)MigrateNet: vm 2098610: 3351: dataSocket 0x430c32cd8700 receive buffer size is 563272

-> scanner-09.ch1.censys-scanner.com.

 

Regards,

 

[Duncan Epping]

 was your host exposed to the internet in any shape or form?

 

[Bmercier]

Yep..
I know is bad idea, we are slowly moving to a private network.

[Duncan Epping]

Censys is a portscanner, so this is definitely not looking good if you ask me.

[Duncan Epping]

 I just spoke with the support team, 6.7u3 without patches indeed has a security warning against it, read about it here:

https://www.vmware.com/security/advisories/VMSA-2021-0002.html

few actions items recommended ASAP:

1. disconnect all hosts from the internet

2. change all passwords 

3. scan for ransomware

[Bmercier]

Thank you depping

In doubt, I reinstalled host with the last 7.0 and very last patch.

[WhateverNvm]

Anyone got an idea if the storage contents should be considered as compromised after this specific exploit found?

[simonwong1508]

Hello,

Check all datastores you may find a new VM folder created and Oracle Linux VM uploaded with Network Adapter 1 connected to LAN_Network.10.

SSH services also started, check and stop service.

[finist_si]

Hello
In my case there were no additional VMs. CPU was randomly skyrocketing, but no VMs were utilizing it, it was the host itself, so I assume that was sort of miner
After changing root password and restarting SSH service CPU got back to normal and what's more important, vmk201 disappeared immediately.
Also I noticed that the license was changed. The original one was free, the changed one was like full package (now I wonder if I can keep using it lol)
Some additional firewall rules were added.

[ajeje1989]

hi,

same problem ..

 

how to fix ?

i update the exsi with path

delete network 

changhe root password.

 

do you think that's enough or should I really review something?