Feature request for Docker support

[Erik Martinez Sanchez]

It would be lovely if VMWare Fusion was able to run Docker inside a Win 11 ARM guest on a M1 host.

[Michael Roy]

This doesn't exactly make sense... Docker is a Linux-only thing. Docker on Windows uses VMs to run Linux. Docker Desktop hides the VM from the user.

You can run a Linux VM and put docker containers in it all day. Personally I use Photon for that because it comes with open-source docker.

Containers in Windows just confuses me. I don't understand the use-case.

[Erik Martinez Sanchez]

I am fairly sure that both Windows and Linux containers exist.

The reason for my wish is that Visual Studio has features for containerisation, for example .NET Aspire. It requires Docker Desktop, but Docker Desktop crashes:

deploying WSL2 distributions
ensuring main distro is deployed: deploying "docker-desktop": importing WSL distro "WSL2 is not supported with your current machine configuration.\r\nPlease enable the \"Virtual Machine Platform\" optional component and ensure virtualization is enabled in the BIOS.\r\nEnable \"Virtual Machine Platform\" by running: wsl.exe --install --no-distribution\r\nFor information please visit https://aka.ms/enablevirtualization\r\nError code: Wsl/Service/RegisterDistro/CreateVm/HCS/HCS_E_HYPERV_NOT_INSTALLED\r\n" output="docker-desktop": exit code: 4294967295: running WSL command wsl.exe C:\Windows\System32\wsl.exe --import docker-desktop <HOME>\AppData\Local\Docker\wsl\main C:\Program Files\Docker\Docker\resources\wsl\wsl-bootstrap.tar --version 2: WSL2 is not supported with your current machine configuration.
Please enable the "Virtual Machine Platform" optional component and ensure virtualization is enabled in the BIOS.
Enable "Virtual Machine Platform" by running: wsl.exe --install --no-distribution
For information please visit https://aka.ms/enablevirtualization
Error code: Wsl/Service/RegisterDistro/CreateVm/HCS/HCS_E_HYPERV_NOT_INSTALLED
: exit status 0xffffffff
checking if isocache exists: CreateFile \\wsl$\docker-desktop-data\isocache\: The network name cannot be found.

[AlessandroDN]

> You can run a Linux VM and put docker containers in it all day. Personally I use Photon for that because it comes with open-source docker.

Interesting!

@Michael Roy I used to use `vctl` but after the recent macOS upgrade it didn't stop working. Do you use Photon in a similar way?

[Michael Roy]

I use Photon to deploy a VM, ssh into it, and do everything from there.

For OP... Windows containers do not need a virtualization layer. Linux containers do. What you're seeing is Windows wanting to use WSL for Docker, but you can't install Docker in Windows for Arm because Apple's Hypervisor API does not support nested VMs (i.e. running a VM [WSL] in a VM)

[SvenGus]

BTW, in macOS 15 Sequoia, nested virtualization finally seems to be available for M3 and newer (see link below), in Apple's Virtualization framework: this new feature could be used also by VMware Fusion (and others), in future products. For example, with this feature enabled, one could indeed run WSL 2 in a Windows 11 ARM VM also on M3+ Macs, exactly as is possible today only on Intel Macs.

https://developer.apple.com/documentation/virtualization/vzgenericplatformconfiguration/4360553-isnestedvirtualizationsupported

[palter]

Unlike Parallels, UTM, and others, VMware Fusion doesn't use the Virtualization framework. It uses the underlying Hypervisor framework and there's no publicly available API for nested virtualization in the Hypervisor framework. 

[SvenGus]

Yes, had completely forgotten about that; and Apple's "lightweight" Virtualization framework - mainly for macOS 12+ and Linux guests - doesn't currently seem to support Windows guests: so, that sadly means still no nested virtualization for Windows guests on M3. Was too good to be true, eh…

[Technogeezer]

Would it be too much for VMware to talk to Apple to get nested virtualization capabilities when using the lower level hypervisor framework?

------------------------------
- Paul (technogeezer)
------------------------------

[SvenGus]

So, it looks liks we'll have to wait until also the Apple Hypervisor framework has nested virtualization built-in.

BTW, strange that third parties cannot enhance Apple's frameworks: for example, all the talk about system extensions that should have replaced kernel extensions… well, it doesn't look like this has happened (there are still few system extensions around), really. But maybe hypervisor add-on features aren't feasible with system extensions…? 

Anyway, let's hope that eventually support for nested virtualization will be added also for the Hypervisor framework: feature parity between Intel and ARM virtualization should be an important goal, IMHO…

[SvenGus]

Yes, that would be a good thing: whether done with evolved Apple APIs, or autonomously (at all possible, without kexts? don't know…). Nested virtualization for Windows ARM guests on Apple ARM hosts (Mx) is a must, for OS enthusiasts and also for IT teachers and programmers: we expect virtualization feature parity with Intel, especially now that M4s are coming. Let's hope that there's a future for all this…

[Technogeezer]

Apple's high level virtualization framework doesn't yet support Windows. Without access to those Apple nested virtualization APIs, Parallels will be in the same boat as Fusion for Windows ARM.

------------------------------
- Paul (technogeezer)
------------------------------

[SvenGus]

So, the Hypervisor framework doesn't yet support nested virtualization on Apple Silicon, while the Virtualization framework now does on M3+ but doesn't yet support Windows (still macOS and Linux only); and third parties seem to depend totally on Apple, for this: what a mess. Well, let's hope that in another year or two things will - but too slowly - evolve. It's regrettable that Apple and Microsoft don't seem to have interest in making Windows for ARM work better on Apple Silicon, not only for virtualization but also for a new Boot Camp (which would be technically possible, if only they collaborated)…

[Michael Roy]

Yah... Apple wants to sell Macs. Microsoft wants to sell Arm Surface books. They don't like each other very much. Users, vendors and app developers get stuck in the middle, and here we are.

If we were allowed to continue using our own KEXTs, we wouldn't have this problem at all.

But everyone wants to totally own and control their respective sandboxes, and don't seem to care about interoperability.

[Technogeezer]

Dissenting opinions here, Michael.

VMware can be included in this discussion about "everyone wanting to own their own sandboxes".  VMware wants to own their own hypervisor sandbox. At least on Apple, VMware made the business decision to avoid using the Apple high level frameworks and indicated that it is to promote their interoperability with vSphere. I'm sure that the situations and problems with Hyper-V on Windows platforms falls under the same umbrella. What vSphere giveth, vSphere also taketh away. There's a lot of functionality that users might like that is left on the table -- for example the use of de-facto standard virtio devices (such as audio devices that lately appear to be more stable than the VMware provided devices, btw), the ability to use Rosetta translation in ARM Linux VMs, and the ability to virtualize macOS on ARM.

It's unfortunately a new world trying to run a hypervisor as an application on a host OS. Both Apple and Microsoft do want to use their built-in hypervisors - and Linux seems to be discouraging the use of custom kernel modules well. My opinion:  

• third party drivers are expensive and time consuming to maintain. VMware had this problem with macOS constantly when Apple made changes to the operating system that broke Fusion, necessitating a new Fusion version for each new OS version.  Linux is in a similar boat - but VMware is guilty for not keeping up with changes to the Linux kernels given the number of problems and reliance on a third party web site to make Workstation work).
• kernel drivers represent a security problem for the host OS -  which is a problem that Apple and Microsoft are solving by de-emphasizing the use of third party virtualization drivers. 

------------------------------
- Paul (technogeezer)
------------------------------

[SvenGus]

Talking about sandboxes… the "Windows Sandbox" is another interesting feature which would require nested virtualization, in order to work also in a Windows 11 ARM VM on Apple Silicon. The easiest thing would probably be that everyone, if possible, uses the Apple Virtualization framework (which now has nested virtualization), which hopefully will evolve and add Windows support in the future…