MacOSX High Sierra Destroys VMs at Root

[robertn]

For all the many years I've used VMware Fusion, I've kept my VM folder at the root of my internal drive.  This works for me, I'm the sole user of the machine.

The install/upgrade of High Sierra just evaporated that folder and its contents on two different MacBook Pros.  Thankfully, I always do a couple of TimeMachine backups before any upgrade like that and could readily restore them but beware of this nasty behavior of the MacOS installer.  I think this happened once before on an OSx upgrade a few years back also.  I'm guessing it's the versions that change the file system; in this case it became APFS on the SSD.

BTW, on another computer, a separate small folder of screensaver art files that sat in the root was left untouched.  Why did /vm go but /art was left intact?

[dempson]

Your /vm directory was lost because at least in some cases High Sierra creates its own /vm directory and puts a 1 GB swapfile0 in there. It still has /private/var/vm and that seems to be the one in active use (e.g. that location has the sleepimage file and more swapfiles) so I don't know why it also needs /vm, nor why /vm should be visible in Finder (that is probably an early version bug).

You got unlucky that your folder name happened to conflict with one now being used by the OS.

My /vm/swapfile0 is definitely not the same file as /private/var/vm/swapfile0 and it is timestamped a little earlier on the day I installed the High Sierra public release (on top of the GM candidate, so it wasn't a side effect of HFS+ to APFS conversion as that had already happened).

APFS does implement VM differently to HFS+: it has a separate "VM" hidden APFS volume inside the same APFS container as the startup volume. The VM volume is mounted at /private/var/vm so it looks like my /vm/swapfile0 is on the main volume. Perhaps it needs that one during boot before it can mount the VM volume? If so, putting it in the root directory seems a bad design choice.

[robertn]

Good to have some insight in what Apple has done to cause the issue.  The key point for other updaters is that the High Sierra installer process deletes or replaces a folder from the root of the drive without any interaction with the owner of that folder (me).  It doesn't leave it alone, it doesn't ask me if it can delete it, it doesn't rename it.  Apple probably won't bother to determine why they delete the files, but it just gives further credence to the standard to never do an upgrade without good backup.  Not only can the process go wrong, but the designers of the process can go blindly forward without concern for consequences.

Based on your info about APFS having some need for its own /vm folder, a simple renaming of my personal VM folder may be in order to stay out of their way.

[ColoradoMarmot]

Putting anything at root is asking for almost as much trouble as using Time Machine to backup VM's :-)

Seriously, / is not the place to store user files on OSX.  It's not UNIX, and Apple often clobbers things outside the user folders - which is the only supported location to store information.

Likewise, Time Machine is extremely risky for backing up virtual machines.  Because they're bundles, it's extremely easy to get pieces of the bundle backed up at different times and different states - especially while the VM is in use.  When purges happen, pieces of different ages can be deleted based on size and disk space needs.  Last, TM is notorious for silent corruption, particularly of very large files, bundles or images.  If you were able to restore, you got lucky - would strongly suggest adopting a different backup strategy.

[vmfh]

robertn, Did you check if it's been moved somewhere else?

dlhotka, It's actually Unix certified.

[uczmeg]

OK, so I'm the idiot who started the High Sierra upgrade without doing a full backup and have lost 200GB with of VMs that were in the root directory. You'd think I'd know better after all these years. :smileysad:

Is there any way to recover these?

It's not completely the end of the world, but there is a whole stack of work I'd rather not re-do!

[ColoradoMarmot]

Unfortunately, not unless you have a backup (other than time machine, which is unreliable).

[ColoradoMarmot]

Certified is not the same as UNIX.  Trying to treat OSX as if it is can cause major problems.  Great example is case sensitive file systems.  They're included for certification, but break a lot of things.

With apple, playing outside the sandbox is dangerous.

[vmfh]

dlhotka​

Seriously, / is not the place to store user files on OSX

Why not?

Certified is not the same as UNIX.

What's there more to being SUS-compliant?

[Michael Roy]

dlhotka

Seriously, / is not the place to store user files on OSX

Why not?

Because of permission issues like this.

Apple makes the assumption that / is for system-level resources only, and acts accordingly to the potential detriment of user-level files residing there.

This is why /Users and /Users/Shared exist.

[vmfh]

Mikero​, could you provide a link to documentation to corroborate that?

[Michael Roy]

Sure:

File System Basics

From the macOS section:

“In macOS, the file system is divided into multiple domains, which separate files and resources based on their intended usage. This separation provides simplicity for the user, who only needs to worry about a specific subset of files. Arranging files by domain also lets the system apply blanket access privileges to files in that domain, preventing unauthorized users from changing files intentionally or inadvertently.”

[vmfh]

The article mentions neither /vm under UNIX-specific directories , nor creation of directories outside of local, system, network and user domains. In any case, a basic check to avoid overwriting and move same name files/directories to another location is what High Sierra installer is missing, I think. But hey, if they missed anyone becoming root (bug), they surely forgot about that one too.

[ColoradoMarmot]

You have it backwards.  MacOS layers on top of UNIX, and imposes additional restrictions on what can and should be done.  Don't expect Apple to change their approach - the whole point of MacOS and OSX has been to hide the technical complexity.   Ironically their own advice yesterday to set a root password has historically caused long-term issues and instability of applications and the OS (and the only recourse was to reinstall the OS).

Every time I have this debate with folks, it comes back to this:  Playing outside Apple's sandbox always eventually results in pain.  Folks can ignore it and try to treat OSX as if it is UNIX, but if they do, they need to accept the risk.  User files go in /Users.   Put it another way, I wish I was younger, taller and thinner.  Thinner is in the sandbox and I can work on it - the others, well, denying reality is expensive and risky.