If your Linux host reports btrfs checksum errors that make no sense, naming files
that turn out to be fine, always with corrected=0, you run VMware Workstation,
and you have a /tmp directory that is a btrfs subvolume on root, what this post describes
is very likely the cause. It took me ten months to find. Workaround first, so
it is useful to you immediately.
== WORKAROUND (works today, costs nothing) ==
Add to /etc/vmware/config:
tmpDirectory = "/dev/shm"
Restart your VMs. Verified on 26.0.0.25388281: the affected mappings move entirely
off the filesystem onto tmpfs. Nothing of them remains under /tmp.
IMPORTANT: this only works in the GLOBAL config. Setting tmpDirectory in an
individual VM's .vmx has no effect at all - I tested that first and it was silently
ignored. If a support article ever tells you to set it per-VM, it will not work.
You are not losing anything. The file in question is transient VMM state that
Workstation unlinks the moment it creates it and discards when the VM exits - see
below. It is 16-140 MB of tmpfs per VM.
== WHAT IS ACTUALLY HAPPENING ==
Workstation maps /tmp/vmware-<user>/ramNNN writable MAP_SHARED (~128 separate
mappings, 16-140 MB depending on the VM). vmmon then acquires those pages with:
vmmon-only/linux/hostif.c:1268
retval = get_user_pages_fast((unsigned long)uvAddr, numPages, 0, ppages);
^ gup_flags = 0
With flags of 0 that takes a plain reference (FOLL_GET). It does not pin (FOLL_PIN),
and does not declare FOLL_WRITE or FOLL_LONGTERM - none of which matches what vmmon
then does with the memory. The guest writes those pages through EPT, which never
consults the host page tables, while the filesystem is writing them out.
So btrfs computes a checksum for a page the guest is still modifying. The bytes that
reach the disk are newer than the checksum recorded for them, and every later scrub
reports corruption in a file that is perfectly healthy.
You can see the mismatch on any running host:
$ grep VmPin /proc/$(pgrep -x vmware-vmx | head -1)/status
VmPin: 0 kB
Zero pinned pages, while vmmon is actively holding those mappings for guest DMA.
The kernel has not been told about any of it, so folio_maybe_dma_pinned() returns
false and the protections that exist for exactly this case never engage.
Notably the calling function is HostIF_LockPage(), and its output parameter is
commented "OUT: pinned page". The intent is documented in the code's own vocabulary;
the call does not implement it.
Guest main memory is NOT affected - it is backed by a shmem segment under /dev/shm
and never touches a checksumming filesystem. Workstation already does the right
thing for the large, obvious allocation. The defect is confined to this smaller
auxiliary region that happens to have been placed on the user's filesystem.
== WHY THIS IS SO HARD TO DIAGNOSE (two traps) ==
1. The file named in the scrub error is the WRONG file. Current btrfs scrub reports
the base address of the 64 KiB stripe rather than the failing sector, so the
backref walk names whatever file owns the first sector of that stripe. The tell:
every reported address is exactly 64 KiB aligned. Over ten months this host
logged 81 distinct addresses and every single one was. I have a fix submitted
upstream, but no shipping kernel has it yet:
Without it, treat the reported address as the start of a 64 KiB window and
resolve all sixteen sectors.
2. The file has no name. Workstation unlinks ramNNN immediately after creating it,
so it has zero directory entries while in use. A backref walk recovers the inode
number but there is no path to print. Match the inode against the VM's open
descriptors (/proc/<pid>/fd) instead.
Between the two, the trail points confidently at an innocent file and then goes
cold. That is where the ten months went.
== THE ACTUAL FIX (for VMware techs) ==
The region is created, opened, and unlinked immediately, and is mapped by exactly
one process. It is already anonymous shared memory - the file is just the mechanism
for obtaining it, using the pre-2014 idiom from before memfd_create() existed.
(memfd_create appears nowhere in the vmware-vmx binary.)
So either:
1. Use pin_user_pages_fast() with FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE, paired
with an mmu_notifier as Documentation/core-api/pin_user_pages.rst CASE 3
requires, and release with unpin_user_pages_dirty_lock(). Note the kernel may
then legitimately refuse the pin on a dirty-tracked file mapping - which is the
correct outcome, and points at option 2.
2. Back the region with memfd_create() or a /dev/shm segment, exactly as guest
main memory already is. Nothing is lost: there is no name to preserve, no
cross-process contract, and no persistence requirement.
3. At minimum, document that tmpDirectory already solves this, and that it is
honoured only in the global config.
The kernel has two independent mechanisms that would have caught this - the -EFAULT
refusal in writable_file_mapping_allowed() (mm/gup.c) and the reclaim check in
shrink_folio_list() (mm/vmscan.c), whose comment describes this exact hazard. Both
key on FOLL_PIN, so both are bypassed. Declaring what vmmon actually does would
either fix the behaviour or fail loudly at VM power-on. The current flags value
produces neither.
== ENVIRONMENT ==
VMware Workstation Pro 26.0.0 build 25388281 (26H1)
openSUSE Leap 16.0, kernel 7.0.11, /tmp on a btrfs subvolume
Reproduced on two unrelated hosts (NVMe software RAID, and a 3ware hardware RAID LU),
which rules out the storage stack.
Note most current distributions put /tmp on tmpfs, where none of this can happen.
openSUSE installations upgraded from 15.x keep /tmp as a btrfs subvolume, as does
any site that deliberately puts /tmp on real storage. The behaviour should not
depend on that.
I have a much longer write-up with the full evidence chain - kernel source
citations, per-mirror-member proof that the pages change mid-write, and the
measurements behind each claim. Happy to attach or post it if that is useful.