Esd2iso "Error while extracting tar file" when getting Windows 11 from Microsoft on Mac M2

[Olivia Dou]

Chip: Apple M2

MacOS: Sonoma 14.6

I first followed this guide  https://docs.vmware.com/en/VMware-Fusion/13/com.vmware.fusion.using.doc/GUID-8319563B-77F8-4F78-B5E2-69FA2DED5E59.html to install Windows 11, but got this error:

I see this screenshot in some other posts, but seems the root cause is not same as those. 

The error message in vmware-vmfusion.log is:

2024-10-22T05:25:47.061Z In(05) host-17109 LogSpawnArgs: Spawning Esd2iso:
2024-10-22T05:25:47.061Z In(05)+ host-17109 'getesdurl' '--edition' 'Professional' '--lang' 'en-us' 
2024-10-22T05:25:48.547Z In(05) VMware Fusion Esd2iso tool failed. ERROR: Error while extracting tar file: exit status 1
2024-10-22T05:25:48.549Z In(05) VMware Fusion OnExit: Esd2iso process is exited.

When trying to use command line version, when sending:

/Applications/VMware\ Fusion.app/Contents/Library/vmw_esd2iso getesd --edition "Professional" --lang "en-us"

response is: 

Edition : Professional

Language : en-us

bsdtar: Error opening archive: Unrecognized archive format

ERROR: Error while extracting tar file: exit status 1

ERROR: Failed to get ESD file.

Can someone help with this issue? Thanks.

[Technogeezer]

The "unrecognized archive format" message points to a couple of possibilities in my mind:

• The file is corrupt or not as expected on Microsoft's side. Try downloading again.
• If you're running a third-party anti-virus or firewall on your Mac, it's possible that it's interfering with the download (false positive?). I'd disable it temporarily while downloading Windows to prevent it from modifying the download.
• Do you have MacPorts or Homebrew installed? If so, can you verify the path of the bsdtar utility by dropping into a Terminal session and issuing which bsdtar . It should return /usr/bin/bsdtar

I know this isn't helpful, but I just was able to successfully download the ESD file from Microsoft using VMware's command line utility.

[Olivia Dou]

Thanks Technogeezer. Seems I did something wrong when posting this my first thread here, that I can only reply to myself.

• I dont' have third-party anti-virus or firewall installed on my Mac. 
• The "which bsdtar" command returns /usr/bin/bsdtar

By saying "try downloading again" are you referring to the windows iso file?

I tried to move the arm64, x86_x64 folders(under isoimages) to backup folder elsewhere, and click on "Get Windows from Microsoft" button again from GUI, it gives me exactly the same error message (Esd2iso tool failed to get ESD url and sha for Windows 11 Professional version for en-us language from Microsoft.), and the arm64, x86_64 folders don't get downloaded again.

list of files are as follows. Can anyone help confirm if the file size is correct? Thanks.

[Technogeezer]

What I meant was to re-run the "Get Windows from Microsoft" tool from the Fusion GUI.

Why did you try to move files out of the Fusion application folder? Those files are part of the Fusion installation and are not modified by the Windows 11 download process. They are needed for the installation of VMware Tools in a virtual machine, and also used by the "Get Windows from Microsoft" process -- either through the command line or the GUI.

If you moved files out of the /Applications/VMware Fusion.app/Contents/Library/isoimages folder, please put them back. 

From looking at my installed Fusion 13.6.1 application, the files that you have under /Applications/VMware Fusion.app/Contents/Library/isoimages/arm64 are correctly sized. 

The /Volumes/VMware Tools folder usually the Windows VMware Tools ISO mounted by whatever you did. You should find a disk "VMware Tools" as a mounted disk on your desktop. Unmount it by dragging it to the trash. 

What you should do (once you put the files back that you moved) is to re-run the download procedure.

The downloaded Windows 11 will be found in the Virtual Machines/VMWisoimages folder in your home folder. 

[Technogeezer]

The first thing that the Fusion tools do when downloading Windows from Microsoft is to download a catalog file from Microsoft that is used to determine the exact file name to download for the edition and language that you want. Let's run a test to make sure that the catalog file can be downloaded properly:

Open a Terminal window.

Execute the following commands:

curl -L "https://go.microsoft.com/fwlink?linkid=2156292" > test.cab

bsdtar tvf test.cab

The output should looks something like this

% curl -L "https://go.microsoft.com/fwlink?linkid=2156292" > test.cab

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100 29251  100 29251    0     0   111k      0 --:--:-- --:--:-- --:--:--  772k

% bsdtar tvf test.cab

-rw-rw-rw-  0 0      0     1743868 Oct  5 21:48 products.xml

If the bsdtar command doesn't list the products.xml file or complains about the test.cab file not being in the correct format, then there's something else going on outside the control of Fusion. Most likely a Microsoft issue -- or something on your network (perhaps a bad cached copy in a proxy gateway or a content firewall stripping out something)?

You can remove the test.cab file after you finish the experiment.

[Olivia Dou]

Hi Technogeezer, thanks again. Yes I've already put the files back and tried re-run the "Get Windows from Microsoft" tool from the Fusion GUI but failed. 

I also tried the test you suggested. It returns the expected result:

% curl -L "https://go.microsoft.com/fwlink?linkid=2156292" > test.cab

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0

100 29251  100 29251    0     0  13797      0  0:00:02  0:00:02 --:--:-- 13797

% bsdtar tvf test.cab

-rw-rw-rw-  0 0      0     1743868 Oct  5 21:48 products.xml

[Olivia Dou]

BTW, "/Volumes/VMware Tools" is from the result of the hdiutil command. I cannot move it to trash but can eject it. After running the hdiutil command again, it's mounted again to the desktop.

% hdiutil attach /Applications/VMware\ Fusion.app/Contents/Library/isoimages/arm64/windows.iso

/dev/disk8                                         /Volumes/VMware Tools

[Technogeezer]

The hdiutil command will mount the ISO - that's to be expected.

Are there any files in the VMWi\IsoImages subfolder found in the Virtual Machines folder in your user's home folder? If so, remove them.

Let's take this one step at a time

Open a Terminal window

Then run

cd ~/Virtual\ Machines.localized/VMWIsoImages

/Applications/VMware\ Fusion.app/Contents/Library/vmw_esd2iso getesdurl --edition Professional --lang en-us

That should return something like this:

% /Applications/VMware\ Fusion.app/Contents/Library/vmw_esd2iso getesdurl --edition Professional --lang en-us

ESDURL : http://dl.delivery.mp.microsoft.com/filestreamingservice/files/a0a4e466-0094-4903-9d46-6421eb4b1e83/26100.2033.241004-2336.ge_release_svc_refresh_CLIENTCONSUMER_RET_A64FRE_en-us.esd

SHA1 : 6e37f71fecf4a74e5c37cd49316c874f061a2fb6

SUCCESS

If that's OK, then run 

/Applications/VMware\ Fusion.app/Contents/Library/vmw_esd2iso getesd --edition Professional --lang en-us

If that succeeds, you should have a file with the .esd extension in the working directory.

If it's there, then mount the VMware tools ISO using

hdiutil attach/Applications/VMware\ Fusion.app/Contents/Library/isoimages/arm64/windows.iso

The VMware Tools disk should now be on your desktop.

Now create the ISO using the following command:

/Applications/VMware\ Fusion.app/Contents/Library/vmw_esd2iso generateiso --esd *.esd --drivers "/Volumes/VMware\ Tools/vmxnet3"

The ISO should build. At the end you'll find it in your current working directory (which should be ~/Virtual\ Machines.localized/VMWIsoImages).

You can dismount the VMware Tools ISO and delete the ESD file after the ISO is built

hdiutil detach /Volumes/VMware\ Tools 

rm *.esd

Post the output of these commands if something still is not working.

[Olivia Dou]

Hi Technogeezer, running the first command still gives me same error as before:

% cd ~/Virtual\ Machines.localized/VMWIsoImages

% /Applications/VMware\ Fusion.app/Contents/Library/vmw_esd2iso getesdurl --edition Professional --lang en-us

bsdtar: Error opening archive: Unrecognized archive format

ERROR: Error while extracting tar file: exit status 1

ERROR: Failed to get ESD url from Microsoft for Professional of language en-us

[Technogeezer]

This is very strange indeed. There's something else going on here that's not normal. It's going to take some digging to figure out exactly what it it. 

Can you post the output of the 'env' command in the Terminal? Would like to make sure the environment is similar to mine where I know things work.

Also, can you post the output of :

ls -alR ~/Virtual\ Machines.localized

[Olivia Dou]

Hi Technogeezer, thanks for your continuous helping! The outputs are as follows:

[Technogeezer]

I see you have Homebrew installed. I've had something similar to this occur. I had built a script that needed to download this catalog file and then untar the container it was in. With the default $PATH set to search the MacPorts folder first, my extract failed with the same error.  Let's make sure something like that isn't happening here.

Can you run the following in a Terminal session:

# Set the $PATH variable so that Homebrew isn't searched This export command is all one command, it should copy/paste into the Teminal

export PATH="/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Applications/VMware Fusion.app/Contents/Public"

# now re-run the vmw_esd2iso command and see if things are different

/Applications/VMware\ Fusion.app/Contents/Library/vmw_esd2iso getesdurl --edition Professional --lang en-us

[Olivia Dou]

Hi Technogeezer, I've executed the export command and now the $PATH has been set to what you listed. But executing the vmw_esd2iso command still gives me that same error.

[Technogeezer]

Can  you open Fusion and collect a support bundle Help > Collect Support Information > Quick, then attach that to a post here?

Also, have you tried dragging the Fusion application to the trash and then re-installing it from a download from the Broadcom software download site?

I'm really grasping at straws here -- perhaps someone from VMware that's listening can suggest a way to get to the bottom of what is causing vmw_esd2iso to fail. They don't provide any documentation for more detailed debugging output.

One other option: have you tried to tried to use CrystalFetch (available for free from the Mac App Store?). It would be very interesting to see if that can build an ISO for you.

[Olivia Dou]

Hi Technogeezer, the support bundle is attached here. 

And I've tried uninstall Fusion then re-install both using prevous download of 3.6.1/new download of 3.6.1/new download of 3.6, all give the same error as before. 

I successed with downloading iso from CrystalFetch, but using that iso to install requires a product key

[Technogeezer]

Good to hear that CrystalFetch worked. By the way,

Every "plain vanilla" Windows 11 ISO I've worked with asks the question about entering a product key during installation.   You should be able to click on "I don't have a product key" (smaller blue text at the bottom of that product key prompt screen" and continue. You'll need to inject the VMware network driver, though to complete the setup (there's information in the unofficial Companion Guide that can help you out with that).

Let me take a look at that support bundle and see if I can find anything.

[Technogeezer]

One other thought. Can you create another account on your Mac (without any customizations that you have in your normal working account), and then try the vmw_esd2iso getesd again? Trying to see if this is a system-wide issue or isolated to your user account.

[Technogeezer]

Also,

can you list the contents of /tmp and check for a file named w11catalog.cab or product.xml? Neither of these files should exist. If they do, can you post the output of:

ls -al /tmp/w11catalog.cab /tmp/product.xml

Can you check the permissions of the /tmp directory with the following command:

ls -ald /private/tmp

The permissions should be: drwxrwxrwt, owner root, group wheel similar to what you see here:

drwxrwxrwt  71 root  wheel  2272 Oct 25 14:28 /private/tmp

[Olivia Dou]

Thanks Technogeezer. I overlooked that link to continue. 

I checked your guide and now I've entered the Windows OOBE with “Choose your country or region” prompt. I opened the command prompt and it opens X:\sources by default. For "Mount the VMware Tools installer and execute the VMware Tools setup.exe utility manually", where shall I go for the VM tools folder here? I can see it from my Mac desktop, shall I do it from there?

File w11catalog.cab or product.xml don't exist on my side as well. And permission of /private/tmp is exactly the same as you provided.

I'll try for the new account later

[Technogeezer]

From what you're saying, I think you've typed Shift-F10 too soon. It sounds like you are at the first screen after boot from the Windows 11 ISO.

First, dismount the VMware Tools ISO from your Mac desktop. It doesn't need to be mounted.

Go back to the VM and close the command prompt window. Continue through this part of the installation process. This is where you select the language to install, select the Windows edition, agree to licensing terms, enter a product key (if you have one, otherwise click "I don't have a product key"), select the hard drive to install to (there should be only one - the VM's virtual hard drive), and copying Windows to the hard drive.

When that part of the process completes, the VM will reboot from the virtual hard drive. At that point the Windows out-of-box experience (OOBE) starts with a screen that says "Is this the right country or region". It's at that point that you type Shift-Fn-F10 to open a command prompt.

From there, go back to the Fusion menu bar and select Virtual Machine > Install VMware Tools. A dialog will then appear asking if you want to mount the Tools installer to the VM (you do). Wait a few seconds for the Tools installer ISO to be mounted to the VM.

Now return to the command prompt in the VM. The Tools installer will be mounted on the D drive. Use the following commands to manually start the Tools installer

d:\setup.exe

The Tools installation wizard should start (you may need to use Alt-Tab to see the window).

After Tools installation, the VM will reboot and you will be returned to the start of the OOBE process (the "Is this the right country or region" screen). Continue the setup of Windows from that point.

[Olivia Dou]

Thanks a lot for your detail instruction. I've now successfully installed windows on Fusion.

[vmomo]

I think the developers should consider changing the way to get Windows image. Esd2iso often goes on strike.

[Technogeezer]

If the built-in tool doesn’t work, you can now download a Windows 11 ARM ISO (starting with 24H2)  directly from Microsoft’s web site. The forum and the unofficial Companion Guide have links and instructions for using Microsoft’s ISO ro build a VM.