ESXI - AMD Threadripper Testing - Having Issues.

[VinceB1]

Hi All,

I have been playing with a new Threadripper 1950X build and was thinking about seeing if I could get ESXi working. So far I haven't been having much luck and my build keeps getting stuck at the following place:

https://s4.postimg.org/3v2ua1t9p/20170821_002755.jpg

I have tried all kinds from disabling all sata ports, disabling SMT, disabling USB3 and generally messing about with stuff in the bios but as yet I simply can't get it past this loading screen. I realise that this chip is not on the supported chips list yet and that its a long shot but figured if anybody would know it would be the good people in here.

Any help you can give me would be very much appreciated. I'm also fairly noob so go easy on me lads.

Thanks

Vince

[TheBobkin]

Hello Vince,

Welcome to Communities, don't worry we will go easy on you - fairly friendly crowd here!

It is likely not actually hanging on vmw_ahci driver but whatever is in the boot-order after that.

You can switch to the boot logging with ALT+F11/12 which *should* give better information as to what it is actually doing when it is stalling at this point.

Can you try this and upload/attach a screenshot?

Also, if you have not tried this with a different install (build, version or ISO source), then this may also be an option worth exploring.

Bob

[VinceB1]

Hi Bobkin,

Thanks for the suggestions. I can't seem to actually get into any menu with any key combination you mention, I can get into a cli menu before it kicks off by hitting tab and it shows, "mboot.c32 -c boot.cfg" but anything past the point of "shift-0" which interrupts the boot to allow boot options everything is a no go. I did notice that once it gets past the loading user section that the lights on the keyboard and mouse go out so when it hangs they are unresponsive.

So far I have tried 5.5 which gives a purple cpu error. Presumably this was the same error people using the Ryzen 7 were getting when hitting 15 core limits. After that I have downloaded two ISO's the file name/build details are as follows: VMware-VMvisor-Installer-6.5.0.update01-5969303.x86_64.iso. On top of this I tried the "Driver Rollup Iso" : VMware-ESXi-6.5U1-RollupISO.iso - Again this fella hangs at the same point.

I decided to have a dig into the boot cfg and its quite interesting:

It looks like the modules outline somewhat the boot order and if I have a search for what we hang on it looks like the next thing inline is:

So I guess this is the next place to start looking. I am sure I disabled USB3 but perhaps I didn't do it properly the motherboard has an absolute plethora of options but also has light blue USB slots alongside the standard dark blue USB3 ones which claim to be usb 3.1. Just to add the motherboard I am using the the AsRock X399 Taichi. Time to have a dig in the motherboards manual I think.

[VinceB1]

Not sure what I did but for one time only I managed to get past this point last night, right to the point where I even got to install ESXI (and it inevitably fell over at about 70% because id murdered the install media)- Problem is by that time I had changed so much, including the ISO, bios options etc that I am not really sure how it got to where it did or which one of the actions I performed got me to that point. With this in mind I have decided to burn the media to CD's rather than the USB install that I was trying. I think I will try again this evening and see where we end up but this time I am going for a more structured approach, or at least that is how it will start no doubt until the frustration kicks in again.

[VinceB1]

Morning Ladies and Gents,

I thought that seen as I opened a thread here that it was time to come back and give you a little update. I've been working on this now for a few days and last night, late into the evening we had a breakthrough. After days of enabling and disabling bios settings and trying again I decided to take a new approach. In workstation I created a vm and enabled EFI boot, I then exposed the usb stick I was using to install ESXi to this VM and fired up ESXi in workstation. To my surprise it finished the install and I could then interact with ESXi so I fired up SSH and through putty was able to send some commands via command line, below is a screenshot of the two commands that I sent via the console to the ESXi Hypervisor:

We knew that it was stopping on either AHCI or USB3 and seen as VMWARE recently updated these drives I decided to disable them both which to my understanding drops the driver version back to a legacy one. After this I simply rebooted and fired up the bare metal from the USB, at this point it was 2am and if it worked or failed I was heading to bed either way but to my surprise it fired straight up, sailed through all the previous issues and before I knew it I had a working Threadripper Hypervisor.

Right now I am pleased as punch so tonight all I need to do is expose my NAS and do some other messing around with drive configs and ill have a fully working lab. Happy days indeed!

[Mcanela1980]

Hi Vince...

Very interested on your results

I wanna also invest in a esxi home server over a 1920x

Epyc  still far far away as a choice

  so will very nice if you had more updates

[VinceB1]

Hi Mate,

I do keep checking back here for questions but haven't done much in the way of keeping a good log of the issues etc here. If you are interested I have been keeping another build log here:

https://forums.overclockers.co.uk/threads/home-lab-threadripper-build-thread.18789497/

This has much more info on the issues, troubleshooting and other random musings related to my threadripper build and esxi.

[Psyllon]

I am getting this error while trying to install ESXi v6.5 on my machine with a Threadripper 1950X CPU. My motherboard is Gigabyte Aorus X399.

Could you be more specific and tell me what options do I need to disable to be able to complete the installation? Do I have to disable the USB3 support or AHCI?

Thank you!

[VinceB1]

Hi Lads,

Forgot to check back here once the majority of my issues were ironed out. I will update more when I am at home this evening in front of the machine.

Vince

[VinceB1]

Can you give me the full file name for the iso you are using?

[VinceB1]

I have made progress with this so thought I better post a little update here. I ended up replacing the board with a Asus X399-A Prime, the results:

https://s1.postimg.org/76cpy0ime7/Untitled4.png

https://s1.postimg.org/1286hec81b/Untitled.png

https://s1.postimg.org/1286hedq1r/Untitled2.png

https://s1.postimg.org/3pxmrr6jnz/Untitled3.png

ESXi 6.5 update 1 installed without issue and I now have access to all of my sata etc. I am not giving up on the Taichi and will build another machine around that at some point to continue working with it as I still feel that it offers a better feature set than the Prime I have replaced it with. Hopefully ASRock and VMWare come through with some kind of fix to make the Taichi viable.

There are a load of pics of the boards side by side and more of my random musings about this in the other thread mentioned above.

[kensiko]

Hello Vince, thank you for your hard work. We are looking for a test build also and we wanted to get the Asrock motherboard. However it seems much easier to make the Asus Prime one works. What are the disadvantages of the prime ? I read that posts are long and some issues with AD error code. Is the Asrock one faster to boot ? I know the differences in features I just wanted to know the practical differences.

I'll read the forum to know how you made the Taichi work, or did you just simply disabled the onboard SATA ? It doesn't matter for us because we plan on using an external RAID card.

[Memnarch]

ASRock professional x399  sailed through esxi 6.7 install, and got GPU passthrough working after some effort, but I'm having problems with USB passthrough hanging the system on reboot.  Details in my separate post but thought I'd point it out here in case people are looking.

[johnhc]

zimou13, please post on the ASRock forum and/or the AMD Processor Forum.  I post on the AMD forum and may be able to help with your memory problem.  Many people have had problems trying to get 128 GB working on ASRock X399 MBs but many are solved.  Please make sure you are not using an XMP and have the very latest UEFI (2.00, not the Beta).  We definitely need to know your complete specifications.  Here are mine as an example:

ASRock Fatal1ty X399 Pro Gaming, Threadripper 1950X, 2xSamsung SSD 960 EVO RAID, 1TB &

500 GB WD Black, G.SKILL [Flare X (for AMD)] F4-3200C14Q-32GFX, Windows 10 x64 Pro,

Enermx Platimax 850, Enermx Liqtech TR4 CPU Cooler, Radeon RX580, BIOS 2.0

Enjoy, John.