VMware Cloud Foundation Events

 View Only

 VCF 5.1 Bring Up Fails..

TryllZ's profile image
TryllZ posted May 26, 2025 07:21 AM

Hi,

This is my 1st time with VCF and am facing an error when during Bring Up in a Nested setup (I'm aware its not supported).

The networks are as below.

4 x ESXi - 192.168.35.0/24
vCenter, NSX, SDDC - 192.168.50.0/24 | vCenter - 192.168.50.5, NSX 192.168.50.6, SDDC - 192.168.50.110
vSAN - 192.168.40.0/24
vMorion - 192.168.45.0/24
NSX Overlay - 192.168.60.0/24

This is my understanding, and any one can correct me.

VCF deploys vCenter, NSX, and SDDC VMs on the 4 ESXi hosts, if this is correct then I'm facing an issue with the below error from the Cloud Builder logs.

The error shows the Cloud Builder trying to ping vCenter's IP address which fails, anyone knows why ?

The vSwitch of the Nested ESXi have their security set to Accept for all the 3.

2025-05-26T09:54:39.948+0000 [bringup,513cd7507dc14668,fa93] ERROR [c.v.e.s.c.u.c.SshCommandExecuter,bringup-exec-2] Download from vcf1.vlab.lab:/scratch/vsan-uuid.txt failed: No such file
2025-05-26T09:54:39.965+0000 [bringup,513cd7507dc14668,2bc8] ERROR [c.v.e.s.c.u.c.SshCommandExecuter,bringup-exec-3] Download from vcf2.vlab.lab:/scratch/vsan-uuid.txt failed: No such file
2025-05-26T09:54:39.968+0000 [bringup,513cd7507dc14668,57eb] ERROR [c.v.e.s.c.u.c.SshCommandExecuter,bringup-exec-4] Download from vcf3.vlab.lab:/scratch/vsan-uuid.txt failed: No such file
2025-05-26T09:54:39.978+0000 [bringup,513cd7507dc14668,dbb8] ERROR [c.v.e.s.c.u.c.SshCommandExecuter,bringup-exec-5] Download from vcf4.vlab.lab:/scratch/vsan-uuid.txt failed: No such file

2025-05-26T09:55:40.357+0000 [bringup,513cd7507dc14668,fa93] INFO  [c.v.e.s.c.h.i.HostPreparationServiceImpl,bringup-exec-2] vSAN UUID not found on vcf1.vlab.lab. Retriable operation 'Execute SSH command on ESXi vcf1.vlab.lab' failed to complete after 3 retries.
2025-05-26T09:55:40.357+0000 [bringup,513cd7507dc14668,fa93] INFO  [c.v.e.s.c.h.i.HostPreparationServiceImpl,bringup-exec-2] Preparing host {"address":"vcf1.vlab.lab","username":"svc-vcf-vcf1","password":"*****"} as a non-primary host with vsanUuid 7beec041-1524-44f3-9dc8-1078fa36ed45

2025-05-25T22:29:27.516+0000 [bringup,c4b7bbfc4f357b08,d071] ERROR [c.v.e.s.c.util.LocalProcessService,pool-3-thread-6] Local Command Failed with exit value 1.
Output Logs :LocalProcess Output: 2025-05-25 22:29:20 - PING vcsa.vlab.lab (192.168.50.105) 56(84) bytes of data.
LocalProcess Output: 2025-05-25 22:29:20 - From sddc.vlab.lab (192.168.50.110) icmp_seq=1 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:20 - From sddc.vlab.lab (192.168.50.110) icmp_seq=2 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:20 - From sddc.vlab.lab (192.168.50.110) icmp_seq=3 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:23 - From sddc.vlab.lab (192.168.50.110) icmp_seq=4 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:23 - From sddc.vlab.lab (192.168.50.110) icmp_seq=5 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:23 - From sddc.vlab.lab (192.168.50.110) icmp_seq=6 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:23 - From sddc.vlab.lab (192.168.50.110) icmp_seq=7 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:27 - From sddc.vlab.lab (192.168.50.110) icmp_seq=8 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:27 - From sddc.vlab.lab (192.168.50.110) icmp_seq=9 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:27 - From sddc.vlab.lab (192.168.50.110) icmp_seq=10 Destination Host Unreachable
LocalProcess Output: 2025-05-25 22:29:27 -
LocalProcess Output: 2025-05-25 22:29:27 - --- vcsa.vlab.lab ping statistics ---
LocalProcess Output: 2025-05-25 22:29:27 - 10 packets transmitted, 0 received, +10 errors, 100% packet loss, time 9220ms
LocalProcess Output: 2025-05-25 22:29:27 - pipe 4

Error Logs are stored at
2025-05-25T22:29:30.356+0000 [bringup,c4b7bbfc4f357b08,d071] ERROR [c.v.e.s.o.model.error.ErrorFactory,pool-3-thread-6] [D6KOJG] VCF_VSPHERE_VCENTER_INSTALLATION_FAILED vCenter installation failed. Check logs for more details.
com.vmware.vcf.common.util.VsphereRuntimeException: vCenter installation failed. Check logs for more details.

Thank You

TryllZ's profile image
TryllZ

The question here is Cloud Builder is deploying vCenter (which fails even before it starts) and tries pinging it which fails because it never deployed vCenter in the 1st place..

TryllZ's profile image
TryllZ

I can confirm Memory of the Hosts was the issue, bumped up the memory and changed VCSA deployment to Tiny, vCenter started deploying..

2025-05-27T20:14:11.937+0000 [bringup,4cc310702c2b94a5,0f92] DEBUG [c.v.v.c.util.CIContractInstaller,pool-4-thread-5] Prepare ESXi host for vSAN Bootstrap completed
2025-05-27T20:14:11.937+0000 [bringup,4cc310702c2b94a5,0f92] DEBUG [c.v.v.c.util.CIContractInstaller,pool-4-thread-5]  [SUCCEEDED] Successfully executed Task 'Running vSAN prepare: Prepare vSAN for
2025-05-27T20:14:11.937+0000 [bringup,4cc310702c2b94a5,0f92] DEBUG [c.v.v.c.util.CIContractInstaller,pool-4-thread-5] vCenter Server deployment.' in TaskFlow 'ci-conf-5495075726955567823' at
2025-05-27T20:14:11.937+0000 [bringup,4cc310702c2b94a5,0f92] DEBUG [c.v.v.c.util.CIContractInstaller,pool-4-thread-5] 08:14:11 PM
2025-05-27T20:14:12.156+0000 [bringup,4cc310702c2b94a5,0f92] DEBUG [c.v.v.c.util.CIContractInstaller,pool-4-thread-5] Deploying vCenter Server Appliance <-----
2025-05-27T20:14:12.156+0000 [bringup,4cc310702c2b94a5,0f92] DEBUG [c.v.v.c.util.CIContractInstaller,pool-4-thread-5]  [START] Start executing Task: Invoke OVF Tool to deploy VCSA for installation,
2025-05-27T20:14:12.156+0000 [bringup,4cc310702c2b94a5,0f92] DEBUG [c.v.v.c.util.CIContractInstaller,pool-4-thread-5] upgrade, and migration at 08:14:12 PM
2025-05-27T20:14:12.194+0000 [bringup,4cc310702c2b94a5,0f92] DEBUG [c.v.v.c.util.CIContractInstaller,pool-4-thread-5] Prepare to execute OVF Tool command

I can see vSAN Datastore in the host as well..

CDN media

MohammadHadi Milani's profile image
MohammadHadi Milani

I suggest checking the vCenter VM to ensure that the VCSA is up and running. If the VCSA VM is operational, it should respond to ping requests atleast, meaning its assigned IP must be active and routable (default gateway)

TryllZ's profile image
TryllZ

Thanks @MohammadHadi

I found the issue to be resource contention, reconfigured Nested ESXi and the deploymemt went ahead..