Unable to install vCD 10.5 in home lab.

[VirtualOz]

Hi VMware Community.

 

I am in the process of building a new test environment with vCloud Director 10.5.
The new lab environment is to evaluate and start the testing of NSX and vCD 10.5.x

This is a home lab environment.

But I have not been successful with installing vCD 10.5.

VMware ESXi, 8.0.1, Build 21813344
vCenter 8.0.1.00300
NSX 4.1.1.0

DVS PortGroups for vCD.
DS-Vlan-192.168.5.0 - vlan 5
DS-Vlan-192.168.6.0 - vlan 6

vlan 5 is routable.
vlan 6 is isolated to Cisco Catalyst 2960 and DVS.

 

Installing from the VMware_Cloud_Director-10.5.0.9946-22080476_OVF10.ovf
Note I have now tried VMware_Cloud_Director-10.5.1.10593-22821417_OVF10.ovf

Deploying the OVF completes without error.

eth0 - 192.168.5.25/24 - PortGroup DS-Vlan-192.168.5.0
eth1 - 192.168.6.25/24 - PortGroup DS-Vlan-192.168.6.0

 

DNS, A and PTR Record's configured and tested.

NTP: au.pool.ntp.org

NFS Transfer share configured on subnet 192.168.6.0/24
NFS build ubuntu-22.04.3.

 

root@ubuntu01:/# tail etc/exports
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)

/nfs/transfer 192.168.6.25(rw,sync,no_subtree_check,no_root_squash)

root@ubuntu01:/# exportfs
/nfs/transfer   192.168.6.25

 

Power on the vCD VM and proceed to <VCD Server on 192.168.5.25 >:5480/setup

Start the setup.
NFS is Green and passwords entered.

The install progresses to Invoking firstrunwizard.sh

 

Then fails.

 

Executing SystemSetup on this node ...
Setting ovf parameters ...
Invoking configure-vcd script ...
Appliance OS Phase has been Completed.
Starting Appliance Cloud Director Configuration Phase.
Invoking check-vcd-params script again.
All the required ovf parameters provided for the Cloud Director configuration phase
Invoking nfs-setup script.
Mounting NFS file share ...
Invoking setupvcd script.
setupvcd script failed to execute.
For more details, check /opt/vmware/var/log/vcd/configure-vcd.log.
An error occurred during configuration, running cleanup...
Starting cleanup script.
Attempting cleanup on primary appliance.
DROP DATABASE
DROP ROLE
returncode:23

 

Extract from /opt/vmware/var/log/vcd/configure-vcd.log

Database configuration complete.
Invoking firstrunwizard.sh now.
2024-01-04 10:16:43 | setupvcd.sh | Invoking firstrunwizard.sh now.
2024-01-04 10:16:45 | firstrunwizard.sh | Running the system-setup CMT command ...
Creating admin user.
Setting system details.
Completing system setup.
System setup is complete.
2024-01-04 10:17:44 | firstrunwizard.sh | Clearing sensitive OVF properties...
2024-01-04 10:17:45 | setupvcd.sh | Copying local responses.properties to NFS share.
2024-01-04 10:17:45 | setupvcd.sh | Done setting up vCD.
Reconfiguring vCD to use a secure database connection.
2024-01-04 10:17:45 | setupvcd.sh | Reconfiguring vCD to use a secure database connection.
Invalid database configuration provided. Aborting configuration of local cell.
2024-01-04 10:18:20 | setupvcd.sh | [ERROR] Failed to run reconfigure-database. See /opt/vmware/vcloud-director/logs/cell-management-tool.log for details. Aborting. <<<<<<<
2024-01-04 10:18:20 | configure-vcd.sh | [ERROR] setupvcd script failed to execute.
2024-01-04 10:18:20 | cleanup-vcd.sh | Starting cleanup script.
2024-01-04 10:18:20 | cleanup-vcd.sh | The following error code was provided as argument to script: 23
2024-01-04 10:18:20 | cleanup-vcd.sh | Attempting cleanup on primary appliance.
2024-01-04 10:18:20 | cleanup-vcd.sh | Executing configure_vcd_primary_cleanup.
2024-01-04 10:18:20 | cleanup-vcd.sh | Executing configure_vcd_base_cleanup.
2024-01-04 10:18:21 | cleanup-vcd.sh | Executing vcd_db_erase.
2024-01-04 10:18:21 | cleanup-vcd.sh | Deleting PostgreSQL database (vcloud) and user (vcloud).
2024-01-04 10:18:21 | cleanup-vcd.sh | Removing files from transfer share placed there by systemSetup, etc.
2024-01-04 10:18:21 | cleanup-vcd.sh | Unmounting NFS share and cleaning up /etc/fstab.
2024-01-04 10:18:21 | cleanup-vcd.sh | Executing clear_ovfenv_params.
configure-vcd-install

 

Extract from /opt/vmware/vcloud-director/logs/cell-management-tool.log

2024-01-04 10:18:20,167 | ERROR    | main                      | ReconfigureDatabaseCommand     | Invalid database configuration provided. Aborting configuration of local cell. |

2024-01-04 10:18:20,168 | DEBUG    | main                      | CellManagementTool             | Exiting with completion code: OTHER |

 

I found this KB for Ubuntu but did not resolve.
https://kb.vmware.com/s/article/94755

 

As the setup is running the /nfs/transfer is populated as expected.
On failure it is cleared uand unmounted.
NFS permissions are following the install guide.

During the setup I can see nfsstat connections of NFS v4.

I am trying to understand if this is a known problem installing vCD.
Or am I doing something wrong and cannot see it?
I have had a support case with VMware and discussed with colleagues with no resolution.
Its now driving me crazy.
Has to be something completely obvious and a fresh set of eyes will see it.

[Ashil Krishnan]

Hi  ,

1. Assuming this is nested and a small size deployment, please check if there are timeout errors reported in this log: /opt/vmware/var/log/vcd/appliance-sync.log.  Check if you notice these errors in the logs:

'Adding certificate for node'

'Timed out adding certificate for node'

2. Try increasing these timeout seconds in '/opt/vmware/appliance/bin/appliance-sync.sh' file. Look for the following config somewhere at the bottom of the file:

/usr/bin/timeout 10s $VCLOUD_HOME/bin/cell-management-tool import-trusted-certificates --source $APPLIANCE_SHARE/$i/vcd_ova.crt --force

Note:

1. This timeout config is specific to events/errors mentioned in step.1.

2. Make sure to copy/backup the file before modifying any of entries.

Hope that helps.

 

 

 

[VirtualOz]

Hi

Thanks for the reply.

The environment is not nested.
VMware ESXi, 8.0.1, 21813344
VSAN and NFS

I have increased the timeout to 30 seconds.
The instillation will then complete as normal.
Only the config is completed I did revert the timeout back to 10 seconds.
Not sure if this is correct.

But https://vcd.virtualoz.local/provider/ is just a blank grey screen.
I am prompted to accept the insecure certificate but never prompted for a login.

https://vcd.virtualoz.local:5480/ works as expected.

Not sure which log file would be the best to proceed.

[VirtualOz]

So after the config complete I see this in the cell.log.

 

root@vcd [ /opt/vmware/vcloud-director/logs ]# tail -f cell.log
com.vmware.vcloud.metrics-core Subsystem Status: [WAITING]
com.vmware.vcloud.multi-site-core Subsystem Status: [WAITING]
com.vmware.vcloud.multi-site-api Subsystem Status: [WAITING]
com.vmware.vcloud.rest-api-handlers Subsystem Status: [WAITING]

Application Initialization: 'Legacy Cell Application' 31% complete. Subsystem 'com.vmware.vcloud.vim-proxy' started
Application Initialization: 'Legacy Cell Application' 36% complete. Subsystem 'com.vmware.vcloud.fabric.foundation' started
Application Initialization: 'Legacy Cell Application' 40% complete. Subsystem 'com.vmware.vcloud.imagetransfer-server' started
Application Initialization: 'Networking Cell Application' 60% complete. Subsystem 'com.vmware.vcloud.fabric.net' started
Application Initialization: 'Legacy Cell Application' 45% complete. Subsystem 'com.vmware.vcloud.fabric.net' started
Application initialization detailed status report: 60% complete
com.vmware.vcloud-common-cell-impl Subsystem Status: [COMPLETE]
com.vmware.vcloud.common-util Subsystem Status: [COMPLETE]
com.vmware.vcloud.fabric.net Subsystem Status: [COMPLETE]
com.vmware.vcloud.backend-core Subsystem Status: [WAITING]
com.vmware.vcloud.networking-web Subsystem Status: [WAITING]

Application initialization detailed status report: 45% complete
com.vmware.vcloud.vcell-runtime-impl Subsystem Status: [COMPLETE]
com.vmware.vcloud-common-cell-impl Subsystem Status: [COMPLETE]
com.vmware.vcloud.common-util Subsystem Status: [COMPLETE]
com.vmware.vcloud.api-framework Subsystem Status: [COMPLETE]
com.vmware.vcloud.common-vmomi Subsystem Status: [COMPLETE]
com.vmware.pbm.placementengine Subsystem Status: [COMPLETE]
com.vmware.vcloud.vim-proxy Subsystem Status: [COMPLETE]
com.vmware.vcloud.fabric.foundation Subsystem Status: [COMPLETE]
com.vmware.vcloud.imagetransfer-server Subsystem Status: [COMPLETE]
com.vmware.vcloud.fabric.storage Subsystem Status: [WAITING]
com.vmware.vcloud.fabric.compute Subsystem Status: [WAITING]
com.vmware.vcloud.fabric.net Subsystem Status: [COMPLETE]
com.vmware.vcloud.backend-core Subsystem Status: [WAITING]
com.vmware.vcloud.faas-behaviors-aws-lambda Subsystem Status: [WAITING]
com.vmware.vcloud.vapp-lifecycle Subsystem Status: [WAITING]
com.vmware.vcloud.content-library Subsystem Status: [WAITING]
com.vmware.vcloud.presentation-api-impl Subsystem Status: [WAITING]
com.vmware.vcloud.statsfeeder-application Subsystem Status: [WAITING]
com.vmware.vcloud.metrics-core Subsystem Status: [WAITING]
com.vmware.vcloud.multi-site-core Subsystem Status: [WAITING]
com.vmware.vcloud.multi-site-api Subsystem Status: [WAITING]
com.vmware.vcloud.rest-api-handlers Subsystem Status: [WAITING]

Application Initialization: 'Legacy Cell Application' 50% complete. Subsystem 'com.vmware.vcloud.fabric.storage' started

 

Somehow they must be related.

[Ashil Krishnan]

 ,

cell startup never completed. Lets look for errors in cell-runtime.log(same location as cell.log).

 

[VirtualOz]

Lots going on.

I have grep's for the warnings.
Here is a sample that keeps repeating.

2024-01-08 22:29:26,513 | WARN | Bootstrap Application | DirectlyConfigurableXmlApplicationContext | 4,577 milliseconds creating bean: configurationService; SLOW BEAN |
2024-01-08 22:29:26,622 | WARN | Bootstrap Application | DirectlyConfigurableXmlApplicationContext | 4,699 milliseconds creating bean: webConfigurationData; SLOW BEAN |
2024-01-08 22:29:41,033 | WARN | Bootstrap Application | DirectlyConfigurableXmlApplicationContext | 8,286 milliseconds creating bean: globalSslContext; SLOW BEAN |
2024-01-08 22:29:49,695 | WARN | Bootstrap Application | DirectlyConfigurableXmlApplicationContext | 4,691 milliseconds creating bean: databaseConfiguration; SLOW BEAN |
2024-01-08 22:29:57,899 | WARN | Module: Legacy Cell Application | DirectlyConfigurableXmlApplicationContext | 1,370 milliseconds creating bean: productBuildDateBean; SLOW BEAN |
2024-01-08 22:29:57,917 | WARN | Module: Legacy Cell Application | DirectlyConfigurableXmlApplicationContext | 1,389 milliseconds creating bean: org.springframework.beans.factory.config.MethodInvokingFactoryBean#c76e7e6; SLOW BEAN |
2024-01-08 22:29:57,941 | WARN | Module: Legacy Cell Application | DirectlyConfigurableXmlApplicationContext | 1,425 milliseconds creating bean: productInfoBean; SLOW BEAN |
2024-01-08 22:29:57,967 | WARN | Module: Legacy Cell Application | DirectlyConfigurableXmlApplicationContext | 1,453 milliseconds creating bean: productInfoService; SLOW BEAN |
2024-01-08 22:30:06,241 | WARN | Module: Legacy Cell Application | DirectlyConfigurableXmlApplicationContext | 8,241 milliseconds creating bean: vfsFileBroker; SLOW BEAN |
2024-01-08 22:30:06,248 | WARN | Module: Legacy Cell Application | DirectlyConfigurableXmlApplicationContext | 8,248 milliseconds creating bean: vfsFileBrokerService; SLOW BEAN |
2024-01-08 22:30:07,444 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 1,447 milliseconds creating bean: jmsBrokerConfiguration; SLOW BEAN |
2024-01-08 22:30:12,156 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 6,249 milliseconds creating bean: jmsBroker; SLOW BEAN |
2024-01-08 22:30:12,931 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 7,025 milliseconds creating bean: globalJmsAdminServiceBean; SLOW BEAN |
2024-01-08 22:30:12,934 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 7,028 milliseconds creating bean: globalJmsServiceBean; SLOW BEAN |
2024-01-08 22:30:14,149 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 8,259 milliseconds creating bean: cacheManager; SLOW BEAN |
2024-01-08 22:30:17,019 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 11,171 milliseconds creating bean: internalSessionFactory; SLOW BEAN |
2024-01-08 22:30:17,039 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 11,192 milliseconds creating bean: sessionfactory; SLOW BEAN |
2024-01-08 22:30:17,082 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 11,284 milliseconds creating bean: repositoryTemplate; SLOW BEAN |
2024-01-08 22:30:17,094 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 12,001 milliseconds creating bean: hibernateAmqpSettingsDaoImpl; SLOW BEAN |
2024-01-08 22:30:17,096 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 12,280 milliseconds creating bean: amqpManager; SLOW BEAN |
2024-01-08 22:30:17,098 | WARN | Module: com.vmware.vcloud-common-cell-impl | DirectlyConfigurableXmlApplicationContext | 12,476 milliseconds creating bean: com.vmware.vcloud.common.service.OsgiServiceRegistrationFactoryBean#0; SLOW BEAN |
2024-01-08 22:30:38,190 | WARN | Module: com.vmware.vcloud.common-util | DirectlyConfigurableXmlApplicationContext | 16,101 milliseconds creating bean: internalSessionFactory; SLOW BEAN |
2024-01-08 22:30:38,194 | WARN | Module: com.vmware.vcloud.common-util | DirectlyConfigurableXmlApplicationContext | 16,104 milliseconds creating bean: globalSessionFactory; SLOW BEAN |
2024-01-08 22:30:38,230 | WARN | Module: com.vmware.vcloud.common-util | DirectlyConfigurableXmlApplicationContext | 16,157 milliseconds creating bean: globalConversationFactory; SLOW BEAN |
2024-01-08 22:30:38,246 | WARN | Module: com.vmware.vcloud.common-util | DirectlyConfigurableXmlApplicationContext | 16,174 milliseconds creating bean: conversationFactoryInitializer; SLOW BEAN |
2024-01-08 22:30:41,866 | WARN | Module: Autoscale Cell Application | DirectlyConfigurableXmlApplicationContext | 36,287 milliseconds creating bean: autoscaleConfiguration; SLOW BEAN |
2024-01-08 22:30:41,867 | WARN | Module: Autoscale Cell Application | DirectlyConfigurableXmlApplicationContext | 37,148 milliseconds creating bean: mqttClientFactory; SLOW BEAN |
2024-01-08 22:30:41,966 | WARN | Module: HTML5 Cell Application | DirectlyConfigurableXmlApplicationContext | 43,676 milliseconds creating bean: validationFilter; SLOW BEAN |
2024-01-08 22:30:43,352 | WARN | Module: Autoscale Cell Application | DirectlyConfigurableXmlApplicationContext | 1,441 milliseconds creating bean: autoscaleApplication; SLOW BEAN |
2024-01-08 22:30:43,376 | WARN | Module: Autoscale Cell Application | DirectlyConfigurableXmlApplicationContext | 1,508 milliseconds creating bean: checkMqttTask; SLOW BEAN |
2024-01-08 22:30:48,880 | WARN | Module: com.vmware.vcloud.consoleproxy | DirectlyConfigurableXmlApplicationContext | 2,046 milliseconds creating bean: deployWar; SLOW BEAN |
2024-01-08 22:30:52,152 | WARN | Module: com.vmware.vcloud.api-framework | DirectlyConfigurableXmlApplicationContext | 1,410 milliseconds creating bean: mqttOutgoingRequestChannel; SLOW BEAN |
2024-01-08 22:30:52,275 | WARN | Module: com.vmware.vcloud.api-framework | DirectlyConfigurableXmlApplicationContext | 1,557 milliseconds creating bean: vcdRemoteGateway; SLOW BEAN |
2024-01-08 22:30:55,165 | WARN | Module: com.vmware.vcloud.api-framework | DirectlyConfigurableXmlApplicationContext | 1,661 milliseconds creating bean: authorizationService; SLOW BEAN |
2024-01-08 22:30:55,187 | WARN | Module: com.vmware.vcloud.api-framework | DirectlyConfigurableXmlApplicationContext | 1,804 milliseconds creating bean: extensibilityFilter; SLOW BEAN |
2024-01-08 22:31:28,266 | WARN | Module: com.vmware.vcloud.common-vmomi | DirectlyConfigurableXmlApplicationContext | 31,913 milliseconds creating bean: vmodlContext; SLOW BEAN |
2024-01-08 22:31:28,565 | WARN | Module: com.vmware.vcloud.common-vmomi | DirectlyConfigurableXmlApplicationContext | 32,213 milliseconds creating bean: vlsiClientServiceOsgi; SLOW BEAN |
2024-01-08 22:31:52,320 | WARN | Module: com.vmware.vcloud.vim-proxy | DirectlyConfigurableXmlApplicationContext | 1,123 milliseconds creating bean: valFabricQueueingExecutor; SLOW BEAN |
2024-01-08 22:31:52,321 | WARN | Module: com.vmware.vcloud.vim-proxy | DirectlyConfigurableXmlApplicationContext | 1,190 milliseconds creating bean: valFabricActivityExecutor; SLOW BEAN |
2024-01-08 22:31:52,332 | WARN | Module: com.vmware.vcloud.vim-proxy | DirectlyConfigurableXmlApplicationContext | 1,202 milliseconds creating bean: valFabricActivityTemplate; SLOW BEAN |
2024-01-08 22:31:53,952 | WARN | Module: com.vmware.vcloud.vim-proxy | DirectlyConfigurableXmlApplicationContext | 1,271 milliseconds creating bean: VimInventoryService; SLOW BEAN |
2024-01-08 22:31:54,065 | WARN | Module: com.vmware.vcloud.vim-proxy | DirectlyConfigurableXmlApplicationContext | 2,992 milliseconds creating bean: fileTransferService; SLOW BEAN |
2024-01-08 22:33:12,438 | WARN | Module: com.vmware.vcloud.vim-proxy | DirectlyConfigurableXmlApplicationContext | 72,998 milliseconds creating bean: providerVdcDao; SLOW BEAN |
2024-01-08 22:33:13,883 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,033 milliseconds creating bean: networkFabric; SLOW BEAN |
2024-01-08 22:33:14,060 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,260 milliseconds creating bean: com.vmware.vcloud.net.impl.HFNetworkPoolManagerImpl#6a62cc5e; SLOW BEAN |
2024-01-08 22:33:14,230 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,453 milliseconds creating bean: networkPoolManager; SLOW BEAN |
2024-01-08 22:33:14,346 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,615 milliseconds creating bean: defaultNetworkAccessibilityManager; SLOW BEAN |
2024-01-08 22:33:14,661 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,954 milliseconds creating bean: networkAccessibilityManager; SLOW BEAN |
2024-01-08 22:33:14,662 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,955 milliseconds creating bean: com.vmware.vcloud.common.service.OsgiServiceRegistrationFactoryBean#19; SLOW BEAN |
2024-01-08 22:33:18,998 | WARN | Module: com.vmware.vcloud.imagetransfer-server | DirectlyConfigurableXmlApplicationContext | 4,265 milliseconds creating bean: deployWar; SLOW BEAN |
2024-01-08 22:33:20,211 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 2,866 milliseconds creating bean: jaxbMarshaller; SLOW BEAN |
2024-01-08 22:33:20,229 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 2,887 milliseconds creating bean: org.springframework.http.converter.xml.MarshallingHttpMessageConverter#4032d597; SLOW BEAN |
2024-01-08 22:33:20,762 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 3,951 milliseconds creating bean: restTemplateNsxVErrorHandler; SLOW BEAN |
2024-01-08 22:33:20,789 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 3,978 milliseconds creating bean: restTemplateErrorNsxVHandlerOsgi; SLOW BEAN |
2024-01-08 22:33:22,244 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,158 milliseconds creating bean: trinityDtos; SLOW BEAN |
2024-01-08 22:33:22,261 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,177 milliseconds creating bean: xstreamMarshallerTrinity; SLOW BEAN |
2024-01-08 22:33:22,262 | WARN | Module: com.vmware.vcloud.fabric.net | DirectlyConfigurableXmlApplicationContext | 1,185 milliseconds creating bean: xstreamMarshallerTrinityOsgi; SLOW BEAN |
2024-01-08 22:34:02,102 | WARN | Module: com.vmware.vcloud.fabric.storage | DirectlyConfigurableXmlApplicationContext | 1,334 milliseconds creating bean: placementManager; SLOW BEAN |
2024-01-08 22:34:03,979 | WARN | Module: com.vmware.vcloud.fabric.storage | DirectlyConfigurableXmlApplicationContext | 1,446 milliseconds creating bean: sdrsPlacementManager; SLOW BEAN |
2024-01-08 22:34:04,128 | WARN | Module: com.vmware.vcloud.fabric.storage | DirectlyConfigurableXmlApplicationContext | 3,401 milliseconds creating bean: baseStorageSelector; SLOW BEAN |
2024-01-08 22:34:04,266 | WARN | Module: com.vmware.vcloud.fabric.storage | DirectlyConfigurableXmlApplicationContext | 3,574 milliseconds creating bean: storageSelectorFactory; SLOW BEAN |
2024-01-08 22:34:04,267 | WARN | Module: com.vmware.vcloud.fabric.storage | DirectlyConfigurableXmlApplicationContext | 3,615 milliseconds creating bean: com.vmware.vcloud.fabric.storage.placement.impl.ResourcePoolPlacementHubFinder#1fc51b6d; SLOW BEAN |
2024-01-08 22:34:04,267 | WARN | Module: com.vmware.vcloud.fabric.storage | DirectlyConfigurableXmlApplicationContext | 3,619 milliseconds creating bean: util:list#413d1be5; SLOW BEAN |
2024-01-08 22:34:04,268 | WARN | Module: com.vmware.vcloud.fabric.storage | DirectlyConfigurableXmlApplicationContext | 3,659 milliseconds creating bean: storageFabricPlacementHubFinder; SLOW BEAN |
2024-01-08 22:35:11,574 | WARN | Module: com.vmware.vcloud.fabric.compute | DirectlyConfigurableXmlApplicationContext | 62,279 milliseconds creating bean: sessionfactory; SLOW BEAN |
2024-01-08 22:35:11,657 | WARN | Module: com.vmware.vcloud.fabric.compute | DirectlyConfigurableXmlApplicationContext | 62,363 milliseconds creating bean: repositoryTemplate; SLOW BEAN |
2024-01-08 22:35:11,706 | WARN | Module: com.vmware.vcloud.fabric.compute | DirectlyConfigurableXmlApplicationContext | 62,439 milliseconds creating bean: computeVmRtDao; SLOW BEAN |
2024-01-08 22:35:12,168 | WARN | Module: com.vmware.vcloud.fabric.compute | DirectlyConfigurableXmlApplicationContext | 63,220 milliseconds creating bean: virtualResourcePoolFactory; SLOW BEAN |
2024-01-08 22:35:42,475 | WARN | Module: HTML5 Cell Application | DirectlyConfigurableXmlApplicationContext | 300,029 milliseconds creating bean: filterUtils; SLOW BEAN |
2024-01-08 22:35:42,476 | WARN | Module: HTML5 Cell Application | DirectlyConfigurableXmlApplicationContext | 300,060 milliseconds creating bean: sessionFilter; SLOW BEAN |
2024-01-08 22:35:42,476 | WARN | Module: HTML5 Cell Application | DirectlyConfigurableXmlApplicationContext | 344,283 milliseconds creating bean: vcdSecuredWebappConfiguration; SLOW BEAN |
2024-01-08 22:35:42,477 | WARN | Module: HTML5 Cell Application | DirectlyConfigurableXmlApplicationContext | Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'vcdSecuredWebappConfiguration': Unsatisfied dependency expressed through field 'sessionFilter'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'sessionFilter': Unsatisfied dependency expressed through field 'filterUtils'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'filterUtils': Unsatisfied dependency expressed through field 'securityService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'securityService': FactoryBean threw exception on object creation; nested exception is java.util.concurrent.TimeoutException: Timed out waiting for service: 'securityService', objectClasses='[interface com.vmware.vcloud.api.presentation.service.SecurityService]', filter='(objectClass=com.vmware.vcloud.api.presentation.service.SecurityService)' |

[VirtualOz]

Latest.

I needed to rule out the NFS transfer.

So I have deployed a W2016 server with NFS services.
As with the Ubuntu server it has two interfaces.
The first on management 192.168.5.0/24 and the second on the NFS transfer network 192.168.6.0/24.
Gateway on the 192.168.5.0/24.

Tested the install and exactly the same result.

Next I created a new DVS isolated with no uplinks.
Created a DVS PortGroup on vlan 6 for 192.168.6.0/24. (vCD NFS Transfer)
Set a VM Group to Host Group DRS rule so only the vCD server and the NFS server were on a single ESXi host.
No other VM's to stop the possibility of CPU contention.
The Cisco 2960 is also taken out of the test.
Exactly the same.

I am new really breaking this down to basics.
Reading through the vCloud Director Install Guide again.

[marti28052]

Hi VirtualOz, could you resolve it? I have tha same issue.

Thanks

Martín

[Christophe NIEL]

I read your original post

eth0 - 192.168.5.25/24 - PortGroup DS-Vlan-192.168.5.0
eth1 - 192.168.6.25/24 - PortGroup DS-Vlan-192.168.6.0

I don't know why you call the eth1 "vcd nfs transfer", eth1 should be used for the management 5480, postgres cluster, and can be used for nfs

eth0 should be used for the portal web pages. 

Be sure check the official deployment guide VMware Cloud Director Architecture

install the nfs prerequisite : Preparing the Transfer Server Storage for Your VMware Cloud Director on Linux

check here what port are use on eth1 Deployment and Initial Configuration of Your VMware Cloud Director Appliance (you 

(to be frank, It's probably not that which is causing the problem, it's just tcpip binding)

one other thing I encountered in the past : there were error deploying the OVF from the vcenter ui, some settings were not passed correctly to the VM when it was running the first boot, and using the same ovf directly from the ESXi web UI (or the ovftool in command line) was working fine

check Deploying Your VMware Cloud Director Appliance by Using VMware OVF Tool for exemple of command line to deploy the ovf

or try from the esxi web ui directly

[marti28052]

2025-02-13 20:16:53,127 | WARN     | Module: HTML5 Cell Application | DirectlyConfigurableXmlApplicationContext | Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'vcdSecuredWebappConfiguration': Unsatisfied dependency expressed through field 'sessionFilter'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'sessionFilter': Unsatisfied dependency expressed through field 'filterUtils'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'filterUtils': Unsatisfied dependency expressed through field 'securityService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'securityService': FactoryBean threw exception on object creation; nested exception is java.util.concurrent.TimeoutException: Timed out waiting for service: 'securityService', objectClasses='[interface com.vmware.vcloud.api.presentation.service.SecurityService]', filter='(objectClass=com.vmware.vcloud.api.presentation.service.SecurityService)' |
2025-02-13 20:16:53,132 | FATAL    | Module: HTML5 Cell Application | ModuleRegistry                 | Error in module: "HTML5 Cell Application"; Bean: vcdSecuredWebappConfiguration, Message: Error creating bean with name 'vcdSecuredWebappConfiguration': Unsatisfied dependency expressed through field 'sessionFilter'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'sessionFilter': Unsatisfied dependency expressed through field 'filterUtils'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'filterUtils': Unsatisfied dependency expressed through field 'securityService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'securityService': FactoryBean threw exception on object creation; nested exception is java.util.concurrent.TimeoutException: Timed out waiting for service: 'securityService', objectClasses='[interface com.vmware.vcloud.api.presentation.service.SecurityService]', filter='(objectClass=com.vmware.vcloud.api.presentation.service.SecurityService)' |
        at com.vmware.vcloud.common.main.bootstrap.DirectlyConfigurableXmlApplicationContext$LoggingBeanFactory.doCreateBean(DirectlyConfigurableXmlApplicationContext.java:54)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:920)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
        at com.vmware.vcloud.common.main.bootstrap.DirectlyConfigurableXmlApplicationContext.refresh(DirectlyConfigurableXmlApplicationContext.java:151)
        at com.vmware.vcloud.common.main.bootstrap.DirectlyConfigurableXmlApplicationContext$LoggingBeanFactory.doCreateBean(DirectlyConfigurableXmlApplicationContext.java:54)
        at com.vmware.vcloud.common.main.bootstrap.DirectlyConfigurableXmlApplicationContext$LoggingBeanFactory.doCreateBean(DirectlyConfigurableXmlApplicationContext.java:54)

at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
        ... 38 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:146)
        at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:127)
        at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
        at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
        ... 48 more
2025-02-12 12:50:41,498 | ERROR    | main                      | ReconfigureDatabaseCommand     | Invalid database configuration provided. Aborting configuration of local cell. |
2025-02-12 12:50:41,499 | DEBUG    | main                      | CellManagementTool             | Exiting with completion code: OTHER |

[AbbedSedkaoui]

Hi all, marti28052,

First: Increase the timeout Ashil Krishnan proposed, that will help complete the install, then shutdown the vm, 

i've written about this https://strivevirtually.net/post/deploying-cloud-director-10.4-in-small-form-factor-the-troubleshoot/

it is caused by slow storage.

Second: increase the vm CPU for the first cell boot to 4 vCPU on small form factor.

If you're also interested in VCD with NSX, i've made a tutorial in images https://strivevirtually.net/post/tutorial-install-vmware-cloud-director-10.5.1.1-and-create-provider-virtual-data-center-pvdc-backed-by-vsphere-with-tanzu-kubernetes-8.0u2-and-nsx-t-4.1.2/

Cheers

[marti28052]

HI AbbedSedkaoui,  

I wanted to let you know that I have increased the delay as indicated in the instructions. I also increased the vCPU, even though I am using a medium-sized cell form factor, going from 4 vCPUs to 8 vCPUs.

However, I am still experiencing the same error-the cell is still not starting up. Do you have any other ideas?

Thanks!

[AbbedSedkaoui]

Hi @marti28052,

There is 2 distinct issue when deploying in lab

1st the setup don't complete with error you had posted
```
2025-02-12 12:50:41,498 | ERROR    | main                      | ReconfigureDatabaseCommand     | Invalid database configuration provided. Aborting configuration of local cell. |
```
=> solved by increasing the timeout

2nd the setup complete successfully but the primary cell don't finish it's firstboot
=> solved by increasing the cpu

Consider redoing everything from scratch like that:

- redeploying

- setup until it fail

- confirm that you have "Timed out" in your appliance-sync.log when adding the certificate node

cat /opt/vmware/var/log/vcd/appliance-sync.log | grep Timed

```
Timed out adding certificate for node node-35184bba-f181-469d-be92-ebb45ff6727d, continuing to other nodes.
```

- increase the delay timeout

```
sed -i s/10s/60s/ /opt/vmware/appliance/bin/appliance-sync.sh
```
resulted line should be:
```
cat /opt/vmware/appliance/bin/appliance-sync.sh | grep force
```

```
/usr/bin/timeout 60s $VCLOUD_HOME/bin/cell-management-tool import-trusted-certificates --source $APPLIANCE_SHARE/$i/vcd_ova.crt --force
```

- follow the log before retry completing setup
```
tail -f /opt/vmware/var/log/vcd/configure-vcd.log
```

You should see something like
```
Reconfiguring vCD to use a secure database connection.
2022-09-25 18:36:29 | setupvcd.sh | Reconfiguring vCD to use a secure database connection.
Database configuration in properties files complete for local cell.
Database configuration was successfully refreshed in running Cell.    <-- at this point the setup is completed successfully, ui should confirm it
2022-09-25 18:36:57 | configure-vcd.sh | Invoking cellstartup script.  <-- note difference in second should be greater the 10s default
```

- then revert back the original setting

```
sed -i s/60s/10s/ /opt/vmware/appliance/bin/appliance-sync.sh

systemctl restart appliance-sync.timer
```

- Shutdown on the VM

- Increase vCPUs from 2 to 4. (Updated)

- Power on the VM

- Wait few minutes for the vcd appliance to fully load before opening http://{VCD_IP}/provider

All that previously shared and it was confirmed working steps.
https://community.broadcom.com/vmware-cloud-foundation/discussion/configure-vcd-script-failed-to-complete

Cheers