Unable to login to Vcenter

[Srikanthan Elangovan]

Hi

I have a vcenter server installed on a Windows server system. The vcenter server is not starting, it is showing an error message as

vSphere Client could not connect to X.X.X.X

An unknown connection error occurred. (The server could not interpret the client's request. (The remote server returned an error: (503) Server Unavailable.))

Request help at the earliest as this is one of the production vcente server.

Thank you.

Srikanthan

[Nawals]

change it delay restart and restart web client service ( that was also stopped) after 1-2 minutes vpxd started it self.

it also take 5-10 minutes for initializing )

If resolve please mark helpful or correct.

[Srikanthan Elangovan]

Hi Nawals,

Thank you for your quick reply, many of the services are not starting. The following are the services which are showing as stopped.

INFO:root:Service: VMwareComponentManager, Action: status

Service: VMwareComponentManager, Action: status

2020-03-17T07:35:20.144Z   State for service VMwareComponentManager: START_PENDING

INFO:root:Service: vmware-license, Action: status

Service: vmware-license, Action: status

2020-03-17T07:35:20.148Z   State for service vmware-license: STOPPED

INFO:root:Service: VMWareNetworkCoredumpWebserver, Action: status

Service: VMWareNetworkCoredumpWebserver, Action: status

2020-03-17T07:35:20.151Z   State for service VMWareNetworkCoredumpWebserver: STOPPED

INFO:root:Service: vmwareServiceControlAgent, Action: status

Service: vmwareServiceControlAgent, Action: status

2020-03-17T07:35:20.156Z   State for service vmwareServiceControlAgent: STOPPED

INFO:root:Service: vmware-network-coredump, Action: status

Service: vmware-network-coredump, Action: status

2020-03-17T07:35:20.160Z   State for service vmware-network-coredump: STOPPED

INFO:root:Service: vapiEndpoint, Action: status

Service: vapiEndpoint, Action: status

2020-03-17T07:35:20.164Z   State for service vapiEndpoint: STOPPED

INFO:root:Service: vPostgres, Action: status

Service: vPostgres, Action: status

2020-03-17T07:35:20.168Z   State for service vPostgres: STOPPED

INFO:root:Service: invsvc, Action: status

Service: invsvc, Action: status

2020-03-17T07:35:20.171Z   State for service invsvc: STOPPED

INFO:root:Service: mbcs, Action: status

Service: mbcs, Action: status

2020-03-17T07:35:20.175Z   State for service mbcs: STOPPED

INFO:root:Service: vpxd, Action: status

Service: vpxd, Action: status

2020-03-17T07:35:20.179Z   State for service vpxd: STOPPED

INFO:root:Service: EsxAgentManager, Action: status

Service: EsxAgentManager, Action: status

2020-03-17T07:35:20.183Z   State for service EsxAgentManager: STOPPED

INFO:root:Service: vmware-autodeploy-waiter, Action: status

Service: vmware-autodeploy-waiter, Action: status

2020-03-17T07:35:20.187Z   State for service vmware-autodeploy-waiter: STOPPED

INFO:root:Service: vimPBSM, Action: status

Service: vimPBSM, Action: status

2020-03-17T07:35:20.190Z   State for service vimPBSM: STOPPED

INFO:root:Service: vdcs, Action: status

Service: vdcs, Action: status

2020-03-17T07:35:20.194Z   State for service vdcs: STOPPED

INFO:root:Service: vmsyslogcollector, Action: status

Service: vmsyslogcollector, Action: status

2020-03-17T07:35:20.197Z   State for service vmsyslogcollector: STOPPED

INFO:root:Service: vmware-vpx-workflow, Action: status

Service: vmware-vpx-workflow, Action: status

2020-03-17T07:35:20.201Z   State for service vmware-vpx-workflow: STOPPED

INFO:root:Service: VServiceManager, Action: status

Service: VServiceManager, Action: status

2020-03-17T07:35:20.205Z   State for service VServiceManager: STOPPED

INFO:root:Service: vspherewebclientsvc, Action: status

Service: vspherewebclientsvc, Action: status

2020-03-17T07:35:20.208Z   State for service vspherewebclientsvc: STOPPED

INFO:root:Service: vmware-perfcharts, Action: status

Service: vmware-perfcharts, Action: status

2020-03-17T07:35:20.212Z   State for service vmware-perfcharts: STOPPED

INFO:root:Service: vmwarevws, Action: status

Service: vmwarevws, Action: status

2020-03-17T07:35:20.216Z   State for service vmwarevws: STOPPED

INFO:root:StartPending:

VMwareComponentManager (VMware Component Manager)

StartPending:

VMwareComponentManager (VMware Component Manager)

INFO:root:Running:

VMWareAfdService (VMware Authentication Framework) VMWareCertificateService (VMware Certificate Service) VMWareDirectoryService (VMware Directory Service) VMwareIdentityMgmtService (VMware Identity Management Service) VMwareSTS (VMware Security Token Service) rhttpproxy (VMware HTTP Reverse Proxy) vmware-cis-config ()

Running:

VMWareAfdService (VMware Authentication Framework) VMWareCertificateService (VMware Certificate Service) VMWareDirectoryService (VMware Directory Service) VMwareIdentityMgmtService (VMware Identity Management Service) VMwareSTS (VMware Security Token Service) rhttpproxy (VMware HTTP Reverse Proxy) vmware-cis-config ()

INFO:root:Stopped:

EsxAgentManager (VMware ESX Agent Manager) VMWareNetworkCoredumpWebserver (VMware vSphere ESXi Dump Collector WebService) VServiceManager (VMware vService Manager) invsvc (VMware Inventory Service) mbcs (VMware Message Bus Configuration Service) vPostgres (VMware Postgres) vapiEndpoint (VMware vAPI Endpoint) vdcs (VMware Content Library Service) vimPBSM (VMware vSphere Profile-Driven Storage Service) vmsyslogcollector (VMware Syslog Collector) vmware-auto

deploy-waiter (VMware vSphere Auto Deploy Waiter) vmware-license (VMware License Service) vmware-network-coredump (VMware vSphere ESXi Dump Collector) vmware-perfcharts (VMware Performance Charts) vmware-vpx-workflow (VMware vCenter Workflow Manager) vmwareServiceControlAgent (VMware Service Control Agent) vmwarevws (VMware System and Hardware Health Manager) vpxd (VMware vCenter Server) vspherewebclientsvc ()

Stopped:

EsxAgentManager (VMware ESX Agent Manager) VMWareNetworkCoredumpWebserver (VMware vSphere ESXi Dump Collector WebService) VServiceManager (VMware vService Manager) invsvc (VMware Inventory Service) mbcs (VMware Message Bus Configuration Service) vPostgres (VMware Postgres) vapiEndpoint (VMware vAPI Endpoint) vdcs (VMware Content Library Service) vimPBSM (VMware vSphere Profile-Driven Storage Service) vmsyslogcollector (VMware Syslog Collector) vmware-auto

deploy-waiter (VMware vSphere Auto Deploy Waiter) vmware-license (VMware License Service) vmware-network-coredump (VMware vSphere ESXi Dump Collector) vmware-perfcharts (VMware Performance Charts) vmware-vpx-workflow (VMware vCenter Workflow Manager) vmwareServiceControlAgent (VMware Service Control Agent) vmwarevws (VMware System and Hardware Health Manager) vpxd (VMware vCenter Server) vspherewebclientsvc ()

Please help......

[Nawals]

Please stop all services command : service-control --stop --all

once all stop services please start: service-control --start --all

If services not come up please reboot the vCenter and check . Also, check the space is available on database server where have the VCDB if vCenter on windows platform. Have any changes on the vCenter server?

[Lalegre]

Try to start all the services and it will show specifically the one that is failing. After that go into the logs /var/vmware/log and search for the service failing.

Paste the output here and lets see.

[Srikanthan Elangovan]

Hi Lalegre,

Tried starting it is throwing error message as unable to start VMWareComponentManager service.

I have installed it on a Windows setup, so please let me know how to get the logs.

[Srikanthan Elangovan]

2020-03-18T07:22:07.691Z   State for service VMwareComponentManager: START_PENDI

NG

2020-03-18T07:22:07.691Z   ERROR Starting service: VMwareComponentManager, Excep

tion: (1056, 'StartService', 'An instance of the service is already running.')

ERROR:root:Unable to start service VMwareComponentManager, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "VMwareComponentManager"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'VMwareCompon

entManager'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Unable to start service VMwareComponentManager, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "VMwareComponentManager"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'VMwareCompon

entManager'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

[Lalegre]

Yes sure go to the next path: C:\ProgramData\VMware\CIS\logs or C:\ProgramData\VMware

There you will need to find the Component Manger folder that states as vmware-cm or CM only.

Inside there order the files by last date and copy the output or the entire log as attached file.

[Srikanthan Elangovan]

Thank you for helping me, please find the log files.

[Lalegre]

No worries man i am here to help you.

Please try to copy the cm.log becasue the one that you provide me is the cm.log.1 that only has 3 lines and it shows no error.

If you can not find it just export the logs and i will review them.

[Srikanthan Elangovan]

Please find the log file.

[Lalegre]

Attivo please show the output of the file eam.properties located it %ProgramData%\VMware\vCenterServer\cfg\vmware-eam.

Also check if the vmware-eam service is started and if not try to start it. If it fails share the log of that service.

Last please check if you have space on your Windows disks.

[Srikanthan Elangovan]

Please find the eam file attached, tried starting the service, it is stuck for a long time and not proceeding further.

[Srikanthan Elangovan]

Please find the eam log files, it never started.

[Lalegre]

Try adding this line to eam.properties and try to start the vmware-eam service again (Please verify if that path is the correct regarding your installation)

eam.web.root=C:/Program Files/VMware/vCenter Server/eam/web

That line must be under this line:

tcserver.tmp.dir

[Srikanthan Elangovan]

please find the attached eam logs

[Srikanthan Elangovan]

Made the changes to the eam properties file and tried starting, after a long time, this is the message it is showing up.

ERROR:root:Unable to start service EsxAgentManager, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "EsxAgentManager"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'EsxAgentMana

ger'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Unable to start service EsxAgentManager, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "EsxAgentManager"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'EsxAgentMana

ger'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

[Lalegre]

Could you please paste my the output of the eam.properties file? I am getting out of ideas.

[Srikanthan Elangovan]

#########################################################################

# Copyright 2013-2014 VMware, Inc.  All rights reserved. VMware Confidential

#########################################################################

vc.proxy.host=localhost

vc.proxy.port=80

# Hostname or IP of the EAM server

# Fill only if EAM is not running on the same host as VC

eam.host=

# EAM service port used to configure the HTTP connector of the application server.

eam.int.http.port=15005

# Port and scheme configuration which is used by the ESX 6.x hosts to reach EAM Vib

# file server.

eam.ext.port=443

eam.ext.scheme=https

# Port and scheme configuration which is used by the ESX 5.x hosts to reach EAM Vib

# file server.

eam.ext.port.deprecated=80

eam.ext.scheme.deprecated=http

eam.support_linked_clone=true

eam.clear_db_on_startup=false

eam.debug_ref_count=false

eam.recent_event_size=20

# Enable/disable VUM integration

vum.integration=true

# Value is specified in minutes (set to 24h = 1440m)

eam.scan_for_unknown_agent_vms=1440

# The timeout to wait for hostd to restart on a host (set to 5m=300s)

eam.hostd_restart_timeout=300

# The following entries will be added verbatim to the advanced options

# of hosts on which EAM is enabled. All are optional.

Net.DVFilterBindIpAddress=169.254.0.1

Net.TrafficFilterIpAddress=

#The IP for the VSWIF NIC on the dvFilter switch (for ESX classic).

Net.DVFilterVswifIpAddress=169.254.0.2

# Resource bundle configuration

eam.resourcebundle.filename=eam-resourcebundle.jar

# VLSI embedded tcServer configuration

#

tcserver.tmp.dir=C:/ProgramData/VMware/vCenterServer/runtime/tmp/vmware/eam/tomcat

eam.web.root=C:/Program Files/VMware/vCenter Server/eam/web

# EAM SSL configuration

#

eam.keystore.type=VKS

eam.key.alias=vpxd-extension

eam.keystore.storename=vpxd-extension

# CM configuration

#

cm.url=http://localhost:18090/cm/sdk/?hostid=f416ae50-62e8-11e5-812b-005056a2166a

cm.wait.attempts=360

cm.wait.intervalSeconds=5

# SSO configuration

#

sso.wait.attempts=360

sso.wait.intervalSeconds=5

# VC SSL configuration

#

vc.truststore.type=VKS

vc.truststore.storename=TRUSTED_ROOTS

vc.tunnelSdkUri.template=https://##{VC_HOST_NAME}##:8089/sdk/vimService

vc.tunnelSdkUri=https://10.16.1.201:8089/sdk/vimService

drs.demandCapacityRatio=100

[Lalegre]

Please upload the next log:

% Programdata%/VMware/vCenterServer/eam/wrapper.log

Could you see the Service Dependencies of the services on the Windows Service Manager and if it has try to start the dependencies and if they fail also upload the logs and describe which services failed.

[Srikanthan Elangovan]

Please find the log file.

[Lalegre]

What about the dependencies?

[Srikanthan Elangovan]

Sorry, there are no other files present in the name of wrapper.

[Lalegre]

Attivo,

I encourage you to open a VMware Support ticket to follow the issue.

[Alex_Romeo]

Hi,

what version of vcenter do you have?

[SOLVED] The remote server returned an error: (503) Server Unavailable - VMware Forum - Spiceworks

ARomeo