vCenter

 View Only
  • 1.  VPXD service not starting.

    Posted Feb 11, 2020 05:41 PM

    My vCenter is using a shared Datastore and that datastore got full recently and my vcenter started throwing this error in UI.

    503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x00007f290400b950] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

    Once i tried to cleanup some space and start the service's back, i am getting this error while starting vpxd:

    Operation not cancellable. Please wait for it to finish...

    Performing start operation on service vpxd...

    Successfully started service vmware-vmon

    Error executing start on service vpxd. Details {

        "resolution": null,

        "componentKey": null,

        "detail": [

            {

                "args": [

                    "vpxd"

                ],

                "translatable": "An error occurred while starting service '%(0)s'",

                "id": "install.ciscommon.service.failstart",

                "localized": "An error occurred while starting service 'vpxd'"

            }

        ],

        "problemId": null

    }

    Service-control failed. Error: {

        "resolution": null,

        "componentKey": null,

        "detail": [

            {

                "args": [

                    "vpxd"

                ],

                "translatable": "An error occurred while starting service '%(0)s'",

                "id": "install.ciscommon.service.failstart",

                "localized": "An error occurred while starting service 'vpxd'"

            }

        ],

        "problemId": null

    }

    Any help would be appreciated!!

    Thanks



  • 2.  RE: VPXD service not starting.

    Posted Feb 12, 2020 04:38 AM

    Are you sure about the enough free space inside the VCSA guest OS?

    Can you check it again please? with running following command in the VCSA shell:

    cd /

    du -ah --max-depth=1



  • 3.  RE: VPXD service not starting.

    Posted Feb 12, 2020 06:40 AM

    I ran the command and here is the result:

    48K     ./root

    36K     ./tmp

    1.3G    ./var

    0       ./srv

    4.0K    ./ssh

    0       ./lib

    0       ./sys

    179M    ./opt

    464K    ./dev

    4.0K    ./vapi-endpoint

    2.5M    ./tftpboot

    306M    ./etc

    33M     ./boot

    0       ./sbin

    3.9G    ./usr

    0       ./media

    200K    ./home

    1.1M    ./vasecurity

    680K    ./run

    12G     ./storage

    12K     ./mnt

    0       ./bin

    16K     ./lost+found

    12K     ./.cache

    0       ./lib64

    du: cannot access './proc/65003/task/65003/fd/4': No such file or directory

    du: cannot access './proc/65003/task/65003/fdinfo/4': No such file or directory

    du: cannot access './proc/65003/fd/3': No such file or directory

    du: cannot access './proc/65003/fdinfo/3': No such file or directory

    0       ./proc

    18G     .



  • 4.  RE: VPXD service not starting.

    Posted Feb 12, 2020 06:55 AM

    Please retry this command for /storage directory:

    cd /storage

    du -ah --max-depth=2

    Check contents of every directory that is occupied with large space.

    I think there is too much log files (or historical logs) or maybe cache content in your VCSA, so it cause the volume fills more than normal and then prevent some critical services to start and interrupted them



  • 5.  RE: VPXD service not starting.

    Posted Feb 12, 2020 07:07 AM

    Here is the output from /storage

    4.0K    ./mbcs/mbcsspec.properties

    8.0K    ./mbcs

    4.0K    ./updatemgr/software-update

    4.0K    ./updatemgr/jetty-temp

    16K     ./updatemgr/lost+found

    4.0K    ./updatemgr/patch-store-temp

    8.0K    ./updatemgr/patch-store

    40K     ./updatemgr

    4.0K    ./imagebuilder/vibs

    4.0K    ./imagebuilder/depot

    16K     ./imagebuilder/lost+found

    4.0K    ./imagebuilder/exports

    32K     ./imagebuilder

    20K     ./sps/tomcat

    24K     ./sps

    16K     ./dblog/lost+found

    161M    ./dblog/vpostgres

    161M    ./dblog

    4.0K    ./sqlite-state/vmca

    4.0K    ./sqlite-state/vmafd

    12K     ./sqlite-state

    16K     ./db/lost+found

    125M    ./db/vpostgres

    4.0K    ./db/journal

    41M     ./db/vmware-vmdir

    68K     ./db/vmware-imagebuilder

    132K    ./db/vmware-vmafd

    20K     ./db/vpostgres_ssl

    165M    ./db

    4.0K    ./vmware-vmon/vapi-endpoint.start.cmd

    132K    ./vmware-vmon/.svcStats

    4.0K    ./vmware-vmon/cm.start.cmd

    4.0K    ./vmware-vmon/vmonapi.start.cmd

    4.0K    ./vmware-vmon/defaultStartProfile

    4.0K    ./vmware-vmon/cis-license.start.cmd

    4.0K    ./vmware-vmon/vsm.start.cmd

    4.0K    ./vmware-vmon/signingcert.crt

    4.0K    ./vmware-vmon/vmonapi-health.xml

    4.0K    ./vmware-vmon/sps.start.cmd

    4.0K    ./vmware-vmon/vsphere-client.start.cmd

    4.0K    ./vmware-vmon/sca.start.cmd

    4.0K    ./vmware-vmon/eam.start.cmd

    4.0K    ./vmware-vmon/content-library.start.cmd

    4.0K    ./vmware-vmon/perfcharts.start.cmd

    4.0K    ./vmware-vmon/vpxd-svcs.start.cmd

    0       ./vmware-vmon/vmonserver

    4.0K    ./vmware-vmon/vsphere-ui.start.cmd

    200K    ./vmware-vmon

    4.0K    ./service-state/updatemgr

    4.0K    ./service-state/imagebuilder

    4.0K    ./service-state/vmware-vsan-health

    4.0K    ./service-state/vmidentity

    4.0K    ./service-state/vsphere-client

    4.0K    ./service-state/vmafd

    4.0K    ./service-state/rbd

    4.0K    ./service-state/vsphere-ui

    4.0K    ./service-state/likewise

    4.0K    ./service-state/vmdir

    44K     ./service-state

    16K     ./autodeploy/lost+found

    4.0K    ./autodeploy/backup

    4.0K    ./autodeploy/cache

    28K     ./autodeploy

    8.0K    ./core/software-update

    469M    ./core/core.postgres.831

    16K     ./core/lost+found

    4.0K    ./core/netdumps

    469M    ./core

    4.0K    ./pre-failover/vmdir_request_restore.sh

    8.0K    ./pre-failover

    4.0K    ./applmgmt/backup_restore

    8.0K    ./applmgmt

    4.0K    ./vsphere-client/ceip

    240K    ./vsphere-client/SerenityDB

    248K    ./vsphere-client

    2.0G    ./log/vmware

    16K     ./log/lost+found

    4.0K    ./log/remote

    2.0G    ./log

    4.0K    ./vmware-pod

    4.0K    ./vsan-health

    4.0K    ./vsphere-ui/ceip

    40K     ./vsphere-ui/SerenityDB

    48K     ./vsphere-ui

    20K     ./vpxd-svcs/tomcat

    24K     ./vpxd-svcs

    16K     ./netdump/lost+found

    20K     ./netdump

    4.0K    ./seat/software-update

    180M    ./seat/vpostgres

    16K     ./seat/lost+found

    180M    ./seat

    4.0K    ./vsan-dps

    9.0G    ./archive/vpostgres

    16K     ./archive/lost+found

    4.0K    ./archive/software-update

    9.0G    ./archive

    12G     .



  • 6.  RE: VPXD service not starting.

    Posted Feb 12, 2020 08:11 AM

    So check the following directory (based on your attached result)

    2.0G./log/vmware
    2.0G./log
    9.0G./archive/vpostgres
    9.0G./archive

    and remove some old and unused (archive) files. But Strongly recommend to provide a backup from the VCSA before removing any old file/folder. Then try again to start the interrupted services or reboot the VM and give me the final result