vCenter

 View Only
  • 1.  no healthy upstream error

    Posted Aug 13, 2022 10:28 PM

    Hi Folks,  

     

    I've seen several others post about this same error and I've tried \ and checked several things but I don't see a resolution based on the posts I've read. 

    I have vCenter Version: 7.0.1.00200 ,  Build number:  17327517.

    I am able to login to the vCenter Server GUI and SSH without problems. When I try and login to vCenter I receive the no healthy upstream.

    Originally when I discovered this my log partition was full.   I followed a KB and cleaned it up.  No other partitions are over 50% util.  DNS works, NTP is configured and time is accurate.

    I have a lot of services that aren't started but I **think** at least some of them should be.

    Screen Shot 2022-08-13 at 7.41.42 PM.png

     

     

    When I try and start them using service-control I get some started then an error.

    TimGlen_0-1660434277781.png

     

     

    I'd greatly appreciate any tips or guidance.    Thanks!

    Tim



  • 2.  RE: no healthy upstream error

    Posted Aug 15, 2022 09:50 AM

    Hey  ,

     

    If you have that many services not running, give the appliance a reboot. If this wont help log SR with VMware, 



  • 3.  RE: no healthy upstream error

    Posted Aug 16, 2022 12:49 PM

    Thanks  , but I've rebooted this appliance several times and similar results.  



  • 4.  RE: no healthy upstream error

    Posted Aug 15, 2022 10:15 AM

    What's in the logs?

    Did you already check the certificates (https://kb.vmware.com/s/article/82332), i.e. whether one or more are expired.

    André



  • 5.  RE: no healthy upstream error

    Posted Aug 17, 2022 12:27 AM

       Thanks for replying! 

    I have looked at the certs and some of the certs that are backup are expired but that's it.  I don't believe that should be a problem.  See the attached screenshot for details. 

    About the logs.     service-control --start --all   stops and errors while trying to start vpxd-svcs so I've cat and less that log , grepped for error and other things but honestly, I have no idea what I'm looking for and while I do see some errors I don't know what is relevant.   

    I've zipped and attached the current vpxd & vpxd-svcs log files.  I would greatly appreciate another set of eyes on them if that is the proper direction to go or any other guidance.     Thanks folks! 

     

    Attachment(s)

    zip
    Archive.zip   3.47 MB 1 version


  • 6.  RE: no healthy upstream error

    Posted Aug 17, 2022 07:46 AM

    Not sure, but there are a lot of errors regarding an invalid, and expired certificate.

    2022-08-05T00:05:36.507-04:00 [Thread-13 ERROR com.vmware.vim.sso.client.impl.SoapBindingImpl opId=] Error communicating to the remote server https://vcenter.theglens.net/sts/STSService/vsphere.local
    com.sun.xml.internal.ws.client.ClientTransportException: HTTP transport error: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching vcenter.theglens.net found.
    2022-08-03T21:38:54.400-04:00 error vpxd[11710] [Originator@6876 sub=vmomi.soapStub[10469]] Resetting stub adapter for server <cs p:00007f83308d7f50, TCP:vcenter.theglens.net:443> : service state request failed: N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
    --> PeerThumbprint: 89:94:99:91:6E:F3:FB:9C:EB:84:B8:A0:F7:9F:31:CA:66:77:8F:3E
    --> ExpectedThumbprint:
    --> ExpectedPeerName: vcenter.theglens.net
    --> The remote host certificate has these problems:
    -->
    --> * certificate has expired)

    Please check whether the certificate for the mentioned FQDN has recently been replaced, or has an issue.

    Although less likely, you may also want to ensure that the STS certificate is ok (see https://kb.vmware.com/s/article/79248)

    André



  • 7.  RE: no healthy upstream error



  • 8.  RE: no healthy upstream error

    Posted Aug 17, 2022 06:51 PM
      |   view attached

    Thank you both for your help.

    This is the output from checksys.py,  looks like the STS certs are valid. 

    TimGlen_0-1660761978802.png

     

    I’m sorry, I should have mentioned this earlier.   

    After I fixed the log partition out of space the errors persisted. At that time on August 4, I realized the Machine Cert had expired. At that time I followed the doc below

    https://kb.vmware.com/s/article/2112283

     

    I did receive an error during that process. The log \ error is below. 

    TimGlen_1-1660762161472.png

     

    The service-control.log from that time period is below. 

    TimGlen_0-1660784751270.png

     

     

     

    I'm uploading the certificate-manager.log to this message. 

    Again, I really appreciate your assistance. 

    Tim

     

     

    Attachment(s)

    log
    certificate-manager.log   206 KB 1 version