Why would the bosh director tile on healthwatch show as down(red)?

[Krystal Peterson]

Hi Everyone,

 

We opened an issue with support about the bosh director showing down (red) in both healthwatch and stack driver.

 

After solving the issue, Stackdriver now shows the bosh director as health, but healthwatch still shows the bosh director as down (red) almost 24 hours later.

 

Does the healthwatch service that is communicating with the Bosh Director need to be checked or is there another step that has to be taken.

 

Thank you,

 

Krystal

[Daniel Mikusa]

That sounds suspicious. What do you see if you login to Ops Manager and navigate to the Director tile -> Status page? Does it show your director is up and running? Similarly, are you able to `bosh login` and run `bosh` commands against your director?

[Daniel Mikusa]

OK, so the good news is that your Director is definitely up and working. The bad news is that it seems like Health Watch isn't. At least it's not getting metrics/info from the Director.

 

It seems like this is the metric that you're referring to, is that correct?

 

https://docs.pivotal.io/pcf-healthwatch/1-4/monitoring.html#bosh_director_probe_availability_percentage

 

If so, I'd suggest trying the two recommended actions from that link:

 

1. Ensure the bosh-health-check app is running in the healthwatch space of the system org. Check the app logs for any obvious errors.
2. SSH into the running bosh-health-check app and copy the BOSH manifest from /home/vcap/app/health_check_manifest.yml. Try to deploy it manually on the BOSH Director and check for errors.

 

[Krystal Peterson]

Hi Daniel,

 

When I log into Ops Manager and navigate to the Director tile -> Status page, it see that the Bosh director is pulling an IP address and all of the metrics to the right have a number value to them. Also, yes I am able to sign into my bosh director and run `bosh -e <alias_name> vms` and everything shows up and running. There are no errors on tasks and all the instance processes are in a running state.

 

Thank you,

 

Krystal

[Krystal Peterson]

Hi Daniel,

 

Yes, I am referring to the Bosh Director health that is located at the bottom of the link that you sent.

 

I will attempt those steps and will report back with findings.

 

Thank you,

 

Krystal