VmWare Tas for VMS failing with mysql montior component on Azure

[lakshmi mekala]

I am trying to install the Tanzu Application Service, and it is consistently failing with the following error:

Task 180 | 11:58:57 | L starting jobs: mysql_monitor/210eca7a-7bfc-4454-b5a6-c1403a9cd478 (0) (canary) (00:05:38)

L Error: 'mysql_monitor/210eca7a-7bfc-4454-b5a6-c1403a9cd478 (0)' is not running after update. Review logs for failed jobs: replication-canary

Task 180 | 12:03:59 | Error: 'mysql_monitor/210eca7a-7bfc-4454-b5a6-c1403a9cd478 (0)' is not running after update. Review logs for failed jobs: replication-canary

Task 180 Started Thu Feb 15 11:41:17 UTC 2024

Task 180 Finished Thu Feb 15 12:03:59 UTC 2024

Task 180 Duration 00:22:42

Task 180 error

Updating deployment:

Expected task '180' to succeed but state is 'error'

Exit code 1

goroutine 1 [running]:

code.cloudfoundry.org/lager.(*logger).Fatal(0xc0000601e0, {0x7b88b6, 0x1d}, {0x8435a0?, 0xc00010cdb0}, {0x0, 0x0, 0xc00009e360?})

/var/vcap/data/compile/replication-canary/replication-canary/vendor/code.cloudfoundry.org/lager/logger.go:162 +0x5c7

main.main()

/var/vcap/data/compile/replication-canary/replication-canary/main.go:87 +0x6aa

panic: Post "https://uaa.tas.westus.cloudapp.azure.com/oauth/token": dial tcp: lookup uaa.test.westus.cloudapp.azure.com on 169.254.0.2:53: no such host

 

Any clues on what might be wrong or what to check to get to the bottom of this? At which step the uaa host be created in small footprint tas for vms

 

Note: ops manager version: 3.0, small foot print tas for vm's version: 5.0.7, for vm's deployed on azure

[Todd Robbins]

Hello lakshmi,

 

Mysql-monitor is a common place to encounter failure if there is a misconfiguration in your installation of TAS. The above error indicates that the system domain is not resolvable by your DNS server. (lookup uaa.test.westus.cloudapp.azure.com on 169.254.0.2:53: no such host)

 

See document: https://docs.vmware.com/en/VMware-Tanzu-Application-Service/3.0/tas-for-vms/configure-pas.html#configure-domains-3

 

In the TAS tile, you enter a system domain and app domain for your foundation. You need to configure a wildcard entry on your DNS server such that those domains will resolve to your load balancer. Please confirm what your system and app domain are and that they are resolvable as a wildcard (*.<system domain> resolves).

[lakshmi mekala]

Hi Todd,

 

We have deployed the ops manager on azure with public ip and domain name(test.westus.cloudapp.azure.com), we have created a dns zone with domain name(test.westus.cloudapp.azure.com) and addeed dns records for *.system, *.apps, * pointing to opsmanager vm ip, not sure why the uaa system domain is not getting resolved. Are we missing anything?

[Todd Robbins]

HI lakshmi,

 

What did you enter in for System and App Domain within the Tanzu Application Service tile? You mentioned:

 

> we have created a dns zone with domain name(test.westus.cloudapp.azure.com) and addeed dns records for *.system, *.apps, * pointing to opsmanager vm ip

 

The system and app domains should typically resolve to a load balancer not OpsManager. See documents on setting up domains and configuring load balancing in Azure. Typically, you would create a wildcard domain entry *.example such that any subdomain (such as uaa.example.com) directs to load balancer configured for TAS.

[lakshmi mekala]

Hi Todd,

 

The ops vm created with DNS label: test.westus.cloudapp.azure.com

The system and app domains name : system.lnmekala.com and apps.lnmekala.com

Created a dns zone in azure with lnmekala,com and added records with wildcard domain entry *, .system, *.apps, pointing to loadbalancer mapped to router vm.., However the domain lnmekala.com is not resolving. Let me know how this can be resolved and get the tas vms deployed successfully.

 

Thank you,

Lakshmi Narayana

[Todd Robbins]

Hi Lakshmi,

 

You should just need to create a wildcard DNS A record such that any hostname under system domain resolves to IP address of your Azure Load Balancer.

 

Example of DNS record:

 

Name Type Data Domain

*.tas A 198.51.100.1 example.com

 

Where 198.51.100.1 is IP of ALB and "tas.example.com" is system domain for TAS tile. If you are still having issues even after attempting this then I'd suggest raising a ticket with Tanzu support.

 

Regards,

Todd

 

[lakshmi mekala]

Hi Todd,

 

Thanks for your inputs., We are using nip.io powerdns to resolve the domain name. However on the go router page "Certificates and private keys for the Gorouter" section we have created a new certificate for domains

*.24.35.68.90.nip.io,*.systesm.24.35.68.90.nip.io, *.apps.24.35.68.90.nip.io,24.35.68.90.nip.io

Now replication canary ends up with below error:

 

 

 

 

goroutine 1 [running]:

code.cloudfoundry.org/lager.(*logger).Fatal(0xc0000601e0, {0x7b88b6, 0x1d}, {0x8435a0?, 0xc0001e32c0}, {0x0, 0x0, 0xc00009e360?})

/var/vcap/data/compile/replication-canary/replication-canary/vendor/code.cloudfoundry.org/lager/logger.go:162 +0x5c7

main.main()

/var/vcap/data/compile/replication-canary/replication-canary/main.go:87 +0x6aa

panic: Post "https://uaa.system.24.35.68.90.nip.io/oauth/token": tls: failed to verify certificate: x509: certificate signed by unknown authority

 

 

We need clarity on few points

• Do we need to purchase a domain for the small footprint TAS setup
• Do we need to purchase valid certificates

We want it to be like dev environment and not a production environment. Let me know how can we succeed with the setup

[lakshmi mekala]

Ip 24.35.68.90 is the IP address of Azure LB, which is mapped to router VM

[Todd Robbins]

Hi lakshmi, I don't think you need to purchase a domain or certificates. You mentioned:

 

we have created a new certificate for domains

*.24.35.68.90.nip.io,*.systesm.24.35.68.90.nip.io, *.apps.24.35.68.90.nip.io,24.35.68.90.nip.io

 

Do you have the root CA that was used to sign this new certificate? If so then you can simply add it to OpsManager > BOSH Director > Trusted Certificates. This will push your CA certificate out to all BOSH deployed VM's and should resolve "failed to verify certificate: x509: certificate signed by unknown authority" errors.

[lakshmi mekala]

Thanks Todd.., enabling include certs in security section of Bosh tile helped to resolve the issue.