Tanzu

 View Only
Expand all | Collapse all

a problem with installation TMC-SM 1.2

  • 1.  a problem with installation TMC-SM 1.2

    Posted 27 days ago

    Hello, 

    I wonder why TMC-SM isn't installed in a part of the package "tmc-local-stack" with a below error.

    So, I've investigated the pod "agent-gateway-server-58xxxxxxx". There is the below message

    and I've executed the below command to dive into the problem..

    $ kubectl logs -n tmc-local agent-gateway-server-58c85cfd47-7kvr7 -c agent-gateway

    below is the result..

    Could you please help me what problem here is?

    Thanks,

    WilliamKim



  • 2.  RE: a problem with installation TMC-SM 1.2

    Posted 24 days ago

    Hi,

    wonder have you configure so kapp-controller trusting your harbor? it is done before cluster creating

    https://docs.vmware.com/en/VMware-Tanzu-Mission-Control/1.2/tanzumc-sm-install/prepare-cluster.html




  • 3.  RE: a problem with installation TMC-SM 1.2

    Posted 23 days ago

    Hi,

    Sure, I've check whether kapp-controller using this command "kapp version"

    but I'm trying to deploy tmc within TKGm so all necessary components are working good. (For Example, Harbor and the below components)




  • 4.  RE: a problem with installation TMC-SM 1.2

    Posted 22 days ago

    Hello,

    if you logon to supervisor and if you type: kubectl get kappcontrollerconfigs.run.tanzu.vmware.com -n <tmc-sm-vSphereNamespace> (can you see if kapp-control is deployed)

    For both cluster and kapp-control must trust the harbor 




  • 5.  RE: a problem with installation TMC-SM 1.2

    Posted 22 days ago
    Edited by William Kim 22 days ago

    Hello, 

    I'm configuring TMC-SM in TKGm (not TKGs which is deployed in TKGs environment).

    So I was trying getting kapp-controllerconfig within the management cluster.

    And my customer want to use Active Directory with LDAPS as an Identify Provider(IDP).

    Thanks,

    William




  • 6.  RE: a problem with installation TMC-SM 1.2

    Posted 21 days ago
    Edited by JohanL1 21 days ago

    Hi,

    Sorry mean management cluster! :) forgot you have a TKGm great that solve. 

    have a example to use ldap you need to specify in your value.yaml file when you deploy tmc

    usergroupfilter and group is not mandatory to use! 

    authenticationType: ldap
    oidc:
      issuerType: pinniped
      issuerURL: https://pinniped-supervisor.<your DNS name>/provider/pinniped
    idpGroupRoles:
      admin: tmc-admin
      member: tmc-member
    ldap:
      type: activedirectory
      host: <FQDN to Domain controller>
      username: "CN=<svcaccount>,OU=path,OU=path,OU=path,DC=domain,DC=domain"
      password: "password"
      domainName: "domain-name"
      userBaseDN: "DC=domain,DC=domain"
      userSearchFilter: "(&(objectClass=person)(sAMAccountName={}))"
      groupBaseDN: "DC=domain,DC=domain"
      groupSearchFilter: "(&(objectClass=group)(member={}))"

      




  • 7.  RE: a problem with installation TMC-SM 1.2

    Posted 20 days ago

    Hello JohanL1,

    Yes, I've this beautiful values.yaml file. The deploying process is stuck at installing tmc-local-stack which is included tmc-sc package.

    harborProject: xxx-harbor3.tanzu.lab/tanzumc-1.2
    dnsZone: xxx.wilm.lab
    clusterIssuer: local-issuer
    postgres:
      userPassword: xxx
      maxConnections: 300
    minio:
      username: root
      password: xxx
    contourEnvoy:
      serviceType: LoadBalancer
      serviceAnnotations: # needed only when specifying load balancer controller specific config like preferred IP
        ako.vmware.com/load-balancer-ip: "10.200.29.207"
      # when using an auto-assigned IP instead of a preferred IP, please use the following key instead of the serviceAnnotations above
      # loadBalancerClass: local
    authenticationType: ldap
    oidc:
      issuerType: pinniped
      issuerURL: https://pinniped-supervisor.xxx.wilm.lab/provider/pinniped
    idpGroupRoles:
      admin: tmc-admin
      member: tmc-kdw
    ldap:
      type: activedirectory
      host: "*.xxx.wilm.lab"
      username: "CN=*.xxx.wilm.lab SvcAcct,DC=wilm,DC=lab"
      # username: "CN=*.xxx.wilm.lab SvcAcct,OU=tmc,DC=wilm,DC=lab"
      password: "xxx"
      domainName: "xxx.wilm.lab"
      userBaseDN: "DC=wilm,DC=lab"
      userSearchFilter: "(&(objectClass=person)(sAMAccountName={}))"
      userSearchAttributeUsername: sAMAccountName
      groupBaseDN: "DC=wilm,DC=lab"
      groupSearchFilter: "(&(objectClass=group)(member={}))"
      rootCA:
        -----BEGIN CERTIFICATE-----
        MIIDbzCCAl~~~~
        -----END CERTIFICATE-----
    # alertmanager: # needed only if you want to turn on alerting
    #  criticalAlertReceiver:
    #    slack_configs:
    #    - send_resolved: false
    #      api_url: https://hooks.slack.com/services/...
    #      channel: '#<slack-channel-name>'
    telemetry:
      ceipOptIn: true
      #  eanNumber: <vmware-ean> # if EAN is available
      ceipAgreement: true
    size: small

    Thanks,

    WilliamKim




  • 8.  RE: a problem with installation TMC-SM 1.2

    Posted 20 days ago
    Edited by Jason McClellan 17 days ago

    Hello,

    Yes I've a values.yaml file to deploying tmc-sc

    Is this right session for deploy tmc with AD and Ldaps? I would like to verify the sessions in this file such as "harborProject", "dnsZone", "clustrerIssuer", "prostgres", "minio" "authenticationType", "oidc", "idpGroupRole", "ldap" etc..

    when I deployed TMC-SM packages, the deploying process is stuck at installing "tmc-local-stack" (Using the below values.yaml)

    harborProject: xxx-harbor3.tanzu.lab/tanzumc-1.2
    dnsZone: xxx.wilm.lab
    clusterIssuer: local-issuer
    postgres:
      userPassword: ********!
      maxConnections: 300
    minio:
      username: root
      password: ********!
    contourEnvoy:
      serviceType: LoadBalancer
      serviceAnnotations: # needed only when specifying load balancer controller specific config like preferred IP
        ako.vmware.com/load-balancer-ip: "10.200.xx.xxx"
      # when using an auto-assigned IP instead of a preferred IP, please use the following key instead of the serviceAnnotations above
      # loadBalancerClass: local
    authenticationType: ldap
    oidc:
      issuerType: pinniped
      issuerURL: https://pinniped-supervisor.xxx.wilm.lab/provider/pinniped
    idpGroupRoles:
      admin: tmc-admin
      member: tmc-xxx
    ldap:
      type: activedirectory
      host: "*.xxx.wilm.lab"
      username: "CN=*.xxx.wilm.lab SvcAcct,DC=wilm,DC=lab"
      # username: "CN=*.xxx.wilm.lab SvcAcct,OU=tmc,DC=wilm,DC=lab"
      password: "Twotech1!"
      domainName: "xxx.wilm.lab"
      userBaseDN: "DC=wilm,DC=lab"
      userSearchFilter: "(&(objectClass=person)(sAMAccountName={}))"
      userSearchAttributeUsername: sAMAccountName
      groupBaseDN: "DC=wilm,DC=lab"
      groupSearchFilter: "(&(objectClass=group)(member={}))"
      rootCA:
        -----BEGIN CERTIFICATE-----
        MIID.......
        -----END CERTIFICATE-----
    # alertmanager: # needed only if you want to turn on alerting
    #  criticalAlertReceiver:
    #    slack_configs:
    #    - send_resolved: false
    #      api_url: https://hooks.slack.com/services/...
    #      channel: '#<slack-channel-name>'
    telemetry:
      ceipOptIn: true
      #  eanNumber: <vmware-ean> # if EAN is available
      ceipAgreement: true
    size: small
    




  • 9.  RE: a problem with installation TMC-SM 1.2

    Posted 20 days ago
    Edited by Jason McClellan 17 days ago

    Looks correct but with type of Loadbalancer do you using? if you using NSX ALB then is correct, have not test tmc-sm with NSX ALB yet :) 

    and not sure about with type certificate you using but in my case i using self-sign certificate so have too add these value:

    trustedCAs:
      local-ca.pem: | # root CA cert of the cluster issuer in cert-manager, if not a well-known CA
        -----BEGIN CERTIFICATE-----
        .....
        -----END CERTIFICATE-----
      harbor-ca.pem: |
        -----BEGIN CERTIFICATE-----
        ....
        -----END CERTIFICATE-----




  • 10.  RE: a problem with installation TMC-SM 1.2

    Broadcom Employee
    Posted 21 days ago

    Hello WilliamKim, 

    Please validate that you have the proper DNS entries configured for 'gts.kdw-tmc.wilm.lab', as well as all of the other TMC endpoints. 

    I have seen the same error previously and the root cause was missing DNS entries.

    Here is the list of needed entries:

    Create the following type A records in your DNS server and point them to a preferred IP in your load balancer's IP pool.

    • <my-tmc-dns-zone>
    • alertmanager.<my-tmc-dns-zone>
    • auth.<my-tmc-dns-zone>
    • blob.<my-tmc-dns-zone>
    • console.s3.<my-tmc-dns-zone>
    • gts-rest.<my-tmc-dns-zone>
    • gts.<my-tmc-dns-zone>
    • landing.<my-tmc-dns-zone>
    • pinniped-supervisor.<my-tmc-dns-zone>
    • prometheus.<my-tmc-dns-zone>
    • s3.<my-tmc-dns-zone>
    • tmc-local.s3.<my-tmc-dns-zone>

    Hopefully this helps. 

    Regards,



    ------------------------------
    Corey Dinkens
    Product Marketing Engineer | VMware Tanzu Marketing

    corey.dinkens@broadcom.com
    ------------------------------



  • 11.  RE: a problem with installation TMC-SM 1.2

    Posted 21 days ago

    Hello Corey Dinkens,

    I've configured the DNS records with wildcard. and ping tests are going well to query the all required dns records.

    I'd like to ask you which components should I deep dive into in? at the below screenshot?

    should the tmc-sc be deployed in management cluster? I think in workload cluster.

    Thanks,

    WilliamKim




  • 12.  RE: a problem with installation TMC-SM 1.2

    Broadcom Employee
    Posted 20 days ago

    WilliamKim,

    You are correct that SM should be deployed onto a workload cluster. Thanks for checking the DNS records - out of curiosity did you validate them from your workstation, or from the cluster having issues? If you checked from your workstation, I would also ask that you try to ssh into the cluster and validate as well. 

    Because of how TMC SM was designed, generally speaking when you see pods failing as you are now, it is due to one of the primary TMC pods failing and the failure is cascading - so in this case it's likely due to the tmc-local-stack failing to reconcile. 

    So here are a few other items for you to check:

    • I found an old internal thread where a user needed to manually add the DNS entry for S3, as wildcard was not resolving for some reason, and that fixed their issue.
      • I do not recall seeing this any other time, so not sure if this was an issue specific to the DNS server being used?
    • If DNS resolution fails from the cluster:
      • Make sure there is no time drift between cluster and DNS server
      • Verify traffic is not being dropped by a firewall
    • If you created the DNS records after trying to deploy TMC SM, you might need to restart some of the pods that are part of the tmc-local-stack deployment
    • If you are attempting a re-install, there is a known issue where not all of the TMC objects are cleaned up on removal, so you want to ensure to delete the tmc-local namespace between deployments 
    • Validate the services and see what has instantiated and that the cluster is actually able to use the LB IP of 
      10.200.29.207

      If the LB IP is not getting assigned, this could be another source of cascading failures

    If none of the above helps, I would suggest checking (and sharing if possible) the logs for tenancy-service-server and api-gateway-server. I am fairly certain the gts service is tied to the api-gateway-server.

    Hopefully this helps.. let us know what you figure out.

    Regards,



    ------------------------------
    Corey Dinkens
    Product Marketing Engineer | VMware Tanzu Marketing

    corey.dinkens@broadcom.com
    ------------------------------



  • 13.  RE: a problem with installation TMC-SM 1.2

    Posted 17 days ago
    Edited by Jason McClellan 17 days ago

    Hello Corey,

    Thanks for your kindly comments.

    I tested the both and it was working good. And I've configured the DNS records before deploying TMC-SM.

    As I saw LB in AVI GUI, the Virtual Service was up with green status.

    And I could access TMC-SM webpage and then I clicked the button "login" after that the page wasn't shown with the error code 503.

    At this time, I couldn't access the webpage with URL "https://tmc.wilm.lab/landing.

    the message is shown "no healthy upstream" after clicking "SIGN IN" button.

    Could you please guide or describe more details related with the old internal thread?
    I added DNS records only s3.xxx.wilm.lab and the others is configured with wildcard.

    Regards,

    William




  • 14.  RE: a problem with installation TMC-SM 1.2

    Broadcom Employee
    Posted 15 days ago

    William,

    Apologies for the delay. The only additional details from the thread I mentioned were manually creating the S3 record - if you have created one, I believe you would probably also want to restart any failed pods to see if things continue to start up as normal. 

    If you want to eliminate the wildcard as the source of the issue, you can manually add the following DNS A records and point them to the LB address of your SM instance:

    • alertmanager.<my-tmc-dns-zone>
    • auth.<my-tmc-dns-zone>
    • blob.<my-tmc-dns-zone>
    • console.s3.<my-tmc-dns-zone>
    • gts-rest.<my-tmc-dns-zone>
    • gts.<my-tmc-dns-zone>
    • landing.<my-tmc-dns-zone>
    • pinniped-supervisor.<my-tmc-dns-zone>
    • prometheus.<my-tmc-dns-zone>
    • s3.<my-tmc-dns-zone>
    • tmc-local.s3.<my-tmc-dns-zone>

    Regards,



    ------------------------------
    Corey Dinkens
    Product Marketing Engineer | VMware Tanzu Marketing

    corey.dinkens@broadcom.com
    ------------------------------