What "pksnsxcli create lb" does as part of pre-start script

[Shanthakumar Karuppusamy]

I would like to know the operations "pksnsxcli create lb" does as part of pre-start script .

[Mark Nagle]

Hi Shanthakumar,

 

I looked into this and my assumption is your talking about the pks-nsx-t-prepare-master-vm job which fails when trying to run the pre-start script in your env. From the bosh jobs it looks like the pre-start script sets env variables for all the certs needed to connect to NSX-T and also the NSX-T connections details. After this it using the pksnsxcli to talk to NSX Manager and tag to pks cluster bosh id to the pks master vm.

 

This function tags a logical port, which is tagged by provided bosh id, with NSX_T_NCP_TAG_KEY_K8SMASTERVM

i don't see any "pksnsxcli create lb" command in this pre-start script. Can you supply further information about where you see this?

 

Regards,

Mark Nagle

[Shanthakumar Karuppusamy]

yes u are right, this query is WRT to the other issue and I want to know what are all the operations lb does as part of the command /var/vcap/jobs/pks-nsx-t-prepare-master-vm/bin/pre-start under master we can find the

proxy_create_lb='false' lbname='lb-pks-d5e63bb8-db98-441e-8f10-4d29ced7e7d9'   if [[ -n $lbname && -n $proxy_create_lb && $proxy_create_lb == "false" ]]; then   echo "Creating Load Balancer"   $pksnsxcli create lb --instance-id="${K8S_CLUSTER_ID}" \ -c "${NSX_MANAGER_CLIENT_CERT_FILE}" \ -k "${NSX_MANAGER_CLIENT_KEY_FILE}" \ --t0-router-id='5cb589b0-b599-4e1e-84f6-6ad048739a0d' \ --loadbalancer-name='lb-pks-d5e63bb8-db98-441e-8f10-4d29ced7e7d9' \ --nsx-ca-cert-path="${NSX_MANAGER_CA_CERT_FILE}" \ --insecure='false' \ --nsx-manager-host='sfo01w01nsx01.sfo01.rainpole.local' || exit $?   echo "Load Balancer is created" fi  

 

[Mark Nagle]

Hi Shanthakumar,

 

I am not sure of the breakdown of the operation level but the purpose of this job is to create a LB to access the kube-api server on the master nodes. What is the error you see when the LB is being created?

 

Regards,

Mark Nagle

[Mustafa Bayramov]

I have following issue.

 

Registering client certificate

b49d8e41-68b8-41e3-8fe4-15de9dbe022a

Registration of client certificate is successful

Checking if client certificate is ready to be used

Client certificate is ready to be used

Creating Load Balancer

unknown error (status 201): {resp:0xc420312990}

unknown error (status 201): {resp:0xc420312990}

 

and in stderr.

 

master/994cda09-ec01-4dff-9584-4af434bd1c27:/var/vcap/sys/log/pks-nsx-t-prepare-master-vm# more pre-start.stderr.log

ResourceDeleteFunc(): unrecognized resource type: LbProfile

time="2018-09-28T22:06:54Z" level=error msg="Failed to CreateLbTcpMonitor: &{LbActiveMonitor:{LbMonitor:{ManagedResource:{RevisionedResource:{Resource:{Links:[] Schema: Self:<nil>} Revision:<nil

>} CreateTime:0 CreateUser: LastModifiedTime:0 LastModifiedUser: SystemOwned:<nil> Description: DisplayName:lb-pks-5b82f22b-78bd-45b4-9b5f-acd778269a22 ID: ResourceType: Tags:[0xc42022c120]} Res

ourceType:LbTcpMonitor} FallCount:3 Interval:10 MonitorPort:8443 RiseCount:3 Timeout:10} Receive: Send:}\n" pks-networking=networkManager

Error: unknown error (status 201): {resp:0xc420312990}

Usage:

 pksnsxcli create [flags]

 

Flags:

   --floating-ip string     PKS cluster's instance UUID

 -h, --help            help for create

   --instance-id string     PKS cluster's instance UUID

   --loadbalancer-name string  Name of the load balancer

 

Global Flags:

 -c, --client-cert-path string  Client certificate for communication with NSX manager

 -k, --client-key-path string  Client private key for communication with NSX manager

   --insecure         Do not validate NSX manager's certificate (default true)

   --nsx-ca-cert-path string  NSX manager's CA certificate path

   --nsx-manager-host string  Hostname of NSX manager

   --t0-router-id string    T0 router ID