Scripts fail Through CA DE, but on Linux Complete Successfully

[David Lock]

Scripts fail Through CA DE, but on Linux Complete Successfully

I write and test a Python script on Linux directly and all works great.
I use #!/bin/python etc
I test running using python ./scriptName <Args> and runs successfully
I test running just ./scriptName <ARGS> and runs successfully

When I hook it up in CA DE UNIX job it fails.
with line 3: import: command not found

I have tried running as script with /bin/bash shell and command defining python path and Python script and args as params

Why does it fail within the UNIX job when it runs successfully in Linux

I test and all is good, configure in CA DE and it fails.
It's very frustrating.

[Don Powell]

Hi, 
The below should work. 
in the SCRIPTNAME put the location of Python. 

SCRIPTNAME /usr/bin/python

In the args put the location of the script and then the args. 

ARGS /opt/scripts/python/test.py argshere

2¢
Don

[David Lock]

Hi Don,

Thanks, I really appreciate your input here.

Yes I did that and I get different output to when I run the script manually.
I am connecting to a SFTP Server.
and get this if run through CA DE
IO Excepton encountered: Bad authentication type; allowed types: ['publickey']

If I run it manually with Password it completes successfully without issue and does not indicate public key at all.

Odd I get different outcome.

[Don Powell]

Hi, 
At least now the script is running. Just a few thoughts

I will ask.....Is there a reason you are using a Python script instead of the SFTP job?  Why reinvent the wheel... 

The error message says only publickey is allowed.  Have you set up keys? 
Is it running as the same user as when you initiate it manually? (is there a user defined in the job) 
Are there relative paths in the script.  They should be fully qualified.  They should start at the beginning like /opt/....  or /etc/...   

2¢
Don

[David Lock]

Hi Don,

Well, the SFTP job doesn't exactly do just what I want.  
I have a script that just checks if a file exists, If I used a SFTP job I would have to download a file to tell me it was there, and if the SFTP job failed for a reason that could be for I/O error etc and not just telling me the file is not present.  Yes I would love that feature in the SFTP job.   On that note I have an issue running in the new 12.2 version where it indicates cannot find or load jars, but that's under another discussion.

This running a Python script manually logs in fine with password and not through DE which frustrates me as you cannot assume or guarantee it will run even if it it runs and validates on a manual run.  Head scratch.

I have not set up keys yet as I have not had to do so.   As for paths yes full paths.

Again, I appreciate your input and help.

[Nitin Pande]

Hi,
You will need to add /bin/python to the agentparm.txt.
See this link for oscomponent.validshell.
Then use the #!/bin/python

HTH,
Nitin Pande​

------------------------------
Support
Broadcom
Toronto
------------------------------

[David Lock]

Hi Nitin,

Thanks for your reply.

1. Adding /bin/python to the validshell is odd.
   1. Python is not a shell?
   2. It had the same result as running as a command and run /bin/python <Script> <Args>
2. I get different behavior from a shell cmd line or via DE UNIX job.
   1. Shell run at the command line is successful.   Connects to SFTP server and checks a file exists without asking for a Public Key and uses password Authentication.
   2. Run in a CA DE UNIX job and I get IO Excepton encountered: Bad authentication type; allowed types: ['publickey']
3. I am using Python paramiko
   1. transport=paramiko.Transport("<HOST>",22)
      transport.connect(username=uName,password=decrypted)
      sftp=paramiko.SFTPClient.from_transport(transport)

To me, the CA DE UNIX job is somehow forcing it to use PUBLIC key.

1. Perhaps hostname needs to be put some where?
2. There is different behavior  between the two.
3. Sure I could go down the path to use a public key, but right now I'm trying to understand why the differences in execution.

[David Lock]

Hi Nitin,

OK, I can replicate on both the command line and CA DE UNIX.

If I align the command line to the CA DE UNIX and mimic CAWA running it

1. /bin/python /home/<USER>/bin/checkSFTPFileExists.py <ARGS>
   • I get  
     • IO Excepton encountered: Bad authentication type; allowed types: ['publickey']
2. If I run in CAWA I get the same
   • IO Excepton encountered: Bad authentication type; allowed types: ['publickey']
3. If I cd to /home/<USER>/bin
   • Run the command from there it runs successfully and authenticates using password.

So it's something to do with where I am issuing the command from and it requiring publickey

[Don Powell]

Hi, 
A couple of odd thoughts. 
If you move your script to some other directory, does it work? 

Off the top of my head I don't know the equivalent Python command to the "env" command in a shell script to see the environment variables. Obviously there is something different. 

It could be one of the things below.  
There is something in that /home/<USER>/bin directory that makes it work correctly.  You could add that path to the path variable.
There is a relative path instead of a full path somewhere. It can't find what it needs. 

As an odd thought to try out in test you could add the initialization parm below to the agentparm.txt file and restart the agent.  

• oscomponent.initialworkingdirectory=/opt/..... 

I think it would be better to figure out what it really needs rather than changing the initial working directory. 
2¢
Don

[Don Powell]

I should have included the link to this in the doc. There are a few options to this parameter... 
https://techdocs.broadcom.com/us/en/ca-enterprise-software/intelligent-automation/workload-automation-system-agent/12-0/reference/ca-wa-agent-for-unix-linux-windows-or-iseries-agent-parameters.html