Autosys environment

[Loki N]

I have 2 servers, let us say server A and server B

I ran the following command to check the Autosys environment on both the servers via Autosys job

command: env | mail -s "env" [e-mail]

 

From the output I received:

On server A I see the environment variable defined eg: ABC=123

On server B I don’t see that environmental variable being picked up by Autosys

 

Those variable are defined under /etc/profile.d/*sh on both the servers

Do you know why Autosys is picking the variable from one server and not on the other?

 

I checked the agent.paramtxt file and see all the attributes are default and similar on both servers

I am also aware that Autosys doesn't read .profile and .bashrc, but still wanted to know how  ABC=123 was  picked up by Autosys. 

 

Thanks

Loki

[Legacy User]

Hello Loki

If you are running the same command with same owner in the job definition and the env command does not return the same output, means to me that you are running the System Agent with Manager specific variables as described at below url:

 

Define Environment Variables - CA Workload Automation System Agent - 11.4 - CA Technologies Documentation 

 

For example on mine:

# Added by WAAE Add Instance
#
oscomponent.environment.variable_manager_ORA_SCH=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/profiles/ORA.txt

 

Here is means that when any job comes from the ORA instance, the System Agent is going to look into this ORA.txt file

If a job comes from another instance, it is not going to look into this file.

 

There is a more global option which is to set:

 

# Agent-wide environment variables
#
oscomponent.environment.variable=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/profiles/WAAE.txt

 

With this setting, the System Agent will be picking up variables in file WAAE.txt for any job coming from any instance

 

And as a reminder, you cannot reference variables by other variables ( like TEST1=$VAR1 ) but you have to hard code them like

 

more WAAE.txt
AUTOSYS=/opt/CA/WorkloadAutomationAE/autosys
AUTOROOT=/opt/CA/WorkloadAutomationAE

 

Regards

Jean Paul

[Loki N]

Hi Jean

 

Thanks for your reply.

Below are some more details.

 

Version of agent is 11.3, Service Pack 6 on both the hosts.

I did check agentparam.txt on both the servers before posting this and I didnt see any differences at all, below I have pasted the same for both the machines.

I am running the job too on the same instance.

 

Server A

oscomponent.libjvmpath=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/jre/lib/i386/libjava.so

oscomponent.classpath=jars/*.jar:jars/ext/*

oscomponent.loginshell=false

oscomponent.defaultshell=/bin/sh

oscomponent.validshell=/usr/bin/sh,/bin/csh,/bin/ksh,/bin/sh,/bin/bash

oscomponent.checkvalidshell=true

oscomponent.lookupcommand=true

oscomponent.joblog=true

oscomponent.environment.variable=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/profiles/WAAE.txt

oscomponent.environment.variable_manager_PR1_SCH=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/profiles/PR1.txt

oscomponent.joblog.success.autocleanup=true

oscomponent.noexitcode=256

oscomponent.auth.pam.svc=sshd

oscomponent.cmdprefix.force=true

oscomponent.initialworkingdirectory=USER_CONT_NOT_AVAIL

oscomponent.noforceprofile=true

oscomponent.profiles.src.delay=true

oscomponent.profiles.src.order.global.first=true

oscomponent.profiles.src.verify=true

oscomponent.environment.variable_manager_DV1_SCH=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/profiles/DV1.txt

 

Server B

oscomponent.libjvmpath=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/jre/lib/i386/libjava.so

oscomponent.classpath=jars/*.jar:jars/ext/*

oscomponent.loginshell=false

oscomponent.defaultshell=/bin/sh

oscomponent.validshell=/usr/bin/sh,/bin/csh,/bin/ksh,/bin/sh,/bin/bash

oscomponent.checkvalidshell=true

oscomponent.lookupcommand=true

oscomponent.joblog=true

oscomponent.environment.variable=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/profiles/WAAE.txt

oscomponent.environment.variable_manager_PR1_SCH=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/profiles/PR1.txt

oscomponent.joblog.success.autocleanup=true

oscomponent.noexitcode=256

oscomponent.auth.pam.svc=sshd

oscomponent.cmdprefix.force=true

oscomponent.initialworkingdirectory=USER_CONT_NOT_AVAIL

oscomponent.noforceprofile=true

oscomponent.profiles.src.delay=true

oscomponent.profiles.src.order.global.first=true

oscomponent.profiles.src.verify=true

oscomponent.environment.variable_manager_DV1_SCH=/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/profiles/DV1.txt

 

Thanks

Loki

[Loki N]

Hi Jean

The issue was resolved post bouncing the agent services.

now, does autosys really read /etc/profile.d/*sh ?

 

I read below in CA doc which says:

Agent switches the job’s owner to the specified user by using the “setuid” system call. This changes the owner of the running process to the user, but doesnot change the user’s environment. Specifically, it does not read the user’s “.profile”, “.login” or other file that typically sets up the environment for a user. So the basic environment is still that of the user that started the System Agent, usually “root”.”

 

Thanks

Loki

[Legacy User]

Hi Loki

At this stage it is now important to know where did you set ABC=123?

 

Is it in your /etc/auto.profile file because in this profile you can also test your instance name with $AUTOSERV  and take different actions

Regards

Jean Paul

 

[Loki N]

Hi Jean

 

Its not in /etc/auto.profile  ( none touched it after initial creation)

 

Here is some background:

I have few users running the jobs on server A and B.

On Server A the  jobs are running fine without any issues.

On server B they get the error "$ABC" not defined. I did investigate and requested them to explicitly define the variable in AutoSys via profile or envvars attribute.

 

The variables are defined under /etc/profile.d/*sh which is also same for both the servers (A and B)

Now, I did say them that Autosys does not look for .profile* files to get the variables, they still wanted to know why it is picking on A and why not B.

 

Thanks

Loki

[Legacy User]

Hi Loki

Better to open a case in support and we can look into your machines in a remote session.

Regards

Jean Paul

[Steve Carrobis]

*sigh*

 

Loki

in the job where did you set the profile: attribute to? 

if you are using the user's login profile make sure you do not have that agent loaded as legacy otherwise it will not source the login profile. 

This is all basic autosys job process.. with these new agents and behavior everyone forgets to use profile attribute for FW and CMD.

or to check teh .profile of the login id in that case/.

 

auto.profile is there only for AUTOSYS variable overrides. 

 

check the agentparm for noprofile and srcdelay. both should be false. 

 

hope that helps 

 

Steve C.

[Loki N]

Hi Steve.

 

Thanks for your inputs. we are not using profile in the "job"

Both the above mentioned attributes are set as "true", which means Agent doesn't outsource user's .profile or .logins

 

we re-started the agent services and agent was able to source the variables in  /etc/profile.d/*sh. do you know why?

As the below attributes are set to "true" I still believe agent shouldn't be sourcing it.

 

oscomponent.profiles.src.delay=true

oscomponent.profiles.src.order.global.first=true

 

 

Thanks

loki

[Steve Carrobis]

If a job is not respecting variables in the profile listed in the profile attribute check on these settings in system agent

 

      

•   legacy mode is chosen:

         

•   oscomponent.noforceprofile=true

         

•   oscomponent.profiles.src.delay=true

      

•   legacy is not chosen and you need the user login profile sourced then you need to make sure you do not delay the profile.

         

•   oscomponent.noforceprofile=false

         

•   oscomponent.profiles.src.delay=false

 

 

 

 

 

Steve C.

 

 

Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.

 

Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.

[Loki N]

Thanks Steve.

I will check on the mentioned attributes and their behavior. 

[Scott Fowler]

Reading through the comments, there seems to be one important thing that hasn't been mentioned.  A job environment will also include any variables that were picked up by the agent during its startup. These will vary depending on how the agent is started.

When the agent is started during system boot, or is started by using the unisrvcntr command, it is getting started within a non-interactive shell.

When the agent is started by directly executing the startup script in /etc/init.d, or is started by running 'cybAgent -a' from the agent installation directory, it is being started within an interactive shell.

Typically, the scripts under /etc/profile.d are only going to be sourced for an interactive shell.

You mentioned that the variable started showing up when you bounced the agent. Assuming nothing else changed in the agent config or job profiles, etc., the fact that it just started showing up after an agent restart tells me that it was picked up by the agent during the startup and became part of the default environment within the agent.

So the question would be, when you restarted the agent, which method did you use to do that?

 

All that said, I agree that you should encourage your users to utilize job profiles, etc. and not rely on environment scripts that are typically meant for interactive shells.

[Loki N]

Hi Scott

 

* We recently upgraded agent from 11 to 11.3.6 on the 2 servers, let us say A and B.

* There was an issue raised by the user saying one of the variable was not sourced on B  (ABC=123) post upgrade.

* We did investigate and requested the user to explicitly define the variable using envvars or profile attribute in the job definition and also advised that AutoSys doesn't source user's profile and login's.

* User wanted to know why the same is being sourced on the machine A and not on B.

* I created test job on both the machines A and B with the command: `env | mail  -s "env"  mymail.com`

* I could see agent sourced ABC=123 on machine A and not on machine B

* Assuming that there could be some configurations changes at agent level I did a comparison of agent parameters on both the machines (A and B), but no differences found, all the parameters set were similar.

* With no option left, I finally decided to bounce the agent using "unisrvnctr" command

* After bounce, I ran same set of test jobs (`command: env | mail  -s "env"  mymail.com`) and could see ABC=123 was sourced out on B as well.

 

I feel agent does source .profile only once i.e during agent re-start, so any changes you make to the .profile or /etc/profile.d/*sh files agent need to be bounced again to pick up those values?

 

Thanks 

Loki

[Steve Carrobis]

did you check the agent parm settings? and you shouldnt allow users to update the /etc profiles these are system settings. 

my previous post explained what needs to be set for both lofin profile and profile attrinute to be set so that the user will have their variables set.

 

Thank you 

Steve C.

[Chandru.Venkataraman]

Loki,

Steve and Scott have already explained how the agent behaves with different parameters set and how the Agent service is started.

I'd like to just hilite the difference between the system (init process) starting the Agent at boot, and a user starting the Agent using service or unisrvcntr commands.

When the init process starts the Agent (as per run control scripts in /etc/rc.d/), it does NOT source the /etc/profile and so the application-specific environment under /etc/profile.d/*) won't be available to the System Agent service.

 

As one can see, the /etc/profile file sources /etc/profile.d/*:

 

Excerpt from /etc/profile (on RHEL)

for i in /etc/profile.d/*.sh ; do
    if [ -r "$i" ]; then
        if [ "${-#*i}" != "$-" ]; then
            . "$i"
        else
            . "$i" >/dev/null
        fi
    fi
done

 

When a user starts the Agent using service or unisrvcntr command, the shell (bash or Bourne) has already sourced the /etc/profile file (and thereby, the /etc/profile.d/* files) and so the Agent inherits the current  shell environment (parent process).

 

So, the next time the server is rebooted and the System Agent is started by the init process, the environment variable ABC won't be available. That is something to keep in mind.

 

I hope that helps!

 

Cheers,

Chandru

[Loki N]

Thanks Steve and Chandru.

The above explanation really helps, thanks for making me understand better.

 

Regards

Loki

[Keith Maconi]

Hi,

 

   The explanation is clear, but does that not beg the question of why CA does not ensure consistent initialization of the agent?  You are basically stating that the only proper way to start the agent is by rebooting the server.

 

Thanks,

Keith