Postconditions not functioning - AE 11.2.2

[Scott Wirth]

I am running into an odd problem with postconditions in AE 11.2.2.  Our organization is new to Automic, and it has been a bit of a slog, since there's a small group of us doing the conversion of items from the old scheduling system ( Tidal ) to Automic. 

 

We have a process that for lack of a better term, consumes files.   The idea is that it processes a certain amount of files based on date/number/size.  The number of files processed for each iteration can be variable, and there are different variants, based on whether it's being based on date or number/size.  Essentially, it may require a variable number of iterations to process all the files, at which point, it should stop.

 

To that end, I've created a workflow which handles a single iteration of the process.  That process works fine, and I can run as many as necessary to process the files.  Where the issue occurs, is getting this to occur repeatedly until all files have been processed.  To this end, there is a 'wrapper' workflow.   The wrapper workflow contains only the single iteration workflow, which has a postcondition that looks in the location where the files are being supplied. 

 

The problem occurs that when I run this wrapper workflow, the postcondition check occurs, and I get:

 

2018-06-01 11:58:49 - IF file '\\SOURCEPATH\FOR\FILES\*' does not exist (check via SERVERAGENT).

2018-06-01 11:58:49 - U00011700 Cannot find information for path '\\SOURCEPATH\FOR\FILES\*'.

2018-06-01 11:58:49 - U00020411 The command 'EXIT' in 'XC_INC.CONDITION.CHECK_FILE', line '00086' stopped the generation of this task.

At which point, the external workflow breaks.

 

Where I am having difficulty is that a workflow that I have set up in the development environment which is conceptually exactly the same (nested workflows and a file postcondition on the internal workflow) behaves flawlessly.  In the environment where it is failing, I've verified with the server engineers that the AD (Active Directory) permissions for all executables in the single iteration workflow are correct, and that is borne out by the fact that running the single iteration workflow consumes files as necessary, and completes ENDED_OK.  The service accounts have all the proper permissions to the directory structure, as well. 

 

I think there is an issue with configuration somewhere, but I am at a loss to determine where it is.  Has anyone had a similar problem or any thoughts on whether there is a configuration item or a UC_SETTINGS variable that we may be missing?

 

We have a workaround, but the fact that it works fine in the development environment and not in the test environment is frustrating, since the hopeful end state of the conversion is being able to promote items from Dev to Test, and from Test to Prod ( this is a frustrating and far more manual process in the old system). So it makes sense that with the exception of the hosts/agents, the configurations should be little different or even identical in Automic.

 

Scott

 

[Pete Wirfs]

You could try this experiment to see if maybe this is still just a credential problem. (I know you believe you've eliminated the possibility, but sometimes we are wrong on such assumptions.)

 

In your DEV environment that is currently working, temporarily remove the grant that lets this work, and run a test to see if you get the identical error messages that you are getting from TEST.  If the error messages are the same, then you will know that you are still fighting an AD grant issue.

 

 

Also regarding changing host/agents when you promote Automic code, you can eliminate this by using HOST GROUPs in your job.  Use identical HOST GROUP names in all clients, but in each client the target HOST GROUP can be coded to point to only the desired agent for that client.

[Pete Wirfs]

One "gotcha" about HOST GROUPs.  There are a few UC4 object types that don't support them.

[FrankMuffke]

I second Petes theory that its a credentials issue.

 

Possibly its another user or th euser who starts the agent has different privileges in both environments.

 

Using UNC paths is often kinda nightmare with AE until you get it to work.

We had a similar issue using file events lately - we got an access error message in the file event. using a WIN job on the same agent with the same file ran fine.

Solution was doing a net use UNC PATH prior to the file event.

 

So if you do a net use (or map  a network drive in a job prior to the postcondition) - does the postcondition UNC check work?

 

cheers, Wolfgang

[Scott Wirth]

Wolfgang,

 

The weird thing is that I have administrator rights in all environments.   Can't check the Prod environment, as we've not even started conversion, but one of the other three admins has confirmed that he sees the same things, and has the unrestricted access level associated with administrators, as well.  I'm not lead on the project, and due to the way we overlap, I'm not always fully in the loop, but it's been confirmed by at least one of the other two administrators.  I will see if the NET USE works for us.   It just seems weird that it has no problem resolving the UNC path in Dev, but balks in Test.  That's why I was thinking that it was some setting or something in  a UC_SETTING variable that we may have overlooked.

 

I'm sure that we're likely to start abstracting to groups and the like, but first we have to get things working properly!  

 

Peter, do you recall what object types don't support the Host Groups, off the top of your head?

[Pete Wirfs]

Part of your challenge is understanding which AD credentials are in play.  If you are checking for file existence inside of a process script of a job, then it would be using the credentials that were used to authenticate to the targeted agent.  But if you are checking for file existence as part of a workflow pre-condition, then it is the credentials that the AE is running under - pre-condition executes prior to connecting to the agent.

 

I seem to remember that JOBF objects have a problem with host groups?  They always require two host specifications for FROM & TO.

 

In our particular situation some of my V11.2.1 windows jobs have to hard code their agent too.  This happens when the process script dynamically switches between DOS and Powershell mode, and it can get confused about what version the agent is running under because it can't tell via a host group. (this is a product bug in my opinion)

 

Others may know of other host group limitations...

[Scott Wirth]

Wolfgang,

 

Didn't get a chance to test this until today ( car trouble, and odd work schedule), but I tried this, and I'm still running into the same problem.

 

I'd placed the internal workflow between two JOBS objects.   The leading object uses NET USE N: \\unc\path /p:y to set a persistent network drive, and the trailing object uses NET USE N: /Delete in order to tear down the mapping.  So far so good.  Unfortunately, it still fails on the postcondition, whether I use the UNC path, or the drive letter.

 

 

2018-06-14 14:51:36 - IF file 'n:\*' does not exist (check via PHPIBIZ01).
2018-06-14 14:51:35 - U00011700 Cannot find information for path 'n:\*'.
2018-06-14 14:51:35 - U00020411 The command 'EXIT' in 'XC_INC.CONDITION.CHECK_FILE', line '00086' stopped the generation of this task.

We've got a workaround ( recurring item with a gap, and an event to trigger the workflow), but this still stumps me.  Server engineer went over AD permissions with me several times and they are all the same between the DEV and Test environments, so the only thing that I can come up with is some sort of configuration difference between the two environments.  The other two folks I'm working with have dealt with environments the most, so I will need to do some further studying.  Maybe export the client and do a diff of the XML to see where there may be a difference in configuration items.  

 

Scott

[PatrickHiggins607207]

An approach that has worked for me.

 

An initial step reads the source directory and populates a VARA object with a row for each acceptable file (filter on extension, file size, file name format , whatever, even using content of the file as part of the criteria)

 

A second job reads that VARA (really a limited format table inside the Automic schema) and if a file is found and not flagged as consumed, it launches the process to consume the file (this could be a job or a workflow).  Last part of the consumer process updates a status column entry for that row as completed.  If needed, the source file could be deleted or copied to an archive location.

 

This second job COULD simply read thru the vara and launch a separate consumer for each row all at once (using limits you define), or the consumer could simply end with a request to start the second job again.

 

This avoids using the POST condition to manipulate the files

 

Your issue is likely to be the permissions/authorities for the source file system for the OS user accounts running the Automic Agent OR the OS user that is invoked as part of the process.   As others have mentioned,  the post-conditions are NOT necessarily run as the OS user running the primary JOB.   Setting up the OS user running the agent to use a network account can be a bit tricky, Encourage you to use Automic support for that.  

[Scott Wirth]

Patrick,

Sorry for the late response, I've got an odd schedule....

 

We've defined service accounts that are used on the machines.  For the most part, anything that happens/launches, etc. is done under the aegis of the service account.   This was done to reduce the threat surface and to align with best practices for security.  Minimum permissions for it to do what it needs to do, it's a security consideration.  One of the reasons why we chose Automic, is that it required significantly less permissions than the prior scheduling software, and significantly less than the other product that we were considering at the time.   The previous scheduling software, and the errrrm.... 'organic' method of system expansion meant that some of the folks scheduling ran with 'Give it everything and figure out how to pare it down later'..... which didn't happen very often.

 

That being said, most of the jobs are being launched with service account credentials, attributes set for the proper host and service account login.  I don't think it's an AD issue.  I've gone over permissions several times with the server engineers and we can't find a reason for it.  The only ones that run with their own set of authorizations differently are core jobs which authenticate with DCOM and CERCOM objects. It's a smaller subset of jobs, but all jobs outside of that should be running with the service account credentials.

 

During the course of some testing... I've ran across a couple of differences between DEV and TEST, but I don't think that they're issues.  

 

I tried setting up a nested workflow as I stated in the initial post, and DEV seemed to work fine, the postcondition on the internal workflow worked fine with a given host, and the postcondition checking a CIFS share ( netapp, I think) for file absence (the idea being.... run this until there are no more files).  

 

I've done the same in the TEST environment, with a similar setup, the internal workflow having a postcondition  checking for file absence.  When I use the same agent as the DEV ( some DEV/TEST crossover, and permissions the same), the postcondition works, but using the agent for the actual host, I receive the error message.  

 

The only thing that I'm using the postcondition for is to check for the presence/absence of files.  If present, it restarts the internal workflow to consume further files, if absent, it falls through and completes the external workflow.  

 

I've looked at the UC_SYSTEM_SETTINGS variables, and the only real difference I've found is that DEV has VAR_SECURITY_LEVEL set to 3, and TEST has VAR_SECURITY_LEVEL set to 0.  I don't think that it's an issue, as I can get the nested workflow working in TEST, just not with that particular agent/host.  

[Pete Wirfs]

I just happened to be working with a similar use-case today, where tasks need to either execute or not-execute based upon the existence of files in a folder.  In my case I used this workflow task precondition rule (V11.2.1);