Automating Wob Definitions

[Dustin Neely]

Good Morning,
We have an intake form that our development community uses to submit new wob for execution for our team to define inside of D-series. I am interested in the possibility of automating the wob definition, meaning I would like our intake system to possibly use the CLI or SOAP API to define wobs into a staging environment. So I have a few questions

1. has anyone done this or something similar?
2. Can someone point me to the documentation that might help me discover this capability. I have read through the D-series manual and find the Action insert keywords interesting but not sure exactly how to use them yet.

Thanks for any conversations around this topic.

[Andy Reimer]

The ESPmgr utility can, I believe, only be used to insert jobs into an active generation of an application.

I've done some experimenting with using the imexutil (Import Export Utility) to export XML files of applications, modify them, then import them back in.  I've had pretty good results but haven't gone as far as what you are suggesting.  That said,  it's possible and if you want help going down that road, I have some experience using Powershell to read/modify the XML that I can lend you.

Are you hoping for full automation, or just some time saving to get the bulk of the WOB created?  I'm curious because our developers don't know enough about Workload Automation to provide all the information needed for something fully automated, but we have discussed the use of an intake form.

------------------------------
Andy Reimer
------------------------------

[Dustin Neely]

Good after noon Andy,

Eventually I am hoping for full Automation into a staging library where we would review and then promote to prod, through our change management process. Our intake form right now is pretty good but could be a lot better in capturing details about job runs etc. I would love to see your powershell process and maybe work with it. And hoping that maybe someone from Broadcom can pop in and let us know if this full automation might be possible via API's in the future.

 

		Dustin Neely Support Analyst W 423.535.5784 M 423.208.6255 E [e-mail] www.bcbst.com Batch Scheduling Wiki

 

 

---

Please see the following link for the BlueCross BlueShield of Tennessee E-mail disclaimer:  https://www.bcbst.com/about/our-company/corporate-governance/privacy-security/email-policy.page

---

This email was sent by "Neely, Dustin" <[e-mail]> securely using Transport Layer Security (TLS).</[e-mail]>

[Andy Reimer]

At a high level, here is how I'd go about it.

• Create a test Workload Automation application in your lower environment.
• Export it with imexutil
• Modify it with Powershell
• Import it with imexutil
• Verify your the results
• Repeat with increasing complexity of of modification with Powershell.  Start small, move up.

When using the imexutil, it will throw an error if it detects malformed XML, but I'd still be very thorough in verifying that the WOBs defined by adding XML elements look exactly as you expect.  The XML structure for various WOB's can vary a bit and also vary depending on which fields are present.

For starters, get a good XML editor (I use Notepad++ with the XML Tools plugin). Save one of your applications as XML (Application Workspace -> right click on application -> Save As...) then open it up in your XML editor and start to get a feel for how the XML is laid out.  The most basic Windows System WOB, which just has the mandatory fields filled in, would look something like this:

<app:nt_job name="JOB01"> <app:dependencies> <app:relcount>0</app:relcount> </app:dependencies> <app:notifynodefaults>true</app:notifynodefaults> <app:alert_notifynodefaults>false</app:alert_notifynodefaults> <app:snmp_notifynodefaults>false</app:snmp_notifynodefaults> <app:hold>false</app:hold> <app:request>false</app:request> <app:conditional>false</app:conditional> <app:criticaljob>false</app:criticaljob> <app:job_ancestor_wait_default_ignore>true</app:job_ancestor_wait_default_ignore> <app:retry> <app:retry_count>0</app:retry_count> <app:retry_interval>0</app:retry_interval> <app:last_notify_email>false</app:last_notify_email> <app:last_notify_alerts>false</app:last_notify_alerts> <app:last_notify_snmp>false</app:last_notify_snmp> </app:retry> <app:subappl_wait>false</app:subappl_wait> <app:agent>SERVER_100</app:agent> <app:userid>DOMAIN\ID</app:userid> <app:args>-command 'D:\Cyb_exec\Job01.ps1'</app:args> <app:cmdname>C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe</app:cmdname> <app:is_interactive_job>false</app:is_interactive_job> </app:nt_job>​
I'll dig up the a powershell script that could be used to create this WOB and post it in a reply.

------------------------------
Andy Reimer
------------------------------

[Don Powell]

Hi, 
I would make one caution in this discussion. 
From release to release the format of the xml can change.  Any process that changes the xml directly could be broken or cause unknown issues after an upgrade.(assuming it is done correctly the first time)  I think asking for an enhancement for a restapi call to define a new job would be a better option....

Something to keep in mind. 
2¢
Don

[Andy Reimer]

It's a good point.  This is clearly 'out of the box'.  I've noticed that there are at least two XML formats that I've seen comparing some of our old applications to newer ones.

------------------------------
Andy Reimer
------------------------------

[Dustin Neely]

That is great information.

Lucky for me I am in a Proof of Concept to move from ESP to D-series so I get the opportunity to vet this stuff out.

 

		Dustin Neely Support Analyst W 423.535.5784 M 423.208.6255 E [e-mail] www.bcbst.com Batch Scheduling Wiki

 

 

---

Please see the following link for the BlueCross BlueShield of Tennessee E-mail disclaimer:  https://www.bcbst.com/about/our-company/corporate-governance/privacy-security/email-policy.page

---

This email was sent by "Neely, Dustin" <[e-mail]> securely using Transport Layer Security (TLS).</[e-mail]>

[Andy Reimer]

Here is a powershell script (saved as a txt file) that will insert a WOB at the end of an APPL xml file.  As you have to define a node for each field that will be used, it could be a considerable amount of work to build something that will get it right every time, when some fields may be left blank on the request form your developers fill out.

Also, as mentioned, no guarantee the xml format won't change in future releases.  I think this technique is a safer than trying to make modifications to DE by issuing SQL statements against the database, but you are still in 'unsupported' country.

I've used this technique to create some 'one off' solutions; for instance adding a specific alert to multiple jobs spread across multiple applications.  I probably could have manually opened, updated, and uploaded 150 applications in less time than it took to get my head around modifying xml with PowerShell, but at least now I know how to do it.

PS. We migrated from ESP to DE about 8 years ago.  If you have any specific questions, feel free to ask.

------------------------------
Andy Reimer
------------------------------

Attachment(s)

Add_WOB.txt   3 KB 1 version

[SHARON SHIMANEK]

With the export/import utility doesn't that export and import all applications/jobs?  Or, can you be more granular?  It is scary to me to export, modify and import ALL when you are only changing a small percent of applications/jobs.

Sharon

[Andy Reimer]

When exporting, you can specify only certain types of artifacts (appl, event, agents, etc).
When importing, you point the utility at a folder, and it will only import the xml files in that folder.

So in my example of adding an alert, the PowerShell script called the imexutil to export all the applications to Folder_1, parsed through all the applications looking for jobs that met my specific criteria, modified the xml on those jobs, logged the application and job name to a separate file for my review, and saved the modified xml to Folder_2.  Once I reviewed the log of modified jobs, the imexutil was once again called to import the applications in Folder_2 which only contained a subset of the applications exported into Folder_1.

------------------------------
Andy Reimer
------------------------------

[SHARON SHIMANEK]

Thank you for the clarification.  I will have to work with the import and export.

Sharon