AutoSys Workload Automation

 View Only
Expand all | Collapse all

Autosys and ServiceNow

  • 1.  Autosys and ServiceNow

    Posted Nov 23, 2022 11:51 AM
    Has anyone integrated Autosys to ServiceNow to automatically create incidents on job failures, if so how?  Splunk?   I know you can use snmp, but your limited on customization then.


  • 2.  RE: Autosys and ServiceNow

    Posted Nov 23, 2022 12:16 PM

    Using snmptraps are good stuff...

    You could:

    1. configure your autouser to send snmp traps on localhost

    SnmpManagerHosts=127.0.0.1
    SnmpCommunity=scheduler


    install snmptrapd and configure snmptrapd.conf like this :

    authCommunity log,execute,net scheduler
    # trapJobFailure
    traphandle .1.3.6.1.4.1.791.3.2.1.0.503 /usr/bin/perl /path/to/my/autosys_trap_hook_script.pl

    (hook only for trapJobFailure)

    And script an api rest call to create a case in servicenow (snow works well with Soap::lite, or even with something like https://metacpan.org/pod/ServiceNow::SOAP).

    The mib of autosys in $AUTOSYS/snmp/CA.my.

    Here are the information you could push into your ticket..

    trapJobFailure TRAP-TYPE
        ENTERPRISE autosys
        VARIABLES  { trapMessage, trapDate, alarmName, alarmJobName,
                     alarmText, alarmCode, alarmExitCode, alarmMachineName,
                     alarmBoxName, alarmApplicationName, alarmGroupName,
                     alarmRunNum, alarmNtry, alarmInstanceName}
        DESCRIPTION
            "Trap to indicate a job has failed"
        ::= 503

    Here is a start for a hook script in snmptrapd, which loads the mib and translates OIDs in proper names...

    #!/bin/env perl -W
    
    use strict;
    use warnings;
    use utf8;
    use English qw( -no_match_vars );
    use Carp;
    use Data::Dumper;
    use SNMP;
    
    my $SNMPv2_SMI_enterprises_CA_id;
    my $SNMPv2_SMI_enterprises;
    my $SNMPv2_ID;
    my $autosys_mib;
    
    my $verbose = "INFO";
    
    my %trapdata;
    my $trapoid;
    my $trapname;
    
    BEGIN {
      $SNMP::save_descriptions = 1;
      $SNMPv2_SMI_enterprises_CA_id = ".791";
      $SNMPv2_SMI_enterprises = ".1.3.6.1.4.1";
      $SNMPv2_ID = sprintf("%s%s", $SNMPv2_SMI_enterprises, $SNMPv2_SMI_enterprises_CA_id);
      $autosys_mib = {};
    
      my $CA_match = qr/^$SNMPv2_ID/;
    
      SNMP::addMibDirs("/usr/share/snmp/mibs/");
      SNMP::addMibFiles("/opt/CA/autosys/snmp/CA.my");
      SNMP::initMib();
    
      foreach my $mib (keys %SNMP::MIB) {
        # skip non autosys objects
        next if $mib !~ $CA_match;
        $autosys_mib->{$mib} = $SNMP::MIB{$mib};
        #print $mib, " - ", $SNMP::MIB{$mib}->{label},"\n";
      }
    }
    
    my $oo = <>; # skip line, useless
    my $bo = <>; # skip line, useless
    
    print "A new trap happened\n" if grep (/$verbose/,("DEBUG", "INFO"));
    
    # translation
    while (my $line = <>) {
      chomp $line;
      next if $line eq "";
      $line =~ s/SNMPv2-SMI::enterprises/$SNMPv2_SMI_enterprises/;
    
      print $line,"\n" if grep (/$verbose/, ("DEBUG"));
      my ($oid, @v) = split(/\s+/, $line);
      if ($oid eq "SNMPv2-MIB::snmpTrapOID.0") {
        $trapoid = $v[0];
        $trapname = $autosys_mib->{$trapoid}->{label};
        $trapdata{"description"} = $autosys_mib->{$trapoid}->{description};
        print "Trap: $trapname - $trapdata{'description'}\n" if grep(/$verbose/,("DEBUG", "INFO", "WARNING", "ERROR", "FATAL"));
      } else {
        if ($oid =~ /$SNMPv2_ID/) {
          my $label = $autosys_mib->{$oid}->{label};
          my $v = join(" ", @v);
          $v =~ s/(\A"|"\Z)//g; # remove leading and trailing double quote...
          $trapdata{$label} = $v;
        }
      }
    }
    
    # do whatever you want from now...
    my $trap = $class->new(%trapdata);

    This will display a much better info in /var/log/snmptrapd.log



    ------------------------------
    [JobTitle]
    [CompanyName]
    ------------------------------



  • 3.  RE: Autosys and ServiceNow

    Posted Jan 05, 2023 07:47 PM
    @Lionel TUAL How happy to see Perl in action still :)
    Python is taking over faster than I'd imagined.
    Thanks for the script! ​​


  • 4.  RE: Autosys and ServiceNow

    Posted Jan 05, 2023 07:49 PM
    we are piloting the email route...
    https://www.servicenow.com/community/itsm-articles/incident-creation-via-e-mail-inbound-action/ta-p/2310819


  • 5.  RE: Autosys and ServiceNow

    Posted Jan 07, 2023 06:30 PM
    Our team has used email INCident creation for a few years now.  So far, it's pretty simple with only a slight delay (2-3 minutes).  Our engineer added Assignment Group and Configuration Item to the created INCidents so they route appropriately.

    ------------------------------
    JOHN
    ------------------------------



  • 6.  RE: Autosys and ServiceNow

    Posted Jan 09, 2023 09:31 AM
    Hi John, Thanks for replying. I was wondering how your engineer added Assignment group and config item?

    Thanks,

    Craig Hanson | HealthPartners | Systems Management | wk 952.853.8638 : cel 651.341.7427 : pgr 612.580.5599
    Workload Automation SharePoint Site<https: sites.healthpartners.com/hp/tis/automation/_layouts/15/start.aspx#/="">
    Workload Automation Object Request<https: healthpartners.service-now.com/sp?id="sc_cat_item&sys_id=ad791f92db8eb3c0be72fa6e0f961996">




  • 7.  RE: Autosys and ServiceNow

    Posted Jan 10, 2023 09:29 PM
    1. To use this capability, we add a notification_msg: attribute to a target job that has the keywords Assign: and Configuration: in it, followed by the exact spellings of an existing Assignment Group and Configuration Item in single quotes.  If the spellings don't match, a default AG & CI are assigned.
    2. We set the notification_emailaddress: attribute to be SNow's standard inbound email address: <org>@service-now.com
    3. You can program SNow to parse inbound emails looking for specific identifiers like the two above.  These appear in the Message: section of the default AutoSys email template.  The engineer did this coding, and I don't know these details.
    4. SNow will then create the INCident with the proper Assignment Group & Configuration Item whenever receiving one of these emails.  We rely on SNow to send out emails to the appropriate first responders.

    It took a while to get everything set up due to email firewall issues, tweaking the SNow programming to work every time correctly, and adding filters in case someone just randomly spams an email to the SNow address.

    Almost all of our 5K+ jobs are using this 4 year old setup with failures going straight to the responsible team.


  • 8.  RE: Autosys and ServiceNow

    Posted Feb 01, 2023 11:33 AM
    Thanks for this message. I had a couple of questions. 1. Have you ever had an issue where you had many jobs fail at once that open up separate tickets for each failure due to an issue in the environment? 2. We have critical jobs that run through AutoSys such as Payroll is there a notification feature that includes a phone call to an oncall person?

    ------------------------------
    Systems Engineer
    JHHS
    MD
    ------------------------------



  • 9.  RE: Autosys and ServiceNow

    Posted Feb 01, 2023 04:07 PM

    Wayne,
    (1) We have a job that runs every 15 minutes.  If it fails, it generates an SNow INCident.  So if you're out over a weekend, you can imagine the mess you'll find if this job started going haywire at 5:15pm on Friday.  I tried to find a way to prevent repeated INCidents, but that would require an additional layer of logic somewhere.

    IBM/Kyndryl built their own logic layer over ServiceNow to implement a "one INCident per 24 hour" rule.  But we couldn't get them to direct our INCidents to specific subteams via Assignment Groups.  These INCidents would only go to their Ops team/call center, who would then manually refer them to our department-wide group (which we would have to triage yet again).

    (2) ServiceNow has functionality for on-call schedules that I haven't explored.  At minimum, you could add fake members to an Assignment Group whose email is an SMS-message drop box.  Not exactly a phone call, but close.




  • 10.  RE: Autosys and ServiceNow

    Posted Feb 11, 2023 01:43 AM
    I can autoping the agent but it says it is unqualified which makes no sense . I tried everything that Broadcom states to resolve the issue . Any ideas ?

    Thanks
    Jerry South | Senior Enterprise Resource Plan Advisor, m. +1.864-608-3436

    NTT DATA | nttdataservices.com<https: us.nttdata.com/en=""> | @nttdataservices
    Consulting | Industry Solutions | Cloud & Digital Services
    Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.




  • 11.  RE: Autosys and ServiceNow

    Broadcom Employee
    Posted Jan 05, 2024 02:45 PM

    Hi Craig and Community,

    We appreciate the contributions in this thread as very interesting and viable ways to implement a ServiceNow and Autosys integration have been found. 

    We now have it natively supported in Autosys, and you can check it out here: https://techdocs.broadcom.com/us/en/ca-enterprise-software/intelligent-automation/autosys-workload-automation/12-1-01/administrating/ae-administration/integration-with-help-desk-applications/configure-help-desk-integration-and-global-variables.html

    Thank you,

    -Shannon



    ------------------------------
    _______________________________________
    Shannon Hebert
    Head of AOD Automation Support
    Broadcom Software | Agile Operations Division
    _______________________________________
    ------------------------------



  • 12.  RE: Autosys and ServiceNow

    Posted Jan 08, 2024 12:00 PM

    I know I am very late to the game, but here is my rusty nickels worth. Go with SNMP traps. You send more data to SNOW and can parse it however you want. We created 3 lists of traps, 1 for jobs, one for servers, and the last was traps to be ignored. Our SNOW Team was happy to make the changes. We even moved our Run Book to SNOW to make life easier for both ourselves and the SNOW Team to do lookups. We allow users to set up an alternate Assignment Group if support is not the primary AG for their Service. 

    I am happy to see that Broadcom is making progress with SNOW integration, but we are still seeing far more flexibility with SNMP traps. It helps to have a great SNOW development team who are willing to work with you to create the best solution. 




  • 13.  RE: Autosys and ServiceNow

    Posted Jan 09, 2024 12:35 AM

    AutoSys 12.1 uses SNMP to integrate with ServiceNow, although not sure how far it is customisable. 




  • 14.  RE: Autosys and ServiceNow

    Broadcom Employee
    Posted Feb 01, 2024 09:03 AM
    Edited by Michael Woods Feb 01, 2024 09:03 AM

    Hi,

    AutoSys 12.1 SP1 has direct integration with ServiceNow to open Incidents.  It uses the published REST API to communicate.

    The body of the request, which is basically the information to include in the ticket is configurable using a template approach.  There are variables that are supplied related to the job run like job name, run number, exit code,... and you can also use job attributes as well.

    You can have a generic template configured (or not) and also override that in specific jobs using different templates.  It also allows to you to attach log files as well to the incident. 

    There is a video that give some detail on what it does and how to configure: https://academy.broadcom.com/automation/autosys/servicenow-integration

    Regards,

    Mike