Automic Workload Automation

Expand all | Collapse all

EXEC VARA to check file system free space

  • 1.  EXEC VARA to check file system free space

    Posted 01-26-2018 06:32 AM
      |   view attached
    I have written a SCRI-EXEC VARA-JOBS combo for checking file system space on our AE server hosts.

    UC4.CHECK_FILESYSTEM_SPACE.SCRI
    :SET &Filesystem# = "/var"
    :SET &VarHnd# = PREP_PROCESS_VAR(UC4.AE_SERVER_FILESYSTEM_INFO.VARA_EXEC)
    :PROCESS &VarHnd#
    :  SET &Line# = GET_PROCESS_LINE(&VarHnd#)
    :  SET &FreeSpace# = GET_PROCESS_LINE(&VarHnd#,4)
    :  SET &FreePrecent# = GET_PROCESS_LINE(&VarHnd#,5)
    :ENDPROCESS
    :CLOSE_PROCESS &VarHnd#
    :PRINT "File system &Filesystem# has &FreeSpace# (&FreePrecent#%) free space."
    Output
    U00007000 'UC4.AE_SERVER_FILESYSTEM_INFO.JOBS' activated with RunID '0267982522'.
    U00020408 File system /var has 552M (8.9G%) free space.

    How does it work? Well, the VARA is an EXEC VARA that passes along the &Filesystem# variable to a JOBS object.

    UC4.AE_SERVER_FILESYSTEM_INFO.VARA_EXEC
    xti39yx0pz4e.pnghttps://us.v-cdn.net/5019921/uploads/editor/r8/xti39yx0pz4e.png" width="441">

    The UC4.AE_SERVER_FILESYSTEM_INFO.JOBS job performs three steps:
    1. Pre-process: Read the&Filesystem#variable
    2. Process: Run thedfcommand on the file system specified in the&Filesystem#variable, and convert the output to a format that is easily parsed by PREP_PROCESS_REP.
    3. Post-process: Parse the output and create a new data sequence with the results, passing along this data sequence to the calling EXEC VARA.
    Pre-process
    :READ &Filesystem#,,
    Process
    df -h &Filesystem# | grep -v Mounted | sed 's/ \+/;/g'
    Post-process
    !Verify that the job ran successfully.
    :SET &RC# = GET_UC_OBJECT_STATUS(, , "RETCODE")
    :PRINT "Job RC: &RC#"
    :SET &Output_Delimiter# = ";"
    :IF &RC# = 0
    :  SET &PPR_ReportType# = "REP"
    :  SET &PPR_Filter# = "*&Filesystem#*"
    :  SET &PPR_Def1# = "COL=DELIMITER"
    :  SET &PPR_Def2# = "DELIMITER=@;


    :  INCLUDE UC4.EXEC_VARA_TOOL.CREATE_DATA_SEQUENCE_FROM_JOB_OUTPUT.JOBI
    :ELSE
    :  PRINT

    Job ended in error."
    :  INCLUDE UC4.EXEC_VARA_TOOL.CREATE_DATA_SEQUENCE_WITH_ERROR_MESSAGE.JOBI
    :ENDIF
    The JOBIs do most of the work. I’ll include all of the objects in an XML attachment.

    The original genesis of this was my frustration at the fact that when we enable AE tracing the trace file can quickly fill up the target file system, and the Automation Engine is not clever enough to stop writing traces before this happens. Now that I have a general-purpose VARA for checking the available space in a file system, I can insert periodic checks into the scripts that we use to capture traces. If the target file system is beginning to get full, we can automatically disable tracing.

    Ideas for further generalizing this approach:
    • Pass the target agent or agent group name as parameter.
    • Pass the command to run as as a parameter.
    • Pass the arguments for PREP_PROCESS_VAR as parameters

    Attachment(s)



  • 2.  EXEC VARA to check file system free space

    Posted 01-26-2018 06:40 AM
    Hi Michael_Lowry

    Thanks for sharing this.

    Did you also consider using a VARA.BACKEND in this case ? It allows to retrieve results of OS commands and arrange them by setting the column's width.

    Just another way of achieving a similar result, with one object less :)

    Best regards,
    Antoine


  • 3.  EXEC VARA to check file system free space

    Posted 01-26-2018 07:16 AM
    Yeah, BACKEND VARAs appear to be quite limited.


  • 4.  EXEC VARA to check file system free space

    Posted 01-26-2018 07:37 AM
    I had a little brainstorm and thought it might be possible to eliminate most of the steps in the main parsing JOBI, replacing them with a much more direct approach that passes to the EXEC VARA the data sequence created by PREP_PROCESS_REP. E.g.,
    :SET &RepHnd# = PREP_PROCESS_REPORT(, , "&PPR_ReportType#", "&PPR_Filter#", "&PPR_Def1#", "&PPR_Def2#")
    :SET &NumColumns# = GET_PROCESS_INFO(&RepHnd#,COLUMNS)
    :SET &NumRows# = GET_PROCESS_INFO(&RepHnd#,ROWS)
    :PRINT "Job report data sequence contains &NumColumns# columns & &NumRows# rows."
    :SET &Result# = CREATE_PROCESS(DUPLICATE,&RepHnd#)
    :PSET &Results# = SAVE_PROCESS(&Result#)
    Unfortunately, this does not work. I think the problem is that the EXEC VARA expects column headings, and the data sequence from PREP_PROCESS_REP lacks them.

    Data sequence from PREP_PROCESS_REP
    oa2m4641j8zq.pnghttps://us.v-cdn.net/5019921/uploads/editor/gd/oa2m4641j8zq.png" width="327">

    Data sequence created by JOBI
    3a5srf30c922.pnghttps://us.v-cdn.net/5019921/uploads/editor/iy/3a5srf30c922.png" width="513">

    In situations where only one value is returned, or where parsing can be done afterward, the above streamlined approach might be acceptable.


  • 5.  EXEC VARA to check file system free space

    Posted 01-26-2018 07:46 AM
    Next I thought I might be able to add the header row to the top of the data sequence like this:
    :SET &RepHnd# = PREP_PROCESS_REPORT(, , "&PPR_ReportType#", "&PPR_Filter#", "&PPR_Def1#", "&PPR_Def2#")
    :SET &NumColumns# = GET_PROCESS_INFO(&RepHnd#,COLUMNS)
    :SET &NumRows# = GET_PROCESS_INFO(&RepHnd#,ROWS)
    :PRINT "Job report data sequence contains &NumColumns# columns & &NumRows# rows."
    !Construct a string for the column headings, based on the number of columns.
    :SET &HeadingsLine# = "Value 1"
    :SET &ResultColumnNum# = 1
    :WHILE &ResultColumnNum# < &NumColumns#
    :  SET &ResultColumnNum# = &ResultColumnNum# + 1
    :  SET &ResultColumnNum# = FORMAT(&ResultColumnNum#)
    :  SET &HeadingsLine# = "&HeadingsLine#&Output_Delimiter#Value &ResultColumnNum#"
    :ENDWHILE
    :PRINT "Headings line: &HeadingsLine#"
    :SET &HeaderRow# = CREATE_PROCESS(NEW)
    !Write column headings.
    :SET &RC# = PUT_PROCESS_LINE(&HeaderRow#,&HeadingsLine#,"&Output_Delimiter#")
    :SET &Result# = CREATE_PROCESS(JOIN,&HeaderRow#,&RepHnd#)
    :PSET &Results# = SAVE_PROCESS(&Result#)
    (Note the command in bold.) Unfortunately, this did not work either. The AE returned this error message:
    U00020684 Runtime error in object 'UC4.EXEC_VARA_TOOL.CREATE_DATA_SEQUENCE_FROM_JOB_OUTPUT2.JOBI', line '00017': data sequences couldn't be joined, because their types are different.
    This tells us that:
    • Not all types of data sequence are equivalent.
    • The AE treats different types of data sequences differently.
    • EXEC VARAs are unable to detect columns in PREP_PROCESS_REP data sequences.


  • 6.  EXEC VARA to check file system free space

    Posted 01-26-2018 08:34 AM
    Michael Lowry said:
    Ideas for further generalizing this approach:
    • Pass the target agent or agent group name as parameter.
    • Pass the command to run as as a parameter.
    • Pass the arguments for PREP_PROCESS_VAR as parameters
    This was a quicker job than I expected. Here is the generalized EXEC VARA:
    4dc2izfb766g.png

    And the corresponding JOBS:

    Pre-process
    :READ &Agent_or_AgentGroup#,,
    :READ &Login#,,
    :READ &Queue#,,
    :READ &Command#,,
    :READ &PPR_ReportType#,,
    :READ &PPR_Filter#,,
    :READ &PPR_Def1#,,
    :READ &PPR_Def2#,,
    :READ &Output_Delimiter#,,
    :INC UC4.RESOLVE_AGENT_GROUP.JOBI
    :PUT_ATT HOST = &Agent#
    :PUT_ATT LOGIN = &Login#
    :PUT_ATT QUEUE = &Queue#
    Process
    &Command#
    Post-process
    !Verify that the job ran successfully.
    :SET &RC# = GET_UC_OBJECT_STATUS(, , "RETCODE")
    :PRINT "Job RC: &RC#"
    :IF &RC# = 0
    :  INCLUDE UC4.EXEC_VARA_TOOL.CREATE_DATA_SEQUENCE_FROM_JOB_OUTPUT.JOBI
    :ELSE
    :  PRINT "Job ended in error."
    :  INCLUDE UC4.EXEC_VARA_TOOL.CREATE_DATA_SEQUENCE_WITH_ERROR_MESSAGE.JOBI
    :ENDIF

    Note: I use the JOBI to resolve the agent group because PUT_ATT is unable to set the HOST attribute to an agent group.

    This solution is very customizable, but it also means that SCRI that runs this EXEC VARA must set all of the input parameters that the EXEC VARA is expecting.
    :SET &Filesystem# = "/var"
    :SET &Agent_or_AgentGroup# = "UC4.AE_SERVERS"
    :SET &Login# = "UC4.LOGIN"
    :SET &Queue# = "UC4"
    :SET &Command# = "df -h &Filesystem# | grep -v Mounted | sed 's/ \+/;/g'"
    :SET &PPR_ReportType# = "REP"
    :SET &PPR_Filter# = "*&Filesystem#*"
    :SET &PPR_Def1# = "COL=DELIMITER"
    :SET &PPR_Def2# = "DELIMITER=@;


    :SET &Output_Delimiter# =

    ;"
    :SET &VarHnd# = PREP_PROCESS_VAR(UC4.BACKEND.VARA_EXEC)
    :PROCESS &VarHnd#
    :  SET &Line# = GET_PROCESS_LINE(&VarHnd#)
    :  SET &FreeSpace# = GET_PROCESS_LINE(&VarHnd#,4)
    :  SET &FreePrecent# = GET_PROCESS_LINE(&VarHnd#,5)
    :ENDPROCESS
    :CLOSE_PROCESS &VarHnd#
    :PRINT "File system &Filesystem# has &FreeSpace# (&FreePrecent#%) free space."
    It works like a charm. Enjoy!


  • 7.  EXEC VARA to check file system free space

    Posted 01-26-2018 08:51 AM
    Honestly for checking single values I prefer good old prep_process + OS command:
    (slightly modified for community)
    :SET &AGENT# = "V112_CENTOS64_01"
    :SET &LOGIN# = "@LOGIN.DEFAULT"
    !
    :SET &CMD# = "df -m / | sed 1d | awk '{print $4}'"
    !
    :SET &HND# = PREP_PROCESS(&AGENT#, "UNIXCMD", "*", "CMD=&CMD#", , "UC_LOGIN=&LOGIN#")
    :PROCESS &HND#
    :  SET &FREE_MB# = GET_PROCESS_LINE(&HND#)
    :ENDPROCESS
    :CLOSE_PROCESS &HND#
    :SET &FREE_MB# = FORMAT(&FREE_MB#)
    !
    :IF &FREE_MB# < 10000
    :  IF &FREE_MB# > 5000
    :    PRINT "#  WARNING only &FREE_MB# MB left."
    :    SET &RET# = SEND_MAIL("uc4@local.com", , "Space alert on &AGENT#", "ALERT: Only &FREE_MB# MB left on Agent: &AGENT# !!!")
    :  ELSE
    :    PRINT "### ALERT: only &FREE_MB# MB left ###"
    :    STOP MSG, 50, "ALERT: only &FREE_MB# MB left on &AGENT#"
    :  ENDIF
    !
    :ELSE
    :  PRINT "Enough Space left: &FREE_MB# MB"
    :ENDIF

    :-)

    if Free Space < 10 GB an email is sent

    if Free Space < 5 GB the check script fails

    cheers, Wolfgang



  • 8.  EXEC VARA to check file system free space

    Posted 01-26-2018 09:31 AM
    That's too simple! If you cannot use it for everything, it's boring. :wink:


  • 9.  EXEC VARA to check file system free space

    Posted 01-26-2018 10:06 AM

    That's too simple! If you cannot use it for everything, it's boring. :wink:

    My colleague hates "keep it simple" as well....

    So I suggested writing an Automic Windows Job that creates a triggerfile, which gets transferred via Filetransfer on a unix Host that runs an OS filewatcher that fires a python script which starts a perl command that determines the Space on the unix server and starts a CALL API whicht takes over the value from the perl command to an SQL Job that writes the value into an external Database.
    Here we have an EVENT.DB - if here changes a value it gets written into a VARA object and every 5 Minutes an EVNT.TIME object sends an email with the value from the VARA object to a mailinglist of the whole company.

    >:)


  • 10.  EXEC VARA to check file system free space

    Posted 01-26-2018 10:25 AM
    What, no carrier pigeon?


  • 11.  EXEC VARA to check file system free space

    Posted 01-26-2018 10:44 AM
    Thats a bit tooo old school..
    But Fax, Telex, Pager services, or Call Center messages would be a great attempt for this chain....