ESP dSeries Workload Automation

 View Only
  • 1.  Error: Out of DB connection issue WLA STOP

    Posted Jul 11, 2020 04:57 AM
    Hello There!
    We have get into an issue of DB connectivity lost got Error: Out of DB connection issue which made our WLA tool got STOP in PRODUCTION environment. our several jobs which ran at midnight load all got delayed - i believe some of they consume all db connection of resource and rest all got delayed through message out of db connection.

    Can someone help me how to set up Auto Alert (email) for DB connection break down/Lost Database connectivity  stating- Likewise No. of active application/DB connection is equal or bigger than the notify limit. Server will stop working something like this. From where i can set up this let me know procedure. Thank you in Advance!


  • 2.  RE: Error: Out of DB connection issue WLA STOP

    Posted Jul 13, 2020 08:22 AM
    Something quick and simple enough that you can do run something like the below that will search for the message you are receiving. I suppose though you would want to schedule it some other way outside of DE in case the same issue delays this job from kicking off. We use SCOM a lot in our org and have had it monitor log files and create tickets if a string of text was found. I don't know all of the capabilities of SCOM but surely you could be emailed also.

    The code below looks in the last day of tracelog files, you can set it up to look in the active file, tracelog.txt

    #Create the file if it doesn't exist
    [ -f /cawa/scripts/dbOutOfConnections ] || touch /cawa/scripts/dbOutOfConnections

    lines=$(find $logsDirectory -name 'tracelog.*' -mtime -1 -exec grep -wa 'Out of DB connections, waiting' {} + | grep -v ShutdownHook | wc -l)

    if [ $lines -eq 0 ];
    then
    echo Did not find any message of the db running out of connections
    exit 0

    else
    echo Found a message of the db running out of connections
    find $logsDirectory -name 'tracelog.*' -mtime -1 -exec grep -wa 'Out of DB connections, waiting' {} + | grep -v ShutdownHook | awk '{print $6,$7,$8,$9,$10,$11,$12}' | uniq >>/cawa/scripts/dbOutOfConnections
    exit 1
    fi


  • 3.  RE: Error: Out of DB connection issue WLA STOP

    Posted Jul 16, 2020 10:09 AM
    Great Thanks! Travis.
    Let me check the possible outcome by applying this scripted solution. thank you


  • 4.  RE: Error: Out of DB connection issue WLA STOP

    Posted Jul 29, 2020 03:40 PM
    Hi, 
    In Define under Palette - MONITORING a job type of Text File Reading and Monitoring.  This can be set to read the logs for errors. But my recommendation would be use the errorrs.txt log vs tracelog.txt.  Reason is when the job loads the Text File Name criteria would show in the tracelog.