Need script to check the JDE services are up or not through Automic

[Gagan S N]

Dear Team,

We require an automated solution to ensure seamless integration between JDE service availability and Automic server operations. The primary objective is to dynamically manage Automic server states (shutdown/startup) based on the operational status of corresponding JDE services.

Specifically, we need a mechanism that:

1. Monitors JDE Service Availability:
   • Detects when JDE services are offline or experiencing issues.
   • Triggers a signal or command (via API or batch job from JDE) to Automic upon detection of JDE service unavailability.
   • Conversely, signals Automic when all JDE services are fully operational.
2. Automates Automic Server Management:
   • Upon receiving a "JDE services offline" signal, Automic should automatically shut down the affected Automic server instance.
   • Upon receiving a "JDE services online" signal, Automic should automatically initiate the startup of the corresponding Automic server.

We encountered a critical issue recently where jobs assigned to an Automic agent group, comprising agents on servers 594 and 596, failed on server 596. This occurred because, while both Automic agents were reported as available, JDE services were only configured and operational on server 594. This led to jobs incorrectly attempting to execute on server 596, resulting in failures.

Therefore, we urgently need a solution that enables Automic to:

• Verify JDE Service Readiness: Before dispatching jobs to a specific Automic agent/server, Automic must be able to confirm that the underlying JDE services on that server are not only configured but also actively running and healthy.
• Intelligent Job Dispatching: Automic should only dispatch jobs to an agent within an agent group if both the Automic agent and its corresponding JDE services are fully available and operational. This would ensure jobs are only triggered on truly "active" and capable servers.

Could you please advise if there are existing features, best practices, or custom configurations within Automic that can facilitate this type of advanced monitoring and intelligent job dispatching based on external service availability? We are open to exploring any recommended approaches, including API integrations, custom scripts, or built-in Automic functionalities.

Your insights and assistance in implementing this crucial functionality are greatly appreciated.

Thanks,
Gagan S N

[Richard Beck]

This might not be quite the solution you are looking for but rather than a reactive detect, stop start.  We put the start of the agent at the tail end of the jde services start script and the agent stop in the stop script.

We turned of auto start/stop in service manager.   This deals with scheduled starts and stops of jde but not crashes.

[Gary Chismar]

We wrote a process to monitor our PeopleSoft server (where our agents reside) and act. If the app server is down, the associated agent will be shut down. When the server comes back up, the agent is restarted.

The process is fairly simple; this is from the document object I created back in 2021:

----------------------------------------------------------------------
Conceptually, the agent monitoring is as follows:
----------------------------------------------------------------------
1) Ping oraprd03 (where the agents reside).
If up, then continue.
If down, end program. There is not anything we can do until it is back up.
No notifications need to be sent out as Nagios will be sending those to the PSAdmins

2) If orpard03 is up then the next step is to ping the app servers.
Record the results and continue.

3) Check the connection to the PeopleSoft domains. This is a Jolt Connection test.
Record the results and continue.

4) Check the Automic agent status associated with its respective PeopleSoft domain.
Record the results.

5) Make a decision as to what action is necessary to take.



----------------------------------------------------------------------
Summary of the above
----------------------------------------------------------------------
1) ORAPRD03
UP - Continue
Down - Stop cannot do anything else until this is resolved
2) App Servers
UP - Continue
Down - Continue
3) Domains
UP - Continue
Down - Continue
4) Automic Agents
UP - Continue
Down - Continue
5) Make decision based on test results of 2,3 and 4.
----------------------------------------------------------------------
Decisions are as follows:
----------------------------------------------------------------------

1) ORAPRD03 - Down
End program. Cannot get to agents to test until this server is back up.

2) ORAPRD03 - Up
App Server - Up
Domain - Up
Agent - Up
No action necessary. Everything is up. All is well

3) ORAPRD03 - Up
App Server - Down
Domain - Down (Will be down by virtue of app server being down)
Agent - Up
Shut Down Agents.

4) ORAPRD03 - Up
App Server - Down
Domain - Down (Will be down by virtue of app server being down)
Agent - Down
No action necessary. Agent already shut down.

5) ORAPRD03 - Up
App Server - Up
Domain - Down
Agent - Up
Shut down appropriate agent. Send notification to PSAdmins and Automic team about the domain being down and shutting down the related agent.

6) ORAPRD03 - Up
App Server - Up
Domain - Up
Agent - Down
Restart appropriate agent. Send notification to Automic team about restarting the agent.

7) ORAPRD03 - Up
App Server - Up
Domain - Down
Agent - Down
No action necessary. Both are down so Automic will fail over to the other agent provided that is up.
----------------------------------------------------------------------



Cheers,

Gary Chismar

Enterprise Development Manager, HCM/FSCM/CS/Automic
PeopleSoft/Automic Development Lead
Information Technology Services | Florida State University
1721 Paul Dirac Dr, Tallahassee FL, 32210