OSMH : Instance suddenly into offline status

Applies to: OSMH On-Prem

Goal: 

OS Management Hub (OSMH) instance suddenly goes into offline status, and the management agent service exits.

Even after restarting the management agent service, the agent process keeps dying/exiting.

$ sudo systemctl restart mgmt_agent $ ps -elf | grep osmh 4 S root      149691  149643  0  80   0 - 27453 -      16:27 ?        00:00:00  /bin/sudo -n -E -u root /opt/oracle/mgmt_agent/plugins/osmh/13/osmh 4 S root      149701  149691  0  80   0 - 180641 -     16:27 ?        00:00:00  /opt/oracle/mgmt_agent/plugins/osmh/13/osmh 0 S root      149778  148836  0  80   0 -  3036 -      16:27 pts/1    00:00:00 grep -- color=auto osmh $ $ ps -elf | grep osmh 4 S root      149793  149750  0  80   0 -  1838 -      16:27 pts/2    00:00:00 tail - f /opt/oracle/mgmt_agent/plugins/osmh/stateDir/log/osmh-agent.log 0 S root      149801  148836  0  80   0 -  3036 -      16:28 pts/1    00:00:00 grep -- color=auto osmh $$ sudo systemctl status mgmt_agent ¿ mgmt_agent.service - mgmt_agent    Loaded: loaded (/etc/systemd/system/mgmt_agent.service; enabled; vendor  preset: disabled)    Active: active (running) since Tue 2023-06-06 16:27:08 IST; 1min 20s ago   Process: 149385 ExecStop=/opt/oracle/mgmt_agent/agent_inst/bin/agentcore  stop sysd (code=exited, status=0/>   Process: 149545 ExecStart=/opt/oracle/mgmt_agent/agent_inst/bin/agentcore  start sysd (code=exited, status=>  Main PID: 149625 (wrapper)     Tasks: 55 (limit: 61749)    Memory: 375.9M    CGroup: /system.slice/mgmt_agent.service            ¿¿149625 /opt/oracle/mgmt_agent/agent_inst/bin/./wrapper  /opt/oracle/mgmt_agent/agent_inst/bin/..>            ¿¿149643 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.372.b07- 4.0.1.el8.x86_64/jre/bin/java -Dorg.tanuki> Jun 06 16:26:57 Management-Station systemd[1]: Starting mgmt_agent... Jun 06 16:26:57 Management-Station agentcore[149545]: Starting mgmt_agent... Jun 06 16:27:00 Management-Station sudo[149691]: mgmt_agent : TTY=unknown ;  PWD=/opt/oracle/mgmt_agen> Jun 06 16:27:03 Management-Station agentcore[149545]: Waiting for  mgmt_agent......... Jun 06 16:27:08 Management-Station agentcore[149545]: .....running:  PID:149625 Jun 06 16:27:08 Management-Station systemd[1]: Started mgmt_agent.