OPMN is failed to start obiee 11.1.1.9 (OPMN server ready. Request handling enabled)

Question

Hi All,

OPMN is failing to start.

opmn.log

[2017-07-24T13:32:20+10:00] [opmn] [TRACE:32] [] [internal] ORACLE_HOME: /home/oracle/Oracle/Middleware/Oracle_BI1

[2017-07-24T13:32:20+10:00] [opmn] [TRACE:32] [] [internal] ORACLE_INSTANCE: /home/oracle/Oracle/Middleware/instances/instance1

[2017-07-24T13:32:20+10:00] [opmn] [NOTIFICATION:1] [90] [ons-internal] ONS server initiated

[2017-07-24T13:32:20+10:00] [opmn] [TRACE:32] [] [internal] Host: anzdatabase; Remote Port: 9505; Local Port: 9504; Pid: 7078; 11.1.1.9.0

[2017-07-24T13:32:20+10:00] [opmn] [TRACE:1] [522] [pm-internal] PM state directory exists: /home/oracle/Oracle/Middleware/instances/instance1/config/OPMN/opmn/states

[2017-07-24T13:32:20+10:00] [opmn] [NOTIFICATION:1] [675] [pm-internal] OPMN server ready. Request handling enabled

opmn.out

[2017-07-24T12:54:34+10:00] [opmn] [TRACE:32] [] [internal] Server shut down: status 4200

[2017-07-24T13:23:17+10:00] [opmn] [TRACE:32] [] [internal] Server shut down: status 4200

[2017-07-24T13:30:39+10:00] [opmn] [TRACE:32] [] [internal] Server shut down: status 4200

Note: NodeManager is running

AdminServer is running

ManagedServers are running.

Tried solutions.

Tried the solution mentioned in the document (Doc ID 2261219.1) , that did not work.

All your suggestions are appreciated.

Thanks,

Anil.

Anil Vemula · Answer

Hi All,

I got the OPMN issue fixed. Below is the solution  (referred doc : Doc ID 1303000.1)

Provided Solution : 
1) Killed all the opmn processes (more than 1 process)
$ netstat -tulpn
$ kill -9 <opmn pid>
2) Verified the hosts file for the correct ip
3) Deleted the opmn .log and .out files from /Middleware/instances/instance1/diagnostics/logs/OPMN/opmn
4) Deleted all the files from states directory (including .local and .opmndat) 
/Middleware/instances/instance1/config/OPMN/opmn/states .
5) Verify the opmn.xml for the correct ports (remote and local)
4) ./opmnctl validate (to make sure the opmn.xml is correct)
5) ./opmnctl start (to start OPMN Process Manager)
6) ./opmnctl status -l 
7) Check the States directory has got new set of files (we deleted in step 4)
8) ./opmnctl stopall (if required)
9) ./opmnctl startall
10) ./opmnctl status

Thanks  Christian Berg and Sherry George for your comments.

Anil.

Anil Vemula · Answer

Hi Sherry,

Good day to you,

Yes the environment was working before, for some reason the vm was down and I found this issues while bringing the environment up.

Thanks,

Anil.

Sherry George · Answer

Was the environment working fine before? If yes, what changed?