6 Replies Latest reply on May 29, 2015 11:48 AM by Dafydd Griffith

    Strange "adopmnctl.sh status" behaviour

    Dafydd Griffith


      Our organisation runs with a RAC production system and six development/test enviroments that are single instances, all cloned from production every month or so.

       

      During our morning checklist, to see which instances have survived the night, we have a couple of scripts that kick off "adopmnctl.sh status" to see what's up or down. 7 out of the 8 instances report "Alive" for "OC4J:oafm", "OC4J:forms", "OC4j:oacore" and "HTTP_Server". All is good. However, one instance always reports that the "OC4J:oafm" process is "Down". If I run it myself, again,  it always reports as being down even though I can go in through the front end and use the EBS system as normal.

       

      #adopmnctl.sh status

      You are running adopmnctl.sh version 120.6.12010000.5

      Checking status of OPMN managed processes...

      Processes in Instance: R12DB_dev02.dev02.mycompanyurl.co.uk
      ---------------------------------+--------------------+---------+---------
      ias-component                    | process-type       |     pid | status 
      ---------------------------------+--------------------+---------+---------
      OC4JGroup:default_group          | OC4J:oafm          |     N/A | Down   
      OC4JGroup:default_group          | OC4J:forms         |   10908 | Alive  
      OC4JGroup:default_group          | OC4J:oacore        |   10832 | Alive  
      HTTP_Server                      | HTTP_Server        |   10728 | Alive  


      adopmnctl.sh: exiting with status 0

      adopmnctl.sh: check the logfile /my_oracle_directory_structure/logs/appl/admin/log/adopmnctl.txt for more information ... 

       

       

       

      When I run adopmnctl.sh start I get:

       

      You are running adopmnctl.sh version 120.6.12010000.5

      Starting Oracle Process Manager (OPMN) ...

      adopmnctl.sh: exiting with status 0

      adopmnctl.sh: check the logfile /my_oracle_directory_structure/logs/appl/admin/log/adopmnctl.txt for more information ...

       

       

       

      and if I run adopmnctl.sh status, it tells me it's still down!

       

       

       

      Anyone have any ideas why it always reports "Down" when in fact it seems to be up and working? Why just this instance and not the other 7?

       

      Thanks.

        • 1. Re: Strange "adopmnctl.sh status" behaviour
          Bashar.

          Hi,

           

          What happens when you run "adoafmctl.sh start"?

           

          Regards,

          Bashar

          1 person found this helpful
          • 2. Re: Strange "adopmnctl.sh status" behaviour
            Dafydd Griffith

            #adoafmctl.sh start

            You are running adoafmctl.sh version 120.8

            Starting OPMN managed OAFM OC4J instance  ...

            adoafmctl.sh: exiting with status 204

            adoafmctl.sh: check the logfile /my_path/logs/appl/admin/log/adoafmctl.txt for more information ... 

             

             

            Forgot to att that if I bounce all services using adstpall.sh apps/pwd   and adstrtal.sh apps/pwd OC4J:oafm is still "Down".

            • 3. Re: Strange "adopmnctl.sh status" behaviour
              Bashar.

              Please post the last section in the log file (adoafmctl.txt).

              • 4. Re: Strange "adopmnctl.sh status" behaviour
                Dafydd Griffith

                I shut down the apps services using adstpall.sh apps/pwd and then ran through the following:

                 

                 1 adopmnctl.sh start

                 2 adalnctl.sh start

                 3 adapcctl.sh start

                 4 adoacorectl.sh start

                 5 adformsctl.sh start

                 6 adoafmctl.sh start

                 7 adcmctl.sh start apps/apps

                 8 jtffmctl.sh start

                 

                Steps 1-5 ran and returned a status of 0. Step 6 came back with a status of 204. A bit of a Google suggests that it is some sort of port conflict and I can see a post by you suggesting this to another person. Here: adoafmctl.sh: exiting with status 204

                 

                What I'm trying to figure out at tghe moment is what is the port number that conflicts with OC4J.

                 

                Here's the contents of the file and thanks for your help.

                 

                 

                ias-component/process-type/process-set:

                      default_group/oafm/default_group/

                 

                  Error

                 

                -->
                Process (index=1,uid=1394358689,pid=25146)

                      failed to start a managed process after the
                maximum retry limit

                 

                    Log:

                      /my_oracle_path/logs/ora/10.1.3/opmn/default_group~oafm~default_group~1.log

                 

                   05/28/15-13:16:17
                :: adoafmctl.sh: exiting with status 204

                 

                   ================================================================================

                  

                05/28/15-13:38:56
                :: adoafmctl.sh version 120.8

                   

                05/28/15-13:38:56
                :: adoafmctl.sh: Stopping OPMN managed OAFM OC4J instance

                 

                   opmnctl:
                stopping opmn managed processes...

                 

                ================================================================================

                 

                opmn
                id=erpdevd02:6240

                 

                    no processes or applications matched this
                request

                 

                   05/28/15-13:38:56
                :: adoafmctl.sh: exiting with status 150

                 

                   ================================================================================

                  

                05/28/15-13:41:46
                :: adoafmctl.sh version 120.8

                 

                  

                05/28/15-13:41:46
                :: adoafmctl.sh: starting OPMN if it is not running

                 

                   opmnctl:
                opmn is already running.

                  

                05/28/15-13:41:46
                :: adoafmctl.sh: Starting OPMN managed OAFM OC4J instance

                 

                opmnctl:
                starting opmn managed processes...

                 

                ================================================================================

                 

                opmn
                id=erpdevd02:6240

                 

                 

                    0 of 1 processes started.

                 

                   ias-instance
                id=R12DB_dev02.dev02.my_co_url.co.uk

                 

                 

                ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

                 

                --------------------------------------------------------------------------------

                 

                ias-component/process-type/process-set:

                      default_group/oafm/default_group/

                 

                   Error

                  -->
                Process (index=1,uid=1394358693,pid=27719)

                 

                    failed to start a managed process after the
                maximum retry limit

                 

                    Log:

                 

                    /my_oracle_path/logs/ora/10.1.3/opmn/default_group~oafm~default_group~1.log

                 

                   05/28/15-13:42:16
                :: adoafmctl.sh: exiting with status 204

                  

                ================================================================================

                • 5. Re: Strange "adopmnctl.sh status" behaviour
                  Kj - Kiran Jadhav

                  Can you check below log file for error details:

                   

                  Log:

                      /my_oracle_path/logs/ora/10.1.3/opmn/default_group~oafm~default_group~1.log

                  • 6. Re: Strange "adopmnctl.sh status" behaviour
                    Dafydd Griffith

                    From some Googling there seems to be two reasons why you get a "adoafmctl.sh: exiting with status 204" error. The first is that there is a clash of port numbers. The second reason is that a .lock file exists in one of the following directories:

                     

                    $INST_TOP/ora/10.1.3/j2ee/oacore/persistence/*

                    $INST_TOP/ora/10.1.3/j2ee/oafm/persistence/*

                    $INST_TOP/ora/10.1.3/j2ee/forms/persistence/*

                     

                    In my case a .lock file existed in $INST_TOP/ora/10.1.3/j2ee/oafm/persistence/*

                    directory

                     

                    Remove/rename the file and either bouncer all the apps services or just restart oafm.