8 Replies Latest reply: Jan 25, 2013 2:28 PM by 732311 RSS

    Webtier fault on fusion middleware 11.1.1.6.0

    732311
      Hi,
      I'm trying to install fusion middleware 11.1.1.6.0 on a Oracle Linux 6.3, but I got one error during the startup of the Web Tier
      I didn't understand the error
      "httpd.pid overwritten -- Unclean shutdown of previous Apache run?"
      I removed it and rerun the startall, without success.
      I disabled SSL
      <data id="start-mode" value="ssl-disabled"/>
      and removed the load of ssl from http.conf
      Any idea?
      Thanks and looking forward to hear from you.
      Regards,
      Lain
      [oracle@server02 bin]$ uname -a
      Linux server02 2.6.39-200.24.1.el6uek.x86_64 #1 SMP Sat Jun 23 02:39:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
      [oracle@server02 bin]$ ./opmnctl startall
      opmnctl startall: starting opmn and all managed processes...
      ================================================================================
      opmn id=server02:6702
      Response: 0 of 1 processes started.
      
      ias-instance id=instance1
      ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
      --------------------------------------------------------------------------------
      ias-component/process-type/process-set:
        ohs1/OHS/OHS/
      
      Error
      --> Process (index=1,uid=1554591991,pid=2761)
        failed to start a managed process after the maximum retry limit
        Log:
        /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/console~OHS~1.log
      [oracle@server02 bin]$ cat /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/console~OHS~1.log
      --------
      13/01/20 01:26:37 Start process
      --------
      /repository/fusion/fmw/Oracle_WT1/ohs/bin/apachectl start: execing httpd
      [Sun Jan 20 01:26:38 2013] [warn] Errors will be logged into /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/ohs1.log
      [Sun Jan 20 01:26:39 2013] [warn] Errors will be logged into /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/ohs1.log
      Audit init
      
      --------
      13/01/20 01:26:41 Start process
      --------
      /repository/fusion/fmw/Oracle_WT1/ohs/bin/apachectl start: execing httpd
      [Sun Jan 20 01:26:42 2013] [warn] Errors will be logged into /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/ohs1.log
      [Sun Jan 20 01:26:43 2013] [warn] Errors will be logged into /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/ohs1.log
      Audit init
      
      $ cat /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/ohs1.log
      
      [2013-01-20T01:26:40.0002+01:00] [OHS] [NOTIFICATION:16] [OHS-9999] [mod_weblogic.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 2751] [tid: 140693356623680] [user: oracle] [VirtualHost: main]  WebLogic Server Plugin version 1.1 <WLSPLUGINS_11.1.1.6.0_LINUX.X64_RELEASE>
      
      [2013-01-20T01:26:40.0161+01:00] [OHS] [WARNING:32] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 2751] [tid: 140693356623680] [user: oracle] [VirtualHost: main]  pid file /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
      
      [2013-01-20T01:26:40.0167+01:00] [OHS] [INCIDENT_ERROR:10] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 2751] [tid: 140693356623680] [user: oracle] [VirtualHost: main] (2)No such file or directory:  Couldn't create accept lock
      
      [2013-01-20T01:26:44.0002+01:00] [OHS] [NOTIFICATION:16] [OHS-9999] [mod_weblogic.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 2761] [tid: 140443039262528] [user: oracle] [VirtualHost: main]  WebLogic Server Plugin version 1.1 <WLSPLUGINS_11.1.1.6.0_LINUX.X64_RELEASE>
      
      [2013-01-20T01:26:44.0161+01:00] [OHS] [WARNING:32] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 2761] [tid: 140443039262528] [user: oracle] [VirtualHost: main]  pid file /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
      
      [2013-01-20T01:26:44.0167+01:00] [OHS] [INCIDENT_ERROR:10] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 2761] [tid: 140443039262528] [user: oracle] [VirtualHost: main] (2)No such file or directory:  Couldn't create accept lock
        • 1. Re: Webtier fault on fusion middleware 11.1.1.6.0
          Kishore Rout
          Hi,

          The HTTP_Server component had crashed or was shutdown uncleanly. Please shutdown the OHS instance, if it is running using the command ./opmnctl stop all. Using the below command please check whether any process is running.

          netstat -a |grep port (port:OHS listening port)
          Ps -ef |grep PID (you will get the pid from httpd.pid file) (if you are getting any process kill it using command kill -9 pid)

          if above command returns zero output please remoe httpd.pid file and restart OHS.

          If this help please mark.

          Regards,
          Kishore
          • 2. Re: Webtier fault on fusion middleware 11.1.1.6.0
            732311
            Hi KishoreRout,
            thanks for your answer. Unfortunately I didn’t find anything and I got the same error:
            what I was missing? or maybe I should stop everything and try to start again.
            Thanks for your time.
            Lain
            [oracle@server02 ~]$ netstat -a | grep 7777
            [oracle@server02 ~]$ netstat -a | grep 7001
            [oracle@server02 ~]$ ps -ef | grep http
            oracle   22638 22455  0 09:49 pts/2    00:00:00 grep http
            [oracle@server02 ~]$ cd /repository/fusion/fmw/Oracle_WT1/instances/instance1/bin
            [oracle@server02 bin]$ rm /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/httpd.pid
            [oracle@server02 bin]$ ./opmnctl startall
            opmnctl startall: starting opmn and all managed processes...
            ================================================================================
            opmn id=server02:6702
            Response: 0 of 1 processes started.
            
            ias-instance id=instance1
            ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
            --------------------------------------------------------------------------------
            ias-component/process-type/process-set:
              ohs1/OHS/OHS/
            
            Error
            --> Process (index=1,uid=1554591993,pid=22910)
              failed to start a managed process after the maximum retry limit
              Log:
              /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/console~OHS~1.log
            
            [oracle@server02 bin]$ cat /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/ohs1.log
            [2013-01-21T09:53:48.0002+01:00] [OHS] [NOTIFICATION:16] [OHS-9999] [mod_weblogic.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 22900] [tid: 140352846595904] [user: oracle] [VirtualHost: main]  WebLogic Server Plugin version 1.1 <WLSPLUGINS_11.1.1.6.0_LINUX.X64_RELEASE>
            
            [2013-01-21T09:53:48.0168+01:00] [OHS] [WARNING:32] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 22900] [tid: 140352846595904] [user: oracle] [VirtualHost: main]  pid file /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
            
            [2013-01-21T09:53:48.0174+01:00] [OHS] [INCIDENT_ERROR:10] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 22900] [tid: 140352846595904] [user: oracle] [VirtualHost: main] (2)No such file or directory:  Couldn't create accept lock
            
            [2013-01-21T09:53:51.0009+01:00] [OHS] [NOTIFICATION:16] [OHS-9999] [mod_weblogic.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 22910] [tid: 140415296210752] [user: oracle] [VirtualHost: main]  WebLogic Server Plugin version 1.1 <WLSPLUGINS_11.1.1.6.0_LINUX.X64_RELEASE>
            
            [2013-01-21T09:53:51.0174+01:00] [OHS] [WARNING:32] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 22910] [tid: 140415296210752] [user: oracle] [VirtualHost: main]  pid file /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
            
            [2013-01-21T09:53:51.0183+01:00] [OHS] [INCIDENT_ERROR:10] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 22910] [tid: 140415296210752] [user: oracle] [VirtualHost: main] (2)No such file or directory:  Couldn't create accept lock
            • 3. Re: Webtier fault on fusion middleware 11.1.1.6.0
              Kishore Rout
              Hi Lain,
              Can you please check below entry in httpd.conf file. Please make sure that Lockfile part should be hashed out inside your httpd.conf file.

              <IfModule !mpm_winnt_module>
              #LockFile "${ORACLE_INSTANCE}/diagnostics/logs/${COMPONENT_TYPE}/${COMPONENT_NAME}/accept.lock"
              </IfModule>

              Regards,
              Kishore
              • 4. Re: Webtier fault on fusion middleware 11.1.1.6.0
                732311
                Dear KishoreRout,
                thanks for your quickly answer.
                Unfortunately it is, this is my httpd.conf in my path: /repository/fusion/fmw/Oracle_WT1/ohs/conf/httpd.conf
                <IfModule !mpm_winnt_module>
                #LockFile "${ORACLE_INSTANCE}/diagnostics/logs/${COMPONENT_TYPE}/${COMPONENT_NAME}/accept.lock"
                </IfModule>
                I tried to put the # in these LockFile, too. Unfortunately nothing changed. I got the same error.
                Thanks,
                <IfModule mpm_prefork_module>
                StartServers         5
                MinSpareServers      5
                MaxSpareServers     10
                MaxClients         150
                MaxRequestsPerChild  0
                AcceptMutex fcntl
                LockFile "${ORACLE_INSTANCE}/diagnostics/logs/${COMPONENT_TYPE}/${COMPONENT_NAME}/http_lock"
                </IfModule>
                
                <IfModule mpm_worker_module>
                StartServers         2
                MaxClients         150
                MinSpareThreads     25
                MaxSpareThreads     75
                ThreadsPerChild     25
                MaxRequestsPerChild  0
                AcceptMutex fcntl
                LockFile "${ORACLE_INSTANCE}/diagnostics/logs/${COMPONENT_TYPE}/${COMPONENT_NAME}/http_lock"
                </IfModule>
                • 5. Re: Webtier fault on fusion middleware 11.1.1.6.0
                  Kishore Rout
                  Hi Lain,
                  You are hashing out http.lock. But I have asked you to hash out accept.lock in httpd.conf file. Can you please try the same.

                  Regards,
                  Kishore
                  • 6. Re: Webtier fault on fusion middleware 11.1.1.6.0
                    732311
                    Yes I did and sorry I didn't understand before.
                    I removed the # from my httpd.conf. This is my file:
                    vi /repository/fusion/fmw/Oracle_WT1/ohs/conf/httpd.conf
                    ...
                    <IfModule !mpm_winnt_module>
                    LockFile "${ORACLE_INSTANCE}/diagnostics/logs/${COMPONENT_TYPE}/${COMPONENT_NAME}/accept.lock"
                    </IfModule>
                    ...
                    but I've got the same error:
                    [oracle@server02 bin]$ ./opmnctl stopall
                    opmnctl stopall: stopping opmn and all managed processes...
                    [oracle@server02 bin]$ rm /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/httpd.pid
                    [oracle@server02 bin]$ ./opmnctl startall
                    opmnctl startall: starting opmn and all managed processes...
                    this is the log:
                    [2013-01-23T17:52:51.0004+01:00] [OHS] [NOTIFICATION:16] [OHS-9999] [mod_weblogic.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 7431] [tid: 140305699870528] [user: oracle] [VirtualHost: main]  WebLogic Server Plugin version 1.1 <WLSPLUGINS_11.1.1.6.0_LINUX.X64_RELEASE>
                    
                    [2013-01-23T17:52:51.0166+01:00] [OHS] [INCIDENT_ERROR:10] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 7431] [tid: 140305699870528] [user: oracle] [VirtualHost: main] (2)No such file or directory:  Couldn't create accept lock
                    
                    [2013-01-23T17:52:54.0002+01:00] [OHS] [NOTIFICATION:16] [OHS-9999] [mod_weblogic.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 7466] [tid: 140559583266624] [user: oracle] [VirtualHost: main]  WebLogic Server Plugin version 1.1 <WLSPLUGINS_11.1.1.6.0_LINUX.X64_RELEASE>
                    
                    [2013-01-23T17:52:54.0159+01:00] [OHS] [WARNING:32] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 7466] [tid: 140559583266624] [user: oracle] [VirtualHost: main]  pid file /repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
                    
                    [2013-01-23T17:52:54.0165+01:00] [OHS] [INCIDENT_ERROR:10] [OHS-9999] [core.c] [host_id: server02] [host_addr: 192.168.10.40] [pid: 7466] [tid: 140559583266624] [user: oracle] [VirtualHost: main] (2)No such file or directory:  Couldn't create accept lock
                    Thanks for your suggest and time.
                    Lain
                    • 7. Re: Webtier fault on fusion middleware 11.1.1.6.0
                      732311
                      I can't go ahead.
                      Someone could help me, please? Every idea could be fine for me.
                      Best regards,
                      Maurizio

                      I found this link: http://docs.oracle.com/cd/E29505_01/fusionapps.1111/e16602/rupinstall.htm
                      >
                      5.7.20.3 Startup Failed for CommonDomain:OHSComponent (Oracle VM Only)

                      Problem

                      The OHS diagnostic log contains the following error message:

                      No such file or directory: Couldn't create accept lock
                      Solution

                      This issue could be the result of the hypervisors going down, resulting in bringing the Oracle VM servers down. Perform the following steps to resolve the error:

                      Find the entry for the lock file in httpd.config, for example:

                      LockFile "/u101/ohs_inst1/diagnostics/logs/OHS/ohs1/http_lock"
                      Confirm whether the directory that contains the lock file exists.

                      Assuming this directory does not exist, create the directory.
                      >

                      The directory exists but empty because it can't create the file. I must find what's the problem, because I'm sure that it's related to LockFile from httpd.conf
                      [OHS11g@server02 bin]$ls "/repository/fusion/fmw/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1/http_lock"
                      [OHS11g@server02 bin]$
                      Regards,

                      Edited by: Lain on 25-gen-2013 20.22
                      • 8. Re: Webtier fault on fusion middleware 11.1.1.6.0
                        732311
                        I resolved the problem.
                        the httpd.conf was wrong.
                        the correct file is that:
                        /repository/fusion/fmw/Oracle_WT1/instances/instance1/config/OHS/ohs1/httpd.conf
                        I was trying to change this one: /repository/fusion/fmw/Oracle_WT1/ohs/conf/httpd.conf
                        Thanks for the support, question resolved!