0 Replies Latest reply: Feb 8, 2013 1:56 AM by 423814 RSS

    ohs1 (Apache child process) keeps crashing

    423814
      I installed and configured the following on a Windows 2008 R2 server:
      1. WLS 10.3.6 32bit
      2. WLS patch PATCH_ID - MYFD, Patch number - 14331527
      3. Forms Services 11.1.2.0.0 32bit
      4. Forms Services 11.1.2.1.0 patch
      5. Created domain

      There were no problems during installation and configuration.

      AdminServer and WLS_FORMS were successfully started. I can run the test.fmx through the ports 8888 and 9001. I can open the FMW welcome page.

      Then I tried to add port 80 to ohs1 using the Enterprise Manager 11g Fusion Middleware Control (Web Tier -> ohs1 ->Oracle HTTP Server -> Administration -> Ports Configuration). It took several minutes before getting the following error message:

      Error in querying ports for component ohs1. Cause: Load failed with IOException

      oracle.sysman.emSDK.app.exception.EMSystemException
           at oracle.sysman.emas.model.vs.AllComponentEndpointsAdminModel.initializeData(AllComponentEndpointsAdminModel.java:230)
           at oracle.sysman.core.model.util.ModelUtil$EMObjectResetter.initOrResetEMObject(ModelUtil.java:319)
           at oracle.sysman.core.model.util.ModelUtil$EMObjectResetter.access$000(ModelUtil.java:280)
      ...

      The root cause is that Apache child process keeps crashing.

      Some lines from ohs1.log:

      ...
      [2013-02-05T08:37:17.8438+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 1272] [tid: 604] [user: SYSTEM] [VirtualHost: main] Parent: child process exited with status 255 -- Restarting.

      [2013-02-05T08:37:19.3570+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 1272] [tid: 604] [user: SYSTEM] [VirtualHost: main] Oracle-Application-Server-11g/11.1.1.6.0 Oracle-HTTP-Server (Win32) mod_plsql/11.1.1.0.0 mod_ssl/2.2.21 OtherSSL/0.0.0 DAV2/1.0.3.3.0-0052 (Oracle OraDAV enabled) configured -- resuming normal operations

      [2013-02-05T08:37:19.3570+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 1272] [tid: 604] [user: SYSTEM] [VirtualHost: main] Server built: Nov 12 2011 03:38:21 , Server Label: unknown

      [2013-02-05T08:37:19.3570+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 1272] [tid: 604] [user: SYSTEM] [VirtualHost: main] Parent: Created child process 8588

      [2013-02-05T08:37:21.3694+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 624] [user: SYSTEM] [VirtualHost: main] Child 8588: Child process is running

      [2013-02-05T08:37:21.3850+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [mod_weblogic.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 624] [user: SYSTEM] [VirtualHost: main] WebLogic Server Plugin version 1.1 <WLSPLUGINS_11.1.1.6.0_NT_111122.1130>

      [2013-02-05T08:37:21.3850+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 624] [user: SYSTEM] [VirtualHost: main] Child 8588: Acquired the start mutex.

      [2013-02-05T08:37:21.3850+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 624] [user: SYSTEM] [VirtualHost: main] Child 8588: Starting 150 worker threads.

      [2013-02-05T08:37:21.4006+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 1572] [user: SYSTEM] [VirtualHost: main] Child 8588: Starting thread to listen on port 8889.

      [2013-02-05T08:37:21.4006+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 1592] [user: SYSTEM] [VirtualHost: main] Child 8588: Starting thread to listen on port 8889.

      [2013-02-05T08:37:21.4006+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 1624] [user: SYSTEM] [VirtualHost: main] Child 8588: Starting thread to listen on port 8890.

      [2013-02-05T08:37:21.4006+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 1644] [user: SYSTEM] [VirtualHost: main] Child 8588: Starting thread to listen on port 8890.

      [2013-02-05T08:37:21.4006+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 1688] [user: SYSTEM] [VirtualHost: main] Child 8588: Starting thread to listen on port 8888.

      [2013-02-05T08:37:21.4006+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 8588] [tid: 1708] [user: SYSTEM] [VirtualHost: main] Child 8588: Starting thread to listen on port 8888.

      [2013-02-05T08:39:22.5530+02:00] [OHS] [NOTIFICATION:16] [OHS-9999] [core.c] [host_id: slogtetia01] [host_addr: 192.168.16.142] [pid: 1272] [tid: 604] [user: SYSTEM] [VirtualHost: main] Parent: child process exited with status 255 -- Restarting.
      ...

      Some lines from Windows event log:
      ...
      Level     Date and Time     Source     Event ID     Task Category
      Error     1.2.2013 14:48:02     Application Error     1000     (100)     "Faulting application name: Apache.exe, version: 2.2.21.0, time stamp: 0x4ebe3e92
      Faulting module name: KERNELBASE.dll, version: 6.1.7601.17651, time stamp: 0x4e211319
      Exception code: 0xebf00baa
      Fault offset: 0x0000b9bc
      Faulting process id: 0x1ddc
      Faulting application start time: 0x01ce007a15e9fef5
      Faulting application path: D:\Oracle\Middleware\as_1\ohs\bin\Apache.exe
      Faulting module path: C:\Windows\syswow64\KERNELBASE.dll
      Report Id: 9cbaa850-6c6d-11e2-a0fe-92cd07cfbce2"
      Error     1.2.2013 14:45:58     Application Error     1000     (100)     "Faulting application name: Apache.exe, version: 2.2.21.0, time stamp: 0x4ebe3e92
      Faulting module name: KERNELBASE.dll, version: 6.1.7601.17651, time stamp: 0x4e211319
      Exception code: 0xebf00baa
      Fault offset: 0x0000b9bc
      Faulting process id: 0x215c
      Faulting application start time: 0x01ce0079cbfdeca7
      Faulting application path: D:\Oracle\Middleware\as_1\ohs\bin\Apache.exe
      Faulting module path: C:\Windows\syswow64\KERNELBASE.dll
      Report Id: 52d0f763-6c6d-11e2-a0fe-92cd07cfbce2"
      ...

      The weird thing is that I have successfully installed the same stuff on a different Win2008R2 running under WMWare. The main difference is that in that system IPV6 is disabled. The problem system is multi-homed vs. the success system has one nic. I have tried to set all listener addresses to ip:port, though. No luck.

      The version of the KERNELBASE.DLL is the same in both systems.

      I have opened an SR, but no help from there yet.

      Any ideas?