2 Replies Latest reply on Apr 29, 2020 1:53 AM by Beauty_and_dBest

    Today, Our EBS 12.2.9 is dying often

    Beauty_and_dBest

      12.2.9

      12c

      OL7

       

       

      Hi ALL,

       

      Our EBS 12.2.9  which is running for sometime,  suddenly today the apps is keep dying with error:

      Then error will turn to > This site can not be reach.

       

      We keep on bouncing it stop/start  but the issue persist after  an hour?

      Could the developers have done something to make this issue?

      I checked database alert log and no issue on the database.

       

      I see some error in EBS_domain.log, but I am not sure if this is the cause?

       

      EBS_domain.log

      ####<Apr 28, 2020 1:03:27 PM PHT> <Error> <ServletContext-/forms> <lccatest.lccgroup.com> <forms_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <005d5B0X1TiDwW0Azzl3hG0005yo000BAn> <1588050207713> <BEA-000000> <Forms session <173> exception stack trace:

      java.io.IOException: FRM-93000: Unexpected internal error.

      Details : No HTTP headers received from runform

      at oracle.forms.servlet.ListenerServlet.forwardResponseFromRunform(Unknown Source)

      at oracle.forms.servlet.ListenerServlet.doPost(Unknown Source)

      at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)

      at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

      at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)

      at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)

       

      AdminServer-diagnostic.log

      Caller should ensure that a DefaultContext is proper for this use.

      Memory leaks and/or unexpected behaviour may occur if the automatic initialization is performed improperly.

      This message may be avoided by performing initADFContext before using getCurrent().

      For more information please enable logging for oracle.adf.share.ADFContext at FINEST level.

       

      Please see the documentation for more information about handling ADFContext leaks.

      For more information about the leaking ADFContext please enable logging for oracle.adf.share.ADFContext at FINEST level.

       

      [2020-04-28T14:42:02.045+08:00] [AdminServer] [NOTIFICATION] [] [oracle.dms.event] [tid: JMX FRAMEWORK Domain Runtime MBeanServer pooling thread] [userId: <WLS Kernel>] [ecid: 0000N6zXLR4DwW0Azzl3hG1Udx0p000002,0] DMS-58081:

      The flight recorder destination (id=JFRDestination) can not function in this environment because although the JVMs flight recorder feature is present it is inactive. Platform:WebLogic, VM:{'name'='Java HotSpot(TM) 64-Bit Server VM', 'version'='24.85-b06', 'spec-version'='1.7'}.[[

       

      Cause: The JVM has a flight recorder feature but the flight recorder feature is not active.

      Action: Review the JVMs flight recorder settings and if you wish DMS to contribute data to the flight recorder then enable the flight recorder feature. This can be accomplished on some JVMs by including these command line options: -XX:+UnlockCommercialFeatures -XX:+FlightRecorder. In the meantime the event route involving the JFRDestination will be disabled.

       

      Oacore_server1.out

      java.lang.OutOfMemoryError: GC overhead limit exceeded

      at java.lang.reflect.Method.copy(Method.java:151)

      at java.lang.reflect.ReflectAccess.copyMethod(ReflectAccess.java:136)

       

       

       

      Please help how to toubleshoot.

       

       

      Kind regards,

      jc

        • 1. Re: Today, Our EBS 12.2.9 is dying often
          mdtaylor

          I would look at $FMW_Home/user_projects/domains/EBS_domain/servers/oacore_server1/logs/oacore_server1.log

           

          Also look at your oacore JVM heap size.  You may need to increase it to 2048M.

           

          <oacore_jvm_start_options oa_var="s_oacore_jvm_start_options">-XX:PermSize=128m -XX:MaxPermSize=384m -Xms1024m -Xmx1024m -Djava.security

           

          Also look at processes and sessions init.ora parameters.  You should probably set sessions to 2000, processes to 1000 and set your EBSDataSource Maximum Capacity to 1000.

          1 person found this helpful
          • 2. Re: Today, Our EBS 12.2.9 is dying often
            Beauty_and_dBest

            Thanks Michael

             

             

            Can I followup my other post please....report error when cloning EBS 12.2.4 from OL6.5  to OL6.10

             

            Cloning EBS 12.2.4 from OL6.5 to OL6.10

            Oracle Reports "Segmentation fault (core dumped)" error

             

            Section 5: Upgrading to Latest Java 7.0 in OracleAS 10.1.2 Oracle_Home

             

            Which Home below is OracleAS 10.1.2?

             

             

            $COMMON_TOP/util/jdk32/bin/java -version

            java version "1.7.0_67"

            Java(TM) SE Runtime Environment (build 1.7.0_67-b01)

            Java HotSpot(TM) Server VM (build 24.65-b04, mixed mode)

             

             

            $FMW_HOME/webtier/jdk/bin/java -version

            java version "1.6.0_29"

            Java(TM) SE Runtime Environment (build 1.6.0_29-b11)

            Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02, mixed mode)

             

             

            $ORACLE_HOME/jdk/bin/java -version

            java version "1.7.0_67"

            Java(TM) SE Runtime Environment (build 1.7.0_67-b01)

            Java HotSpot(TM) Server VM (build 24.65-b04, mixed mode)

             

             

            Kind regards,