3 Replies Latest reply on Aug 11, 2018 4:46 AM by Beauty_and_dBest

    Increase JavaVM size to 1024

    Beauty_and_dBest

      EBS 12.1.3

      11g

      OL5

       

       

      Hi ALL,

       

      We tried to increase JavaVM memory of our appsTier  context file  from 512 to 1024.

       

      But when we startup the apptier using adstrtal.sh,  we encountered error:

       

      Can't load '/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/auto/Sys/Hostname/Hostname.so' for module Sys::Hostname: /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/auto/Sys/Hostname/Hostname.so: wrong ELF class: ELFCLASS64 at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/XSLoader.pm line 70.

      at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/Sys/Hostname.pm line 23

       

       

      Please help....

       

      Kind regards,

      jc

        • 1. Re: Increase JavaVM size to 2048
          Beauty_and_dBest

          Hi ALL,

           

          We noticed that this error has been occuring since Feb. 2015 and yet the system is available for use.

          The only difference we noticed for the Aug 10 trail is that it mentioned looking at the HTTP Server log file.

           

           

          From the adsrtal log of node-1:

           

          Starting OPMN managed Oracle HTTP Server (OHS) instance ...

          opmnctl: opmn is already running.

          opmnctl: starting opmn managed processes...

          ================================================================================

          opmn id=node-1.fitc.com.ph:6201

          0 of 1 processes started.

           

          ias-instance id=ABC_node-1.node-1.fitc.com.ph

          ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

          --------------------------------------------------------------------------------

          ias-component/process-type/process-set:

          HTTP_Server/HTTP_Server/HTTP_Server/

           

          Error

          --> Process (index=1,uid=372911512,pid=25937)

          failed to start a managed process after the maximum retry limit

          Log:

          /fin_app/ABC/inst/apps/ABC_node-1/logs/ora/10.1.3/opmn/HTTP_Server~1.log

           

          From the HTTP Server :

           

          18/08/10 22:55:13 Start process

          --------

          /fin_app/ABC/inst/apps/ABC_node-1/ora/10.1.3/Apache/Apache/bin/apachectl startssl: execing httpd

          httpd: bad group name applABC

           

          We noticed that the above message appeared only once during the time it was set up and after our Aug 10 activity. In between these dates, this error did not appear.

           

          I am attaching the complete HTTP server log file and the Startup log file.

           

          Below are the steps that were done during yesterday's activity :

           

          For your info on the activity done @Aug 10 8:45pm

          1. Backup the Application Context File. This would be necessary if ever there is a backout.

          As applABC in node-1:

          $ cp $CONTEXT_FILE $HOME/INC000000013134

          As applABC in node-3:

          $ cp $CONTEXT_FILE $HOME/INC000000013134

          As applABC in node-2:

          $ cp $CONTEXT_FILE $HOME/INC000000013134

           

          2.Shutdown Application Tier

           

          3. Houseeking of logs in Application servers

          3.1You can delete the BNE.log file while the application is down.

           

          As applABC in node-1:

          rm /fin_app/ABC/apps/apps_st/appl/bne/12.0.0/log/BNE.log

           

          3.2. You can delete the temporary files from the following directories while the application is down.

           

          As applABC in node-1 server:

          rm /fin_app/ABC/inst/apps/ABC_node-1/appltmp/OF*.t

           

          As applABC in node-3 server:

          rm /fin_app/ABC/inst/apps/ABC_node-3/appltmp/OF*.t

           

          As applABC in node-2 server:

          rm /fin_app/ABC/inst/apps/ABC_node-2/appltmp/OF*.t

           

          3.3. You can delete files from the following directories while the application is down.

           

          As applABC in node-1 server:

          cd /fin_app/ABC/inst/apps/ABC_node-1/logs/ora/10.1.2/reports/cache

          rm *.*

           

          As applABC in node-3 server:

          cd /fin_app/ABC/inst/apps/ABC_node-3/logs/ora/10.1.2/reports/cache

          rm *.*

           

          As applABC in node-2 server:

          cd /fin_app/ABC/inst/apps/ABC_node-2/logs/ora/10.1.2/reports/cache

          rm *.*

           

          3.4. You can delete the following files older than your retention policy (i.e. 30 days)

           

          As applyABC in node-1 server:

          find /fin_app/ABC/inst/apps/ABC_node-1/logs/ora/10.1.3/Apache/access_log.* -mtime +30 -delete

          find /fin_app/ABC/inst/apps/ABC_node-1/logs/ora/10.1.3/Apache/error_log.* -mtime +30 -delete

           

          As applyABC in node-3 server:

          find /fin_app/ABC/inst/apps/ABC_node-3/logs/ora/10.1.3/Apache/access_log.* -mtime +30 -delete

          find /fin_app/ABC/inst/apps/ABC_node-3/logs/ora/10.1.3/Apache/error_log.* -mtime +30 -delete

           

          As applyABC in node-1 node-2:

          find /fin_app/ABC/inst/apps/ABC_node-2/logs/ora/10.1.3/Apache/access_log.* -mtime +30 -delete

          find /fin_app/ABC/inst/apps/ABC_node-2/logs/ora/10.1.3/Apache/error_log.* -mtime +30 -delete

           

          5. Update Context File for the JVM parameters as below

          s_oacore_jvm_start_options = -Xmx1024M -Xms1024M

          s_oacore_jvm_stop_options = -Xmx1024M -Xms1024M

           

          As applABC in node-1, node-3 and node-2

          $ vi $CONTEXT_FILE

           

          6. Run Autoconfig in Application Tier. It was completed successfully

           

          7. Startup the application TIer. The 3 nodes status is exited 0.

          When we verify the OACORE status, node-3 and node-2 are ok but not node-1 because the HTTP_SERVER is down.

           

          8. What we did is repeat the steps for node-1 and same thing happened.Unfortunately, we can't startup as well the node-2.

           

          9. Now, we can't even shutdown the services of node-1 and node-2

           

           

           

          Please help....

           

           

          Kind regards,

          • 2. Re: Increase JavaVM size to 1024
            Beauty_and_dBest

            Hi ALL,

             

            adstrtall.log

            =================================================================

             

            08/10/18-23:32:11 :: You are running adstrtal.sh version 120.15.12010000.3

             

             

            Setting Service Group Root Service to mode 2

            Setting Service Group Web Entry Point Services to mode 2

            Setting Service Group Web Application Services to mode 2

            Setting Service Group Batch Processing Services to mode 2

            Service Group Other Services is disabled.

             

             

            Executing service control script:

            /fin_app/ABC/inst/apps/ABC_node-1/admin/scripts/adopmnctl.sh start

            Timeout specified in context file: 100 second(s)

             

             

            script returned:

            ****************************************************

             

             

            You are running adopmnctl.sh version 120.6.12010000.5

             

             

            Starting Oracle Process Manager (OPMN) ...

            opmnctl: opmn started.

             

             

            adopmnctl.sh: exiting with status 0

             

             

            adopmnctl.sh: check the logfile /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/admin/log/adopmnctl.txt for more information ...

             

             

             

             

            .end std out.

             

             

            .end err out.

             

             

            ****************************************************

             

             

             

             

            Executing service control script:

            /fin_app/ABC/inst/apps/ABC_node-1/admin/scripts/adalnctl.sh start

            Timeout specified in context file: 100 second(s)

             

             

            script returned:

            ****************************************************

             

             

            adalnctl.sh version 120.3

             

             

            Checking for FNDFS executable.

            Starting listener process APPS_ABC.

             

             

            adalnctl.sh: exiting with status 0

             

             

             

             

            adalnctl.sh: check the logfile /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/admin/log/adalnctl.txt for more information ...

             

             

             

             

            .end std out.

             

             

            .end err out.

             

             

            ****************************************************

             

             

             

             

            Executing service control script:

            /fin_app/ABC/inst/apps/ABC_node-1/admin/scripts/adapcctl.sh start

            Timeout specified in context file: 100 second(s)

             

             

            script returned:

            ****************************************************

             

             

            You are running adapcctl.sh version 120.7.12010000.2

             

             

            Starting OPMN managed Oracle HTTP Server (OHS) instance ...

            opmnctl: opmn is already running.

            opmnctl: starting opmn managed processes...

            ================================================================================

            opmn id=node-1.fitc.com.ph:6201

                0 of 1 processes started.

             

             

            ias-instance id=ABC_node-1.node-1.fitc.com.ph

            ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

            --------------------------------------------------------------------------------

            ias-component/process-type/process-set:

                HTTP_Server/HTTP_Server/HTTP_Server/

             

             

            Error

            --> Process (index=1,uid=372911518,pid=5324)

                failed to start a managed process after the maximum retry limit

                Log:

                /fin_app/ABC/inst/apps/ABC_node-1/logs/ora/10.1.3/opmn/HTTP_Server~1.log

             

             

             

             

            adapcctl.sh: exiting with status 0

             

             

            adapcctl.sh: check the logfile /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/admin/log/adapcctl.txt for more information ...

             

             

             

             

            .end std out.

             

             

            .end err out.

             

             

            ****************************************************

             

             

             

             

            Executing service control script:

            /fin_app/ABC/inst/apps/ABC_node-1/admin/scripts/adoacorectl.sh start

            Timeout specified in context file: 100 second(s)

             

             

            script returned:

            ****************************************************

             

             

            You are running adoacorectl.sh version 120.13

             

             

            Starting OPMN managed OACORE OC4J instance  ...

             

             

            adoacorectl.sh: exiting with status 0

             

             

            adoacorectl.sh: check the logfile /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/admin/log/adoacorectl.txt for more information ...

             

             

             

             

            .end std out.

             

             

            .end err out.

             

             

            ****************************************************

             

             

             

             

            Executing service control script:

            /fin_app/ABC/inst/apps/ABC_node-1/admin/scripts/adformsctl.sh start

            Timeout specified in context file: 100 second(s)

             

             

            script returned:

            ****************************************************

             

             

            You are running adformsctl.sh  version 120.16.12010000.3

             

             

            Starting OPMN managed FORMS OC4J instance  ...

            Calling txkChkFormsDeployment.pl to check whether latest FORMSAPP.EAR is deployed...

            Program : /fin_app/ABC/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkChkFormsDeployment.pl started @ Fri Aug 10 23:32:40 2018

             

             

            *** Log File = /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/rgf/TXK/txkChkFormsDeployment_Fri_Aug_10_23_32_39_2018/txkChkFormsDeployment_Fri_Aug_10_23_32_39_2018.log

             

             

            File "/fin_app/ABC/apps/tech_st/10.1.3/j2ee/forms/applications/forms/formsweb/WEB-INF/lib/frmsrv.jar" exists. Proceeding to check the size...

             

             

            =============================================

            *** Latest formsapp.ear has been deployed ***

            =============================================

             

             

             

             

            Program : /fin_app/ABC/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkChkFormsDeployment.pl completed @ Fri Aug 10 23:32:40 2018

             

             

            Perl script txkChkFormsDeployment.pl got executed successfully

             

             

             

             

             

             

            adformsctl.sh: exiting with status 0

             

             

            adformsctl.sh: check the logfile /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/admin/log/adformsctl.txt for more information ...

             

             

             

             

            .end std out.

            Can't load '/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/auto/Sys/Hostname/Hostname.so' for module Sys::Hostname: /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/auto/Sys/Hostname/Hostname.so: wrong ELF class: ELFCLASS64 at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/XSLoader.pm line 70.

            at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/Sys/Hostname.pm line 23

            *** ALL THE FOLLOWING FILES ARE REQUIRED FOR RESOLVING RUNTIME ERRORS

            *** Log File = /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/rgf/TXK/txkChkFormsDeployment_Fri_Aug_10_23_32_39_2018/txkChkFormsDeployment_Fri_Aug_10_23_32_39_2018.log

             

             

            .end err out.

             

             

            ****************************************************

             

             

             

             

            Executing service control script:

            /fin_app/ABC/inst/apps/ABC_node-1/admin/scripts/adoafmctl.sh start

            Timeout specified in context file: 100 second(s)

             

             

            script returned:

            ****************************************************

             

             

            You are running adoafmctl.sh version 120.8

             

             

            Starting OPMN managed OAFM OC4J instance  ...

             

             

            adoafmctl.sh: exiting with status 0

             

             

            adoafmctl.sh: check the logfile /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/admin/log/adoafmctl.txt for more information ...

             

             

             

             

            .end std out.

             

             

            .end err out.

             

             

            ****************************************************

             

             

             

             

            Executing service control script:

            /fin_app/ABC/inst/apps/ABC_node-1/admin/scripts/adcmctl.sh start

            Timeout specified in context file: 1000 second(s)

             

             

            script returned:

            ****************************************************

             

             

            You are running adcmctl.sh version 120.17.12010000.5

             

             

            Starting concurrent manager for ABC ...

            Starting ABC_0810@ABC Internal Concurrent Manager

            Default printer is noprint

             

             

            adcmctl.sh: exiting with status 0

             

             

             

             

            adcmctl.sh: check the logfile /fin_app/ABC/inst/apps/ABC_node-1/logs/appl/admin/log/adcmctl.txt for more information ...

             

             

             

             

            .end std out.

             

             

            .end err out.

             

             

            ****************************************************

             

             

             

             

            Executing service control script:

            /fin_app/ABC/inst/apps/ABC_node-1/admin/scripts/jtffmctl.sh start

            Timeout specified in context file: 100 second(s)

             

             

            script returned:

            ****************************************************

             

             

            You are running jtffmctl.sh version 120.3

             

             

            Validating Fulfillment patch level via /fin_app/ABC/apps/apps_st/comn/java/classes

            Fulfillment patch level validated.

            Starting Fulfillment Server for ABC on port 9301 ...

             

             

            jtffmctl.sh: exiting with status 0

             

             

             

             

            .end std out.

             

             

            .end err out.

             

             

            ****************************************************

             

             

             

             

             

             

            [Service Control Execution Report]

            The report format is:

              <Service Group>            <Service>                              <Script>         <Status>

             

             

              Root Service                                                                       Enabled

              Root Service               Oracle Process Manager for ABC_node-1  adopmnctl.sh     Started

              Web Entry Point Services                                                           Enabled

              Web Entry Point Services   Oracle HTTP Server ABC_node-1          adapcctl.sh      Started

              Web Entry Point Services   OracleTNSListenerAPPS_ABC_node-1       adalnctl.sh      Started

              Web Application Services                                                           Enabled

              Web Application Services   OACORE OC4J Instance ABC_node-1        adoacorectl.sh   Started

              Web Application Services   FORMS OC4J Instance ABC_node-1         adformsctl.sh    Started

              Web Application Services   OAFM OC4J Instance ABC_node-1          adoafmctl.sh     Started

              Batch Processing Services                                                          Enabled

              Batch Processing Services  OracleConcMgrABC_node-1                adcmctl.sh       Started

              Batch Processing Services  Oracle Fulfillment Server ABC_node-1   jtffmctl.sh      Started

              Other Services                                                                     Disabled

              Other Services             OracleFormsServer-Forms ABC_node-1     adformsrvctl.sh  Disabled

              Other Services             Oracle Metrics Client ABC_node-1       adfmcctl.sh      Disabled

              Other Services             Oracle Metrics Server ABC_node-1       adfmsctl.sh      Disabled

              Other Services             Oracle MWA Service ABC_node-1          mwactlwrpr.sh    Disabled

             

             

             

             

            ServiceControl is exiting with status 0

             

             

             

             

            08/10/18-23:32:55 :: You are running adstrtal.sh version 120.15.12010000.3

             

             

             

             

            =================================================================

            • 3. Re: Increase JavaVM size to 1024
              Beauty_and_dBest

              Hi ALL,

               

              I found this in blog:

               

              Reason:

              The environment is created as shared storage multi node. The second node was cloned from the first, but in the context file, the PERL related paths were not identical to that in first node

              The paths in failed node

              <ADPERLPRG oa_var="s_adperlprg" osd="unix">/usr/bin/perl</ADPERLPRG>

              <PERL5LIB oa_var="s_perl5lib" osd="LINUX_X86-64">/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi:/usr/lib/perl5/site_perl/5.8.8:/usr/lib/perl5/site_perl:/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi:/usr/lib/perl5/vendor_perl/5.8.8:/usr/lib/perl5/vendor_perl:/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi:/usr/lib/perl5/5.8.8:.:/u01/E-BIZ/apps/apps_st/appl/au/12.0.0/perl:/u01/E-BIZ/apps/apps_st/appl/au/12.0.0/perl</PERL5LIB>


              Fix:


              Edited the perl paths (
              ADPERLPRG  and PERL5LIB) in the context file and made it identical to that in first node and restarted the services.

              Paths after editing:

              <ADPERLPRG oa_var="s_adperlprg" osd="unix">/u01/E-BIZ/apps/tech_st/10.1.3/perl/bin/perl</ADPERLPRG>

              <PERL5LIB oa_var="s_perl5lib" osd="LINUX_X86-64">/u01/E-BIZ/apps/tech_st/10.1.3/perl/lib/5.8.3:/u01/E-BIZ/apps/tech_st/10.1.3/perl/lib/site_perl/5.8.3:/u01/E-BIZ/apps/apps_st/appl/au/12.0.0/perl:/u01/E-BIZ/apps/tech_st/10.1.3/Apache/Apache/mod_perl/lib/site_perl/5.8.3/i686-linux-thread-multi</PERL5LIB>

              Restart service in second node after this.

               

               

              How do I know what is the right value to substitute? or where do I copy them?

              Please help...

               


              Kind regards,