1 2 3 Previous Next 32 Replies Latest reply on Jun 1, 2020 9:35 AM by WF001

    Apply AD R12.2 and CPU Upgrade patches issues

    WF001

      Hi Experts,

       

      I'm trying to upgrade EBS from 12.1.3 to 12.2 in 19c DB,  I've completed R12.2 rapid install to lay down the file system and technology stack (After laying down the File System using the Rapid Install). And then applied the latest application tier technology stack patches (using checkMTpatch.sh in R12.2).

       

      1) But I am confused with the readme of patch 10117518 (Apply the AD Upgrade Patch 10117518 by merging it with the latest Consolidated Upgrade Patch (CUP) for AD (Patch 28853096:R12.AD.C)) and the Preparation Tasks is run adgrants.sql, does it mean there is using the adgrants.sql that from merged directory, not from patch 10117518 directory?I tried to run the "@adgrants.sql apps" that from merged directory ($PATCH_TOP/10117518_CUP10/admin/adgrants.sql), but it shown errors as below (Can I ignored it?):

      ###############

      --- adgrants.sql started at 2020-05-18 21:10:32 ---

      Creating PL/SQL profiler objects.

      ........

      Start of PURGE DBA_RECYCLEBIN.

      PL/SQL procedure successfully completed.

       

      declare

      *

      ERROR at line 1:

      ORA-00942: table or view does not exist

      ORA-06512: at line 20

       

      The following ERRORS and WARNINGS have been encountered during this adgrants session:

      ERROR:

      ORA-24338: statement handle not executed

       

      SP2-0625: Error printing variable "result_set"

       

      Grants given by this script have been written to the ad_zd_logs table.

       

      You can run $AD_TOP/sql/ADZDSHOWLOG.sql to produce a report showing these grants.

       

      Commit complete.

       

      Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

       

      Version 19.5.0.0.0

       

      #################

       

      2) And then I tried to apply the merged patch driver u_10117518_cup10.drv, but seem the patch is hanging on "completed file afsys.sql".  Worker 3-8 is running with no progress for more than 48 hours while the other workers 1-2 are in wait state. (I'm using 8 workers to run it, can I increase the no. of workers to apply patches in 4 cpu cores AIX server?)

      - adpatch log:

      #################

      ......

      There are now 79 jobs remaining (current phase=A2):

          0 running, 1 ready to run and 78 waiting.

       

      Assigned: file ADZDCTXB.pls on worker  1 for product ad  username APPLSYS.

      Completed: file ADZDCTXB.pls on worker  1 for product ad  username APPLSYS.

       

      Starting phase 3 (A3): con

       

      There are now 78 jobs remaining (current phase=A3):

          0 running, 9 ready to run and 69 waiting.

       

      Assigned: file ADZDTABC.sql on worker  1 for product ad  username APPLSYS.

      Assigned: file addpapuu.sql on worker  2 for product ad  username APPLSYS.

      Assigned: file ad_objects.xdf on worker  3 for product ad  username APPS.

      Assigned: file ad_patched_tables.xdf on worker  4 for product ad  username APPS.

      Assigned: file ad_zd_ddl_handler.xdf on worker  5 for product ad  username APPS.

      Assigned: file ad_zd_ddl_handler_ddl_s.xdf on worker  6 for product ad  username APPS.

      Assigned: file ad_zd_logs.xdf on worker  7 for product ad  username APPS.

      Assigned: file ad_zd_logs_s.xdf on worker  8 for product ad  username APPS.

      Completed: file ADZDTABC.sql on worker  1 for product ad  username APPLSYS.

      Assigned: file afsys.sql    on worker  1 for product fnd username APPLSYS.

      Completed: file addpapuu.sql on worker  2 for product ad  username APPLSYS.

      Completed: file afsys.sql    on worker  1 for product fnd username APPLSYS.

       

      #################

       

      - The status of Adworkers:

      1 Run AutoPatch R120 Wait

      2 Run AutoPatch R120 Wait

      3 Run AutoPatch R120 ad_objects.xdf Running

      4 Run AutoPatch R120 ad_patched_tables.xdf Running

      5 Run AutoPatch R120 ad_zd_ddl_handler.xdf Running

      6 Run AutoPatch R120 ad_zd_ddl_handler_ddl_s.xd Running

      7 Run AutoPatch R120 ad_zd_logs.xdf Running

      8 Run AutoPatch R120 ad_zd_logs_s.xdf Running

       

      - I checked workers3-8 logs and I found ORA- errors.  See the error below:

      - adwork003 to adwork008 logs

      ###############

      .....

      NOTE: You may not use this utility for custom development

            unless you have written permission from Oracle Corporation.

       

      Reading product information from file...

       

      Reading language and territory information from file...

       

      Reading language information from applUS.txt ...

        Temporarily resetting CLASSPATH to:

       

      "/R12.2/oracle/SID/fs1/EBSapps/appl/ad/12.0.0/java/adjava.zip:/R12.2/oracle/SID/fs1/EBSapps/comn/util/jdk32/lib/dt.jar:/R12.2/oracle/SID/fs1/EBSapps/comn/util/jdk32/lib/tools.jar:/R12.2/oracle/SID/fs1/EBSapps/comn/util/jdk32/jre/lib/rt.jar:/R12.2/oracle/SID/fs1/EBSapps/comn/shared-libs/ebs-3rdparty/WEB-INF/lib/ebs3rdpartyManifest.jar:/R12.2/oracle/SID/fs1/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-fmw/WEB-INF/lib/ebsFMWManifest.jar:/R12.2/oracle/SID/fs1/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-appsborg/WEB-INF/lib/ebsAppsborgManifest.jar:/R12.2/oracle/SID/fs1/FMW_Home/oracle_common/modules/oracle.uix_11.1.1/uix2.jar:/R12.2/oracle/SID/fs1/EBSapps/comn/java/classes:/R12.2/oracle/SID/fs1/EBSapps/10.1.2/forms/java:/R12.2/oracle/SID/fs1/EBSapps/10.1.2/forms/java/frmall.jar:/R12.2/oracle/SID/fs1/EBSapps/10.1.2/jlib/ewt3.jar:/R12.2/oracle/SID/fs1/EBSapps/10.1.2/j2ee/OC4J_BI_Forms/applications/formsapp/formsweb/WEB-INF/lib/frmsrv.jar"

       

        Calling /R12.2/oracle/SID/fs1/EBSapps/comn/util/jdk32/jre/bin/java ...

      Exception in thread "main" java.sql.SQLRecoverableException: ORA-01034: ORACLE not available

      ORA-27101: shared memory realm does not exist

      IBM AIX RISC System/6000 Error: 2: No such file or directory

      Additional information: 2820

      Additional information: -1500894085

       

      at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:462)

      at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:397)

      at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:389)

      at oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:451)

      at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:481)

      at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:205)

      at oracle.jdbc.driver.T4CTTIoauthenticate.doOSESSKEY(T4CTTIoauthenticate.java:412)

      at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:403)

      at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:678)

      at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:235)

      at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:34)

      at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:567)

      at java.sql.DriverManager.getConnection(DriverManager.java:583)

      at java.sql.DriverManager.getConnection(DriverManager.java:227)

      at oracle.apps.ad.worker.AdJavaWorker.getAppsConnection(AdJavaWorker.java:1041)

      at oracle.apps.ad.worker.AdJavaWorker.main(AdJavaWorker.java:276)

       

      ##################

       

      Many thanks for your advice.

        • 1. Re: Apply AD R12.2 and CPU Upgrade patches issues
          yoonas

          Hi,

           

          2. Compare the version of adgrants.sql(UNIX) in $APPL_TOP/admin
               to that in patch directory.

          if i read the read me properly, The preparation tasks says to run adgrants from the path directory without any mentioning of merging

           

          Did you try to run adgrants from the patch ?

           

          There is comment also in the comment discussion section

          Selection_018.png

           

          Regards,

          Yoonas

          • 2. Re: Apply AD R12.2 and CPU Upgrade patches issues
            WF001

            Hi yoonas

             

            Yes, at first round, I tried to run adgrants from the patch 10117518 directory without any mentioning of merging, then I tried to apply the merged patch driver u_10117518_cup10.drv, but it shows AutoPatch error as below. So I try to run the "@adgrants.sql apps" that from merged directory ($PATCH_TOP/10117518_CUP10/admin/adgrants.sql)

             

            AutoPatch error:

            Please apply adgrants.sql file in database tier before applying this patch

             

            Thanks

            • 3. Re: Apply AD R12.2 and CPU Upgrade patches issues
              yoonas

              Exception in thread "main" java.sql.SQLRecoverableException: ORA-01034: ORACLE not available

              ORA-27101: shared memory realm does not exist

              Noticed it just now

              Database is not up!!, did you check alert log ?

              • 4. Re: Apply AD R12.2 and CPU Upgrade patches issues
                WF001

                Hi yoonas

                 

                Found the following errors in alert log, do you have any advise? Many thanks.


                2020-05-18T22:15:02.280193+08:00

                Completed checkpoint up to RBA [0x110.2.10], SCN: 8291732257163

                2020-05-18T22:23:37.026440+08:00

                Incremental checkpoint up to RBA [0x110.1a4ad.0], current log tail at RBA [0x110.2bdc7.0]

                2020-05-18T22:37:59.278620+08:00

                Thread 1 cannot allocate new log, sequence 273

                Private strand flush not complete

                  Current log# 2 seq# 272 mem# 0: /ofdb/ofcdb/OFCDB/onlinelog/o1_mf_2_h60sgxj9_.log

                  Current log# 2 seq# 272 mem# 1: /ofdb/fast_recovery_area/OFCDB/onlinelog/o1_mf_2_h60sh3cl_.log

                2020-05-18T22:38:02.504498+08:00

                Beginning log switch checkpoint up to RBA [0x111.2.10], SCN: 8291734578949

                2020-05-18T22:38:02.528505+08:00

                Thread 1 advanced to log sequence 273 (LGWR switch)

                  Current log# 3 seq# 273 mem# 0: /ofdb/ofcdb/OFCDB/onlinelog/o1_mf_3_h60sgxks_.log

                  Current log# 3 seq# 273 mem# 1: /ofdb/fast_recovery_area/OFCDB/onlinelog/o1_mf_3_h60sh4r2_.log

                2020-05-18T22:40:01.096163+08:00

                SID(3):Errors in file /oracle/diag/rdbms/ofcdb/ofcdb/trace/ofcdb_j003_15794604.trc:

                ORA-12012: error on auto execute of job "SYS"."ORA$AT_SA_SPC_SY_7911"

                ORA-01555: snapshot too old: rollback segment number 11 with name "_SYSSMU11_3275849776$" too small

                ORA-06512: at "SYS.DBMS_ADVISOR", line 226

                ORA-06512: at "SYS.PRVT_ADVISOR", line 3546

                ORA-06512: at "SYS.PRVT_ADVISOR", line 932

                ORA-06512: at "SYS.WRI$_ADV_OBJSPACE_TREND_T", line 1882

                ORA-06512: at "SYS.PRVT_ADVISOR", line 915

                ORA-06512: at "SYS.PRVT_ADVISOR", line 3451

                ORA-06512: at "SYS.DBMS_ADVISOR", line 276

                ORA-06512: at "SYS.DBMS_ADVISOR", line 221

                ORA-06512: at "SYS.DBMS_SPACE", line 2484

                ORA-06512: at "SYS.DBMS_SPACE", line 2774

                2020-05-18T22:43:06.240215+08:00

                Completed checkpoint up to RBA [0x111.2.10], SCN: 8291734578949

                2020-05-18T22:43:45.998427+08:00

                • 5. Re: Apply AD R12.2 and CPU Upgrade patches issues
                  yoonas

                  This is the only error you could see in the alert log?

                   

                  Exception in thread "main" java.sql.SQLRecoverableException: ORA-01034: ORACLE not available

                  ORA-27101: shared memory realm does not exist

                  Database has been restarted after you have seen this error?

                   

                  What is the patch status now?

                   

                  Can you please attach worker and alert log file.

                  • 6. Re: Apply AD R12.2 and CPU Upgrade patches issues
                    WF001

                    Hi yoonas

                    Thanks again,

                     

                    I didn't restart the database, and I check the database start time is in last month if sql is correct (p.s. The database of EBS R12.1.3 have been upgraded from 11g to 19c in Apr by DBA, and apply the AD R12.2 and CPU Upgrade patches is in May).

                    SQL> SELECT INSTANCE_NAME,TO_CHAR(STARTUP_TIME, 'HH24:MI DD-MON-YY') FROM DBA_HIST_DATABASE_INSTANCE ORDER BY STARTUP_TIME DESC;

                    INSTANCE_NAME  TO_CHAR(STARTUP_TI

                    ------------------------- -------------------------------------

                    ofcdb                       21:49 28-APR-20

                     

                    - The status of workers  :

                    Control

                    Worker Code Context Filename Status

                    ------ -------- ----------------- -------------------------- --------------

                    1 Run AutoPatch R120 Wait

                    2 Run AutoPatch R120 Wait

                    3 Run AutoPatch R120 ad_objects.xdf Running

                    4 Run AutoPatch R120 ad_patched_tables.xdf Running

                    5 Run AutoPatch R120 ad_zd_ddl_handler.xdf Running

                    6 Run AutoPatch R120 ad_zd_ddl_handler_ddl_s.xd Running

                    7 Run AutoPatch R120 ad_zd_logs.xdf Running

                    8 Run AutoPatch R120 ad_zd_logs_s.xdf Running

                     

                     

                    - Sorry, I don't know how to upload the file in here.
                    For example, The adwork003.log is as below

                    ************* Start of AD Worker session *************

                    AD Worker version: 12.2.0

                    AD Worker started at: Mon May 18 2020 22:21:03

                     

                    APPL_TOP is set to /oracle/SID/fs1/EBSapps/appl

                     

                    Worker id = 3

                    Started by: adpatch

                    ...

                    ............

                    Connecting to SYSTEM......Connected successfully.

                     

                    Already asked for the ORACLE username and password

                    of Application Object Library.

                    (The given username/password was: APPLSYS/*****)

                     

                    Connecting to APPLSYS......Connected successfully.

                     

                    The status of various features in this run of AD Worker is:

                                                               <-Feature version in->

                    Feature                          Active?   APPLTOP    Data model    Flags

                    ------------------------------   -------   --------   -----------   -----------

                    CHECKFILE                        Yes       1          1             Y N N Y N Y

                    PREREQ                           Yes       6          6             Y N N Y N Y

                    CONCURRENT_SESSIONS              No        2          2             Y Y N Y Y N

                    PATCH_TIMING                     Yes       2          2             Y N N Y N Y

                    PATCH_HIST_IN_DB                 Yes       6          6             Y N N Y N Y

                    SCHEMA_SWAP                      Yes       1          1             Y N N Y Y Y

                    JAVA_WORKER                      Yes       1          1             Y N N Y N Y

                    CODELEVEL                        Yes       1          1             Y N N Y N Y

                    Reading product information from file...

                     

                    Reading language and territory information from file...

                     

                    Reading language information from applUS.txt ...

                     

                    Connecting to APPS......Connected successfully.

                     

                    AD Worker warning:

                    Product Data File

                    /oracle/SID/fs1/EBSapps/appl/admin/zpbprod.txt

                    does not exist for product "zpb".

                    This product is registered in the database but the

                    above file does not exist in APPL_TOP.  The product

                    will be ignored without error.

                    ...

                    ...........

                     

                    AD Worker warning:

                    Product Data File

                    /oracle/SID/fs1/EBSapps/appl/admin/cusprod.txt

                    does not exist for product "cus".

                    This product is registered in the database but the

                    above file does not exist in APPL_TOP.  The product

                    will be ignored without error.

                     

                    Connecting to APPLSYS......Connected successfully.

                     

                    Currently, the following language is installed:

                     

                    Code   Language                                Status

                    ----   --------------------------------------- ---------

                    US     American English                        Base        

                     

                    Your base language will be AMERICAN.

                     

                    Connecting to SYSTEM......Connected successfully.

                    Connecting to APPLSYS......Connected successfully.

                    Connecting to APPS......Connected successfully.

                    Connecting to APPLSYS......Connected successfully.

                    Connecting to APPS......Connected successfully.

                     

                    Ready to run jobs.

                    Setting parallel context: PARALLEL AUTOPATCH AT R120

                     

                    Time when worker started job: Mon May 18 2020 22:21:08

                     

                    Start time for file is: Mon May 18 2020 22:21:08

                     

                    sqlplus -s APPS/***** @/oracle/SID/fs1/EBSapps/appl/ad/12.0.0/patch/115/sql/adsqlwrapper.sql '/oracle/SID/fs1/EBSapps/appl/ad/12.0.0/patch/115/sql/ADZDUSRC.sql &systempwd'

                    Connected.

                     

                    PL/SQL procedure successfully completed.

                     

                    Time when worker completed job: Mon May 18 2020 22:21:11

                     

                    JDBC connect string from AD_APPS_JDBC_URL is

                    (DESCRIPTION=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=AIX)(PORT=1561)))(CONNECT_DATA=(SID=SID)))

                     

                    Time when worker started job: Mon May 18 2020 22:21:28

                    Start time for file is: Mon May 18 2020 22:21:28

                    JDBC connect string from AD_APPS_JDBC_URL is

                    (DESCRIPTION=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=AIX)(PORT=1561)))(CONNECT_DATA=(SID=SID)))

                     

                    adjava -ms128m -mx256m oracle.apps.ad.worker.AdJavaWorker "(DESCRIPTION=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=AIX)(PORT=1561)))(CONNECT_DATA=(SID=SID)))" APPS ***** thin 3 /oracle/SID/fs1/EBSapps/appl/admin/adUtilMapNew.txt

                     

                                         Copyright (c) 2002, 2012 Oracle Corporation

                                            Redwood Shores, California, USA

                                                        AD Java

                                                     Version 12.2.0

                     

                    NOTE: You may not use this utility for custom development

                          unless you have written permission from Oracle Corporation.

                     

                    Reading product information from file...

                    Reading language and territory information from file...

                    Reading language information from applUS.txt ...

                      Temporarily resetting CLASSPATH to:

                    "/oracle/SID/fs1/EBSapps/appl/ad/12.0.0/java/adjava.zip:/oracle/SID/fs1/EBSapps/comn/util/jdk32/lib/dt.jar:/oracle/SID/fs1/EBSapps/comn/util/jdk32/lib/tools.jar:/oracle/SID/fs1/EBSapps/comn/util/jdk32/jre/lib/rt.jar:/oracle/SID/fs1/EBSapps/comn/shared-libs/ebs-3rdparty/WEB-INF/lib/ebs3rdpartyManifest.jar:/oracle/SID/fs1/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-fmw/WEB-INF/lib/ebsFMWManifest.jar:/oracle/SID/fs1/FMW_Home/Oracle_EBS-app1/shared-libs/ebs-appsborg/WEB-INF/lib/ebsAppsborgManifest.jar:/oracle/SID/fs1/FMW_Home/oracle_common/modules/oracle.uix_11.1.1/uix2.jar:/oracle/SID/fs1/EBSapps/comn/java/classes:/oracle/SID/fs1/EBSapps/10.1.2/forms/java:/oracle/SID/fs1/EBSapps/10.1.2/forms/java/frmall.jar:/oracle/SID/fs1/EBSapps/10.1.2/jlib/ewt3.jar:/oracle/SID/fs1/EBSapps/10.1.2/j2ee/OC4J_BI_Forms/applications/formsapp/formsweb/WEB-INF/lib/frmsrv.jar"

                     

                    Calling /oracle/SID/fs1/EBSapps/comn/util/jdk32/jre/bin/java ...

                    Exception in thread "main" java.sql.SQLRecoverableException: ORA-01034: ORACLE not available

                    ORA-27101: shared memory realm does not exist

                    IBM AIX RISC System/6000 Error: 2: No such file or directory

                    Additional information: 2820

                    Additional information: -1500894085

                     

                    at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:462)

                    at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:397)

                    at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:389)

                    at oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:451)

                    at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:481)

                    at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:205)

                    at oracle.jdbc.driver.T4CTTIoauthenticate.doOSESSKEY(T4CTTIoauthenticate.java:412)

                    at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:403)

                    at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:678)

                    at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:235)

                    at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:34)

                    at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:567)

                    at java.sql.DriverManager.getConnection(DriverManager.java:583)

                    at java.sql.DriverManager.getConnection(DriverManager.java:227)

                    at oracle.apps.ad.worker.AdJavaWorker.getAppsConnection(AdJavaWorker.java:1041)

                    at oracle.apps.ad.worker.AdJavaWorker.main(AdJavaWorker.java:276)

                     

                     

                    - The adpatch_u10117518_cup10.log

                    ************* Start of AutoPatch session *************

                    AutoPatch version: 12.2.0

                    AutoPatch started at: Mon May 18 2020 22:08:57

                     

                    APPL_TOP is set to /oracle/SID/fs1/EBSapps/appl

                    ...

                    .....

                    You are about to apply a patch to the installation of Oracle E-Business Suite

                    in your ORACLE database 'SID'

                    using ORACLE executables in '/oracle/SID/fs1/EBSapps/10.1.2'.

                     

                    Is this the correct database [Yes] ? Yes

                     

                    AutoPatch needs the password for your 'SYSTEM' ORACLE schema

                    in order to determine your installation configuration.

                     

                    Enter the password for your 'SYSTEM' ORACLE schema: *****

                    Connecting to SYSTEM......Connected successfully.

                     

                    The ORACLE username specified below for Application Object Library

                    uniquely identifies your existing product group: APPLSYS

                     

                    Enter the ORACLE password of Application Object Library [APPS] : *****

                     

                    AutoPatch is verifying your username/password.

                    Connecting to APPLSYS......Connected successfully.

                     

                    The status of various features in this run of AutoPatch is:

                                                               <-Feature version in->

                    Feature                          Active?   APPLTOP    Data model    Flags

                    ------------------------------   -------   --------   -----------   -----------

                    CHECKFILE                        Yes       1          1             Y N N Y N Y

                    PREREQ                           Yes       6          6             Y N N Y N Y

                    CONCURRENT_SESSIONS              No        2          2             Y Y N Y Y N

                    PATCH_TIMING                     Yes       2          2             Y N N Y N Y

                    PATCH_HIST_IN_DB                 Yes       6          6             Y N N Y N Y

                    SCHEMA_SWAP                      Yes       1          1             Y N N Y Y Y

                    JAVA_WORKER                      Yes       1          1             Y N N Y N Y

                    CODELEVEL                        Yes       1          1             Y N N Y N Y

                     

                    Connecting to SYSTEM......Connected successfully.

                    Connecting to APPLSYS......Connected successfully.

                    Identifier for the current session is 18125

                    Reading product information from file...

                    Reading language and territory information from file...

                    Reading language information from applUS.txt ...

                    Connecting to APPS......Connected successfully.

                     

                    *** Patching the File System (in Classic mode) ***

                    AutoPatch warning:

                    Product Data File

                    /oracle/SID/fs1/EBSapps/appl/admin/zpbprod.txt

                    does not exist for product "zpb".

                    This product is registered in the database but the

                    above file does not exist in APPL_TOP.  The product

                    will be ignored without error.

                     

                    ...

                    .....

                    Time is: Mon May 18 2020 22:21:28

                    Assigned: file ad_zd_logs.xdf on worker  7 for product ad  username APPS.

                    Time is: Mon May 18 2020 22:21:28

                    Assigned: file ad_zd_logs_s.xdf on worker  8 for product ad  username APPS.

                    Time is: Mon May 18 2020 22:21:28

                    Completed: file ADZDTABC.sql on worker  1 for product ad  username APPLSYS.

                    Time is: Mon May 18 2020 22:21:28

                    Assigned: file afsys.sql    on worker  1 for product fnd username APPLSYS.

                    Time is: Mon May 18 2020 22:21:28

                    Completed: file addpapuu.sql on worker  2 for product ad  username APPLSYS.

                    Time is: Mon May 18 2020 22:21:29

                    Completed: file afsys.sql    on worker  1 for product fnd username APPLSYS.

                    Time is: Mon May 18 2020 22:21:40

                     

                     

                    - The alert log

                    2020-05-18T22:00:00.057522+08:00

                    Setting Resource Manager plan SCHEDULER[0x4D19]:DEFAULT_MAINTENANCE_PLAN via scheduler window

                    Setting Resource Manager CDB plan DEFAULT_MAINTENANCE_PLAN via parameter

                    SID(3):Setting Resource Manager plan SCHEDULER[0xCA020]:DEFAULT_MAINTENANCE_PLAN via scheduler window

                    SID(3):Setting Resource Manager plan DEFAULT_MAINTENANCE_PLAN via parameter

                    2020-05-18T22:00:04.642068+08:00

                    Begin automatic SQL Tuning Advisor run for special tuning task  "SYS_AUTO_SQL_TUNING_TASK"

                    2020-05-18T22:00:35.656041+08:00

                    End automatic SQL Tuning Advisor run for special tuning task  "SYS_AUTO_SQL_TUNING_TASK"

                    2020-05-18T22:03:21.565981+08:00

                    Incremental checkpoint up to RBA [0x10f.23f4f.0], current log tail at RBA [0x10f.2d713.0]

                    2020-05-18T22:09:51.983775+08:00

                    Thread 1 cannot allocate new log, sequence 272

                    Private strand flush not complete

                      Current log# 1 seq# 271 mem# 0: /ofdb/ofcdb/OFCDB/onlinelog/o1_mf_1_h60sgxgt_.log

                      Current log# 1 seq# 271 mem# 1: /ofdb/fast_recovery_area/OFCDB/onlinelog/o1_mf_1_h60sh3mf_.log

                    2020-05-18T22:09:55.377276+08:00

                    Beginning log switch checkpoint up to RBA [0x110.2.10], SCN: 8291732257163

                    2020-05-18T22:09:55.401234+08:00

                    Thread 1 advanced to log sequence 272 (LGWR switch)

                      Current log# 2 seq# 272 mem# 0: /ofdb/ofcdb/OFCDB/onlinelog/o1_mf_2_h60sgxj9_.log

                      Current log# 2 seq# 272 mem# 1: /ofdb/fast_recovery_area/OFCDB/onlinelog/o1_mf_2_h60sh3cl_.log

                    2020-05-18T22:15:02.280193+08:00

                    Completed checkpoint up to RBA [0x110.2.10], SCN: 8291732257163

                    2020-05-18T22:23:37.026440+08:00

                    Incremental checkpoint up to RBA [0x110.1a4ad.0], current log tail at RBA [0x110.2bdc7.0]

                    2020-05-18T22:37:59.278620+08:00

                    Thread 1 cannot allocate new log, sequence 273

                    Private strand flush not complete

                      Current log# 2 seq# 272 mem# 0: /ofdb/ofcdb/OFCDB/onlinelog/o1_mf_2_h60sgxj9_.log

                      Current log# 2 seq# 272 mem# 1: /ofdb/fast_recovery_area/OFCDB/onlinelog/o1_mf_2_h60sh3cl_.log

                    2020-05-18T22:38:02.504498+08:00

                    Beginning log switch checkpoint up to RBA [0x111.2.10], SCN: 8291734578949

                    2020-05-18T22:38:02.528505+08:00

                    Thread 1 advanced to log sequence 273 (LGWR switch)

                      Current log# 3 seq# 273 mem# 0: /ofdb/ofcdb/OFCDB/onlinelog/o1_mf_3_h60sgxks_.log

                      Current log# 3 seq# 273 mem# 1: /ofdb/fast_recovery_area/OFCDB/onlinelog/o1_mf_3_h60sh4r2_.log

                    2020-05-18T22:40:01.096163+08:00

                    SID(3):Errors in file /oracle/diag/rdbms/ofcdb/ofcdb/trace/ofcdb_j003_15794604.trc:

                    ORA-12012: error on auto execute of job "SYS"."ORA$AT_SA_SPC_SY_7911"

                    ORA-01555: snapshot too old: rollback segment number 11 with name "_SYSSMU11_3275849776$" too small

                    ORA-06512: at "SYS.DBMS_ADVISOR", line 226

                    ORA-06512: at "SYS.PRVT_ADVISOR", line 3546

                    ORA-06512: at "SYS.PRVT_ADVISOR", line 932

                    ORA-06512: at "SYS.WRI$_ADV_OBJSPACE_TREND_T", line 1882

                    ORA-06512: at "SYS.PRVT_ADVISOR", line 915

                    ORA-06512: at "SYS.PRVT_ADVISOR", line 3451

                    ORA-06512: at "SYS.DBMS_ADVISOR", line 276

                    ORA-06512: at "SYS.DBMS_ADVISOR", line 221

                    ORA-06512: at "SYS.DBMS_SPACE", line 2484

                    ORA-06512: at "SYS.DBMS_SPACE", line 2774

                    2020-05-18T22:43:06.240215+08:00

                    Completed checkpoint up to RBA [0x111.2.10], SCN: 8291734578949

                    2020-05-18T22:43:45.998427+08:00

                    Incremental checkpoint up to RBA [0x111.ba2.0], current log tail at RBA [0x111.41f1.0]

                    2020-05-18T23:03:49.413863+08:00

                    Incremental checkpoint up to RBA [0x111.446d.0], current log tail at RBA [0x111.769d.0]

                    • 7. Re: Apply AD R12.2 and CPU Upgrade patches issues
                      yoonas

                      for uploading files you can use advanced editor then attach

                       

                       

                       

                      I was reading the through the post again to understand the scenario better and saw

                       

                      1) But I am confused with the readme of patch 10117518 (Apply the AD Upgrade Patch 10117518 by merging it with the latest Consolidated Upgrade Patch (CUP) for AD (Patch 28853096:R12.AD.C)) and the Preparation Tasks is run adgrants.sql, does it mean there is using the adgrants.sql that from merged directory, not from patch 10117518 directory?I tried to run the "@adgrants.sql apps" that from merged directory ($PATCH_TOP/10117518_CUP10/admin/adgrants.sql), but it shown errors as below (Can I ignored it?):

                      ###############

                       

                       

                       

                      But the patch readme instructs to merger 29992541 as well, but your list above does not seem to have that one included.

                      Your merged patch includes 29992541:R12.AD.C ?

                      • 8. Re: Apply AD R12.2 and CPU Upgrade patches issues
                        WF001

                        Yes, we have merged patch includes 29992541:R12.AD.C. Thanks a lot.

                        - Refer patch 10117518 readme instructs and Oracle E-Business Suite Release Notes, Release 12.2 (Doc ID 1320300.1)

                        - (Doc ID 1320300.1) 3.1 Apply Consolidated Upgrade Patch and Run 12.2.0 Upgrade (required)

                            Note: This patch is only intended for upgrade customers who are upgrading to Release 12.2.0. If you are already at Release 12.2.0 or beyond, do NOT apply this patch.

                         

                        My taken action as below:

                        su - applmgr

                        echo $ORACLE_HOME

                        /oracle/SID/fs1/EBSapps/10.1.2

                         

                        mkdir /PATCH_TOP/AD_source/

                        - Download and unzip R12.AD.C (patch#10117518), 12.2.0 AD CUP10 patch#28853096:R12.AD.C and 29992541:R12.AD.C under /PATCH_TOP/AD_source/

                         

                        cd /PATCH_TOP/AD_source/

                        unzip p10117518_R12_AIX64-5L.zip

                        unzip p28853096_R12.AD.C_R12_AIX64-5L.zip

                        unzip p29992541_R12.AD.C_R12_AIX64-5L.zip

                         

                        mkdir -p /PATCH_TOP/10117518_CUP10

                        admrgpch -s /PATCH_TOP/AD_source -d /PATCH_TOP/10117518_CUP10 -merge_name 10117518_cup10 -admode

                        Executing the merge of the patch drivers

                        -- Processing patch: /PATCH_TOP/AD_source/10117518

                        -- Processing file: /PATCH_TOP/AD_source/10117518/u10117518.drv

                        -- Done processing file: /PATCH_TOP/AD_source/10117518/u10117518.drv

                        -- Done processing patch: /PATCH_TOP/AD_source/10117518

                         

                        -- Processing patch: /PATCH_TOP/AD_source/28853096

                        -- Processing file: /PATCH_TOP/AD_source/28853096/u28853096.drv

                        -- Done processing file: /PATCH_TOP/AD_source/28853096/u28853096.drv

                        -- Done processing patch: /PATCH_TOP/AD_source/28853096

                         

                        -- Processing patch: /PATCH_TOP/AD_source/29992541

                        -- Processing file: /PATCH_TOP/AD_source/29992541/u29992541.drv

                        -- Done processing file: /PATCH_TOP/AD_source/29992541/u29992541.drv

                        -- Done processing patch: /PATCH_TOP/AD_source/29992541

                         

                        Copying files...

                        5% complete. Copied 47 files of 927...

                        10% complete. Copied 93 files of 927...

                        15% complete. Copied 140 files of 927...

                        20% complete. Copied 186 files of 927...

                        25% complete. Copied 232 files of 927...

                        30% complete. Copied 279 files of 927...

                        35% complete. Copied 325 files of 927...

                        40% complete. Copied 371 files of 927...

                        45% complete. Copied 418 files of 927...

                        50% complete. Copied 464 files of 927...

                        55% complete. Copied 510 files of 927...

                        60% complete. Copied 557 files of 927...

                        65% complete. Copied 603 files of 927...

                        70% complete. Copied 649 files of 927...

                        75% complete. Copied 696 files of 927...

                        80% complete. Copied 742 files of 927...

                        85% complete. Copied 788 files of 927...

                        90% complete. Copied 835 files of 927...

                        95% complete. Copied 881 files of 927...

                        100% complete. Copied 927 files of 927...

                         

                        Character-set converting files...

                          3 unified drivers merged.

                        Patch merge completed successfully

                        Please check the log file at ./admrgpch.log.

                         

                        <Refer readme - Preparation tasks>

                        su - orausr

                        echo $ORACLE_HOME

                        /oracle/product/19c/db_1

                         

                        i) Create $ORACLE_HOME/appsutil/admin on the database server.

                        mkdir $ORACLE_HOME/appsutil/admin

                         

                        ii) Compare the version of adgrants.sql(UNIX) in $APPL_TOP/admin to that in patch directory.

                        cat /oracle/SID/fs1/EBSapps/appl/admin/adgrants.sql |grep Header |head -1

                        REM $Header: adgrants.sql 120.67.12020000.8 2012/10/05 15:19:21 sstomar ship $

                         

                        cat /PATCH_TOP/AD_source/10117518/admin/adgrants.sql |grep Header |head -1

                        REM $Header: adgrants.sql 120.67.12020000.8 2012/10/05 15:19:21 sstomar ship $

                         

                        cat /PATCH_TOP/10117518_CUP10/admin/adgrants.sql |grep Header |head -1

                        REM $Header: adgrants.sql 120.67.12020000.64 2019/02/07 07:28:39 rsatyava ship $

                         

                        iii) Copy the adgrants.sql (UNIX) from merged directory to $ORACLE_HOME/appsutil/admin.

                        cp /PATCH_TOP/10117518_CUP10/admin/adgrants.sql $ORACLE_HOME/appsutil/admin/.

                         

                        iv) set the environment to point to ORACLE_HOME on the database server

                        Note: If the Database is CDB-PDB enabled,source using the following steps

                        a) source $ORACLE_HOME/<cdb-name>_<host-name>.env (already added 19c $ORACLE_HOME/SID_AIX.env in /home/orausr/.profile)

                        . $ORACLE_HOME/ofcdb_AIX.env

                         

                        b) export ORACLE_PDB_SID=<pdb-name>

                        export ORACLE_PDB_SID=SID

                        echo $ORACLE_PDB_SID

                        SID

                         

                        v) Run the script: (by orausr)

                        sqlplus / as sysdba

                        @$ORACLE_HOME/appsutil/admin/adgrants.sql apps

                        • 9. Re: Apply AD R12.2 and CPU Upgrade patches issues
                          yoonas

                          hi,

                           

                          . $ORACLE_HOME/ofcdb_AIX.env

                           

                          b) export ORACLE_PDB_SID=<pdb-name>

                          export ORACLE_PDB_SID=SID

                           

                          v) Run the script: (by orausr)

                          sqlplus / as sysdba

                          Please source as above , connect and then  post output of below query and listener status

                           

                          show con_name
                          

                           

                          select name,open_mode from v$containers;
                          

                           

                          select name from cdb_services;
                          

                           

                          on the database host

                           

                          lsnrctl services;
                          lsnrctl status;
                          
                          • 10. Re: Apply AD R12.2 and CPU Upgrade patches issues
                            WF001

                            Thanks yoonas

                             

                            sqlplus / as sysdba

                            SQL> show con_name

                            CON_NAME

                            ---------

                            CDB$ROOT

                             

                            SQL> select name,open_mode from v$containers;

                            NAME                           OPEN_MODE

                            ------------------------------ ----------

                            CDB$ROOT                       READ WRITE

                            PDB$SEED                       READ ONLY

                            SID                                    READ WRITE

                             

                            SQL> select name from cdb_services;

                            NAME

                            ------------------------------

                            SYS$BACKGROUND

                            SYS$USERS

                            ofcdb

                            EBS_PATCH

                            ofcdbXDB

                            ofcdb.domain.com

                            SID

                            ebs_SID

                             

                            lsnrctl services;

                            LSNRCTL for IBM/AIX RISC System/6000: Version 19.0.0.0.0 - Production on 27-MAY-2020 11:12:08

                            Copyright (c) 1991, 2019, Oracle.  All rights reserved.

                            Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))

                            TNS-12541: TNS:no listener

                            TNS-12560: TNS:protocol adapter error

                            TNS-00511: No listener

                            IBM/AIX RISC System/6000 Error: 79: Connection refused

                             

                            lsnrctl status;

                            LSNRCTL for IBM/AIX RISC System/6000: Version 19.0.0.0.0 - Production on 27-MAY-2020 11:12:25

                            Copyright (c) 1991, 2019, Oracle.  All rights reserved.

                            Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))

                            TNS-12541: TNS:no listener

                            TNS-12560: TNS:protocol adapter error

                              TNS-00511: No listener

                               IBM/AIX RISC System/6000 Error: 79: Connection refused

                             

                            lsnrctl status ofcdb;

                            LSNRCTL for IBM/AIX RISC System/6000: Version 19.0.0.0.0 - Production on 27-MAY-2020 11:14:47

                            Copyright (c) 1991, 2019, Oracle.  All rights reserved.

                            Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=AIX.domain.com)(PORT=1561)))

                            STATUS of the LISTENER

                            ------------------------

                            Alias                     ofcdb

                            Version                   TNSLSNR for IBM/AIX RISC System/6000: Version 19.0.0.0.0 - Production

                            Start Date                28-APR-2020 21:50:43

                            Uptime                    28 days 13 hr. 24 min. 4 sec

                            Trace Level               off

                            Security                  ON: Local OS Authentication

                            SNMP                      OFF

                            Listener Parameter File   /oracle/product/19c/db_1/network/admin/SID_AIX/listener.ora

                            Listener Log File         /oracle/product/19c/db_1/admin/SID_AIX/diag/tnslsnr/AIX/ofcdb/alert/log.xml

                            Listening Endpoints Summary...

                              (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=AIX.domain.com)(PORT=1561)))

                            Services Summary...

                            Service "8939c30fdc8c01b2e0530af11a191106" has 1 instance(s).

                              Instance "ofcdb", status READY, has 1 handler(s) for this service...

                            Service "a2f076d6089c01dce053c0a8080a56c1" has 1 instance(s).

                              Instance "ofcdb", status READY, has 1 handler(s) for this service...

                            Service "SID" has 2 instance(s).

                              Instance "SID", status UNKNOWN, has 1 handler(s) for this service...

                              Instance "ofcdb", status READY, has 1 handler(s) for this service...

                            Service "ebs_SID" has 1 instance(s).

                              Instance "ofcdb", status READY, has 1 handler(s) for this service...

                            Service "ebs_patch" has 1 instance(s).

                              Instance "ofcdb", status READY, has 1 handler(s) for this service...

                            Service "ofcdb" has 1 instance(s).

                              Instance "ofcdb", status READY, has 1 handler(s) for this service...

                            Service "ofcdbXDB" has 1 instance(s).

                              Instance "ofcdb", status READY, has 1 handler(s) for this service...

                            The command completed successfully

                            • 11. Re: Apply AD R12.2 and CPU Upgrade patches issues
                              yoonas

                              I am unable to see anything helpful so far

                              What's patch status now (adctrl)

                              can you please collect all the log files (adpatach,workers,alertlog,etc) and share

                              • 12. Re: Apply AD R12.2 and CPU Upgrade patches issues
                                WF001

                                - The patch status now (adctrl) (No any change)

                                        Control

                                Worker  Code      Context            Filename                    Status

                                ------  --------  -----------------  --------------------------  --------------

                                    1  Run      AutoPatch R120                                Wait

                                    2  Run      AutoPatch R120                                Wait

                                    3  Run      AutoPatch R120    ad_objects.xdf              Running

                                    4  Run      AutoPatch R120    ad_patched_tables.xdf      Running

                                    5  Run      AutoPatch R120    ad_zd_ddl_handler.xdf      Running

                                    6  Run      AutoPatch R120    ad_zd_ddl_handler_ddl_s.xd  Running

                                    7  Run      AutoPatch R120    ad_zd_logs.xdf              Running

                                    8  Run      AutoPatch R120    ad_zd_logs_s.xdf            Running

                                 

                                - Attached file about patch logs, adwork files, alert file...: files.zip

                                 

                                Many thanks for your help again~

                                • 13. Re: Apply AD R12.2 and CPU Upgrade patches issues
                                  yoonas

                                    Calling /oracle/SID/fs1/EBSapps/comn/util/jdk32/jre/bin/java ...

                                  Exception in thread "main" java.sql.SQLRecoverableException: ORA-01034: ORACLE not available

                                  ORA-27101: shared memory realm does not exist

                                  IBM AIX RISC System/6000 Error: 2: No such file or directory

                                  Additional information: 2820

                                  Additional information: -1500894085

                                   

                                   

                                  all the workerlog files from 3-8 are having same error.

                                   

                                  Can you please try connecting to database from application tier with following connect string and post status

                                   

                                  sqlplus apps/urpwd@'(DESCRIPTION=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=AIX))(PORT=1561)))(CONNECT_DATA=(SID=SID)))'
                                  

                                   

                                   

                                  sqlplus apps/urpwd@'(DESCRIPTION=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=AIX)(PORT=1561)))(CONNECT_DATA=(SERVICE_NAME=SID)))'
                                  
                                  • 14. Re: Apply AD R12.2 and CPU Upgrade patches issues
                                    WF001

                                    sqlplus apps/urpwd@'(DESCRIPTION=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=AIX))(PORT=1561)))(CONNECT_DATA=(SID=SID)))' 

                                    SQL*Plus: Release 10.1.0.5.0 - Production on Thu May 28 02:04:28 2020

                                    Copyright (c) 1982, 2005, Oracle.  All rights reserved.

                                    Usage: SQLPLUS [ [<option>] [<logon>] [<start>] ]

                                    where <option> ::= -H | -V | [ [-C <v>] [-L] [-M <o>] [-R <n>] [-S] ]

                                          <logon>  ::= <username>[/<password>][@<connect_identifier>] | / | /NOLOG

                                          <start>  ::= @<URL>|<filename>[.<ext>] [<parameter> ...]

                                            "-H" displays the SQL*Plus version banner and usage syntax

                                            "-V" displays the SQL*Plus version banner

                                            "-C" sets SQL*Plus compatibility version <v>

                                            "-L" attempts log on just once

                                            "-M <o>" uses HTML markup options <o>

                                            "-R <n>" uses restricted mode <n>

                                            "-S" uses silent mode

                                     

                                    sqlplus apps/urpwd@'(DESCRIPTION=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=AIX)(PORT=1561)))(CONNECT_DATA=(SERVICE_NAME=SID)))' 

                                    SQL*Plus: Release 10.1.0.5.0 - Production on Thu May 28 02:05:01 2020

                                    Copyright (c) 1982, 2005, Oracle.  All rights reserved.

                                    Connected to:

                                    Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

                                    SQL>

                                     

                                    Thank you very much indeed!

                                    1 2 3 Previous Next