5 Replies Latest reply on Aug 7, 2018 1:53 AM by Beauty_and_dBest

    Adop 12.2.7

    Beauty_and_dBest

      EBS 12.2.7

      12c

      OL6

       

       

      Hi ALL,

       

      I am apply patch using adop.

       

      I am now in phase>>  adop phase=cutover    but the ICM is not going down

      It has been running for 3 hrs, the patch is only 0.3Mb

       

      $ adop phase=cutover

       

      Enter the APPS password:

      Enter the SYSTEM password:

      Enter the WLSADMIN password:

       

      Validating credentials.

       

      Initializing.

          Run Edition context  : /u03/appv1227/V1227/fs2/inst/apps/V1227_trnsrvr/appl/admin/V1227_trnsrvr.xml

          Patch edition context: /u03/appv1227/V1227/fs1/inst/apps/V1227_trnsrvr/appl/admin/V1227_trnsrvr.xml

          Patch file system free space: 598.58 GB

       

      Validating system setup.

          Node registry is valid.

          [WARNING]   ETCC: The following required database fixes have not been applied to node trnsrvr:

                        24589081

                        25099339

                        25919622

                      Refer to My Oracle Support Knowledge Document 1594274.1 for instructions.

          [WARNING]   MTCC has not been run. Apply Patch 17537119 and run checkMTpatch.sh.

       

      Checking for existing adop sessions.

          Continuing with existing session [Session ID: 63].

       

      ===========================================================================

      ADOP (C.Delta.10)

      Session ID: 63

      Node: trnsrvr

      Phase: cutover

      Log: /u03/appv1227/V1227/fs_ne/EBSapps/log/adop/63/20180806_012548/adop.log

      ===========================================================================

       

      Validating configuration on node: [trnsrvr].

          Log: /u03/appv1227/V1227/fs_ne/EBSapps/log/adop/63/20180806_012548/cutover/validate/trnsrvr

       

      Checking if finalize required.

          Finalize phase is not required

       

      Submitting request for Internal Concurrent Manager shutdown.

       

      You are running adcmctl.sh version 120.19.12020000.7

       

      Shutting down concurrent managers for V1227 ...

      ORACLE Password:

      Submitted request 7667150 for CONCURRENT FND SHUTDOWN

       

      adcmctl.sh: exiting with status 0

       

      adcmctl.sh: check the logfile /u03/appv1227/V1227/fs2/inst/apps/V1227_trnsrvr/logs/appl/admin/log/adcmctl.txt for more information ... 

          Cancelling ADZDPATCH concurrent request

       

      Waiting for Internal Concurrent Manager to go down.

       

       

       

      How do I forced it to shutdown?

       

      Can I run adstpall.sh?

       

       

      Please help.....

       

       

      Kind regards,

      jc

        • 1. Re: Adop 12.2.7
          Mohammed Ali A. Syed

          Hi,

           

          Just kill all the FND processes.

           

          $ps -ef|grep FND

           

          Note the process id, which is second column

           

          $kill -9 <process ids> you found.

           

          Thanks

          • 2. Re: Adop 12.2.7
            Beauty_and_dBest

            Thanks EBSDBA,

             

            I got lots of grap FND output , it also list all FNDLIBR and FNDSM, etc.

             

            Is it not possible to just run adstpall.sh?

             

             

            Please help....

            • 3. Re: Adop 12.2.7
              Magnanimous

              There are certain ways -

              - If its not production instance you can kill the session using - ps -ef | grep FND

              - Yes you can kill those all with FNDLIB

              - While doing the cutover you can also mentioned cm_wait=3 (3 Minute)

              - Check the alert whats the process is running and its not needed kill the same.

              - You cant run adstpall.sh - As ADOP needs to have weblogic up and running.

               

              Prashant

              • 4. Re: Adop 12.2.7
                Beauty_and_dBest

                Thanks Prasant and EBS,

                 

                I tried to kill but the adop session got error

                 

                 

                [appv1227@trnsrvr ~]$ adop phase=cutover

                 

                Enter the APPS password:

                Enter the SYSTEM password:

                Enter the WLSADMIN password:

                 

                Validating credentials.

                 

                Initializing.

                    Run Edition context  : /u03/appv1227/V1227/fs2/inst/apps/V1227_trnsrvr/appl/admin/V1227_trnsrvr.xml

                    Patch edition context: /u03/appv1227/V1227/fs1/inst/apps/V1227_trnsrvr/appl/admin/V1227_trnsrvr.xml

                    Patch file system free space: 598.58 GB

                 

                Validating system setup.

                    Node registry is valid.

                    [WARNING]   ETCC: The following required database fixes have not been applied to node trnsrvr:

                                  24589081

                                  25099339

                                  25919622

                                Refer to My Oracle Support Knowledge Document 1594274.1 for instructions.

                    [WARNING]   MTCC has not been run. Apply Patch 17537119 and run checkMTpatch.sh.

                 

                Checking for existing adop sessions.

                    Continuing with existing session [Session ID: 63].

                 

                ===========================================================================

                ADOP (C.Delta.10)

                Session ID: 63

                Node: trnsrvr

                Phase: cutover

                Log: /u03/appv1227/V1227/fs_ne/EBSapps/log/adop/63/20180806_012548/adop.log

                ===========================================================================

                 

                Validating configuration on node: [trnsrvr].

                    Log: /u03/appv1227/V1227/fs_ne/EBSapps/log/adop/63/20180806_012548/cutover/validate/trnsrvr

                Checking if finalize required.

                    Finalize phase is not required

                Submitting request for Internal Concurrent Manager shutdown.

                You are running adcmctl.sh version 120.19.12020000.7

                Shutting down concurrent managers for V1227 ...

                ORACLE Password:

                Submitted request 7667150 for CONCURRENT FND SHUTDOWN

                adcmctl.sh: exiting with status 0

                adcmctl.sh: check the logfile /u03/appv1227/V1227/fs2/inst/apps/V1227_trnsrvr/logs/appl/admin/log/adcmctl.txt for more information ... 

                    Cancelling ADZDPATCH concurrent request

                 

                ========================================

                Waiting for Internal Concurrent Manager to go down.

                ========================================

                 

                Stopping patch edition Admin Server.

                You are running adadminsrvctl.sh version 120.10.12020000.10

                The AdminServer is already shutdown

                AdminServer logs are located at /u03/appv1227/V1227/fs1/FMW_Home/user_projects/domains/EBS_domain_V1227/servers/AdminServer/logs

                adadminsrvctl.sh: exiting with status 2

                adadminsrvctl.sh: check the logfile /u03/appv1227/V1227/fs1/inst/apps/V1227_trnsrvr/logs/appl/admin/log/adadminsrvctl.txt for more information ... 

                Stopping patch edition Node Manager.

                You are running adnodemgrctl.sh version 120.11.12020000.12

                The Node Manager is already shutdown

                NodeManager log is located at /u03/appv1227/V1227/fs1/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1

                 

                adnodemgrctl.sh: exiting with status 2

                 

                adnodemgrctl.sh: check the logfile /u03/appv1227/V1227/fs1/inst/apps/V1227_trnsrvr/logs/appl/admin/log/adnodemgrctl.txt for more information ... 

                Stopping the application tier services on current node.

                    Log: /u03/appv1227/V1227/fs_ne/EBSapps/log/adop/63/20180806_012548/cutover/trnsrvr

                Stopping run edition Admin Server.

                You are running adadminsrvctl.sh version 120.10.12020000.10

                Stopping WLS Admin Server...

                Refer /u03/appv1227/V1227/fs2/inst/apps/V1227_trnsrvr/logs/appl/admin/log/adadminsrvctl.txt for detail

                AdminServer logs are located at /u03/appv1227/V1227/fs2/FMW_Home/user_projects/domains/EBS_domain_V1227/servers/AdminServer/logs

                adadminsrvctl.sh: exiting with status 0

                adadminsrvctl.sh: check the logfile /u03/appv1227/V1227/fs2/inst/apps/V1227_trnsrvr/logs/appl/admin/log/adadminsrvctl.txt for more information ... 

                 

                Stopping run edition Node Manager.

                You are running adnodemgrctl.sh version 120.11.12020000.12

                NodeManager log is located at /u03/appv1227/V1227/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1

                adnodemgrctl.sh: exiting with status 0

                adnodemgrctl.sh: check the logfile /u03/appv1227/V1227/fs2/inst/apps/V1227_trnsrvr/logs/appl/admin/log/adnodemgrctl.txt for more information ... 

                 

                Running stored cutover DDLs.

                    Log: /u03/appv1227/V1227/fs_ne/EBSapps/log/adop/63/20180806_012548/cutover/trnsrvr/log/cutover.log

                Loading JAR files into database.

                    No JAR files found to load

                Performing database cutover.

                    Log: @ADZDSHOWLOG.sql "2018/08/06 11:26:03"

                Reference of statement-handler is not-valid.

                DB-Handle Error Code: 28

                DB-Handle Error Message: ORA-00028: your session has been killed (DBD ERROR: OCIStmtExecute/Describe)

                Reference of statement-handler is not valid, attempting database reconnect

                getConnNExec: Getting db-handler

                getConnNExec: reattempting sql statement

                getConnNExec: Returning 0E0

                Statement Handler Error Code: 20001

                Statement Handler Error Message: ORA-20001: Error while calling ad_zd_log.message(ad.bin.adop,WARNING,).ORA-01400: cannot insert NULL into ("APPLSYS"."AD_ZD_LOGS"."MESSAGE_TEXT")

                ORA-06512: at line 6 (DBD ERROR: OCIStmtExecute)

                Reference of statement-handler is valid, not attempting database reconnect.

                Failed to execute SQL statement:

                                 declare

                                 begin

                                    ad_zd_log.message('ad.bin.adop','WARNING','','trnsrvr');

                                 exception

                                    when others then

                                       raise_application_error(-20001,'Error while calling ad_zd_log.message(ad.bin.adop,WARNING,).' || sqlerrm);

                                 end;

                 

                Error Message:

                ORA-20001: Error while calling ad_zd_log.message(ad.bin.adop,WARNING,).ORA-01400: cannot insert NULL into ("APPLSYS"."AD_ZD_LOGS"."MESSAGE_TEXT")

                ORA-06512: at line 6 (DBD ERROR: OCIStmtExecute)

                Error calling ad_zd_log.message to log messages to the database.

                Please check if database is up

                 

                [STATEMENT] Please run adopscanlog utility, using the command

                 

                "adopscanlog -latest=yes"

                 

                to get the list of the log files along with snippet of the error message corresponding to each log file.

                 

                 

                adop exiting with status = 1 (Fail)

                [appv1227@trnsrvr ~]$

                 

                ==============================================================

                ADOP (C.Delta.10)

                Session Id: 63

                Command: status

                Output: /u03/appv1227/V1227/fs_ne/EBSapps/log/adop/63/20180806_121809/adzdshowstatus.out

                ===============================================================

                Node Name       Node Type  Phase           Status          Started              Finished             Elapsed

                --------------- ---------- --------------- --------------- -------------------- -------------------- ------------

                trnsrvr         master     PREPARE         COMPLETED       2018/08/06 01:06:46  2018/08/06 01:11:56  0:05:10

                                           APPLY           COMPLETED       2018/08/06 01:14:07  2018/08/06 01:17:16  0:03:09

                                           FINALIZE        COMPLETED       2018/08/06 01:21:42  2018/08/06 01:22:40  0:00:58

                                           CUTOVER         RUNNING         2018/08/06 01:26:41                       10:51:43

                                           CLEANUP         NOT STARTED

                 

                File System Synchronization Type: Full

                 

                 

                 

                Please help how to restart...

                 

                 

                Kind regards,

                 

                 

                 

                • 5. Re: Adop 12.2.7
                  Beauty_and_dBest

                  I just run again adop phase=cutover , and it went good