How to fix failed adop session

Asmirnov-Oracle

    Hi,

     

    I did run adop phase=cutover on a patch, but didn’t wait long enough, stopped it, did run phase=abort, but then I killed the adop process and now I have adop as if it is running in ABORT phase, so I can't run another adop session.

     

    Now, trying to run adop on another patch, I am getting:

     

    [oracle@ebs 23510855]$ adop phase=prepare patches=23510855

    Enter the APPS password:

    Enter the SYSTEM password:

    Enter the WLSADMIN password:

    Please wait. Validating credentials...

    RUN file system context file: /u01/install/VISION/fs2/inst/apps/EBSDB_ebs/appl/admin/EBSDB_ebs.xml

    PATCH file system context file: /u01/install/VISION/fs1/inst/apps/EBSDB_ebs/appl/admin/EBSDB_ebs.xml

    Execute SYSTEM command : df /u01/install/VISION/fs1

    Worker count determination...

    Validation successful. All expected nodes are listed in ADOP_VALID_NODES table.

    [EVENT] [START 2017/05/28 20:49:02] Performing database sanity checks

    [EVENT] [END   2017/05/28 20:49:02] Finished performing database sanity checks

      There is already a session which is incomplete. Details are:

    Session Id :   13

    Prepare phase status  :   COMPLETED

    Apply phase status    :   COMPLETED

    Cutover  phase status :   NOT COMPLETED

    Abort phase status    :   RUNNING

    Session status        :   RUNNING

    [ERROR]     Unable to continue because ADOP tool is already in use

    [STATEMENT] Please run adopscanlog utility, using the command

    "adopscanlog -latest=yes"

    to get the list of the log files along with snippet of the error message corresponding to each log file.

    adop exiting with status = 1 (Fail)

    [oracle@ebs 23510855]$ adopscanlog -latest=yes

    Scanning /u01/install/VISION/fs_ne/EBSapps/log/adop/13/ directory ...

     

    Is there a way I can fix it?

     

    Thank you!

      • 1. Re: How to fix failed adop session
        #MVivek

        Please share the output of the adop status -

         

        $ adop -status

         

         

        Thanks

        Vivek

        • 2. Re: How to fix failed adop session
          #MVivek

          First recommendation is to abort the adop session which is stuck trying to ssh to a dead node.

           

          adop --help

          Applications DBA Online Patching Tool (adop)

           

          Usage: adop [phase=<phase,phase,...>] [patches=<patch#,patch#,...>]

                      [<parameter>=<value> ...] [input_file=<filename>]

                 adop -help

                 adop -examples

                 adop -status [<session_id>]

                 adop -validate

           

          See Oracle E-Business Suite Maintenance Guide for a full description of

          adop features, operation, and usage.

           

          The phase parameter specifies the parts (phases) of the online

          patching cycle to be executed. The five standard phases are executed

          in the order shown below.

           

          Standard phases:

            prepare       - Prepare the instance for patch application.

            apply         - Apply patches (to the patch edition).

            finalize      - Ready the instance for cutover.

            cutover       - Make the patch edition the new run edition.

            cleanup       - Drop obsolete objects and data from old editions.

           

          There are also three special phases, for use when needed.

           

          Special phases:

            abort         - Abort the current online patching cycle.

            actualize_all - Create actual copies of all code objects in the patch

                            edition.

            fs_clone      - Copy the run file system to the patch file system.

           

          So run adop phase=abort, cleanup cleanup_mode=full

           

          https://docs.oracle.com/cd/E26401_01/doc.122/e22954/T202991T531062.htm

           

          Aborting an Online Patching Cycle

          If a patching cycle is failing and the issue cannot be resolved quickly, it is possible to abort the patching cycle and return to normal runtime operation. The patch edition will be dropped.

          You can abandon a patching cycle (without applying any patches) by running the command:

          $ adop phase=abort

          Tip: This abort command can only be used before successful completion of the cutover phase. After cutover, the system is running on the new edition, and abort is no longer possible for that patching cycle.

          Aborting a patching cycle will drop the patch edition, but you must then run the cleanup and fs_clone phases before starting a new patching cycle. The cleanup must be a full cleanup.

          For example:

          $ adop phase=prepare $ adop phase=apply patches=123456 [Patch application encounters problems and you want to abort

           

           

          Optionally, you can combine the abort and cleanup commands as follows:

          $ $ adop phase=abort,cleanup cleanup_mode=full

          Note: You cannot abort application of a patch applied in hotpatch mode or downtime mode.

           

          Then try to run adop again.

          • 3. Re: How to fix failed adop session
            Asmirnov-Oracle

            Vivek, please see below, thanks for quick response!

            Current Patching Session ID: 13

             

            Node Name       Node Type       Phase           Status          Started                        Finished                       Elapsed

            --------------- --------------- --------------- --------------- ------------------------------ ------------------------------ ------------

            ebs             master          PREPARE         SESSION ABORTED 25-MAY-17 22:26:11 -04:00      25-MAY-17 22:28:57 -04:00      0:02:46

                                            APPLY           SESSION ABORTED 25-MAY-17 22:35:33 -04:00      28-MAY-17 18:09:06 -04:00      67:33:33

                                            FINALIZE        SESSION ABORTED 28-MAY-17 20:04:44 -04:00

                                            CUTOVER         SESSION ABORTED 28-MAY-17 20:04:37 -04:00

                                            CLEANUP         NOT STARTED

            File System Synchronization Used in this Patching Cycle: None

            INFORMATION: Patching cycle aborted, so fs_clone will run automatically on ebs node in prepare phase of next patching cycle.

            For more information, run ADOP Status Report by using -detail option

            Generating ADOP Status Report at location: /u01/install/VISION/fs_ne/EBSapps/log/status_20170528_221728/adzdshowstatus.out

            Please wait...

            Done...!

             

            adop exiting with status = 0 (Success)

            • 4. Re: How to fix failed adop session
              Asmirnov-Oracle

              log file:

               

               

              Current Patching Session ID: 13

               

              Node Name       Node Type       Phase           Status          Started                        Finished                       Elapsed

              --------------- --------------- --------------- --------------- ------------------------------ ------------------------------ ------------

              ebs             master          PREPARE         SESSION ABORTED 25-MAY-17 10:26:11 -04:00      25-MAY-17 10:28:57 -04:00      0:02:46

                                              APPLY           SESSION ABORTED 25-MAY-17 10:35:33 -04:00      28-MAY-17 06:09:06 -04:00      67:33:33

                                              FINALIZE        SESSION ABORTED 28-MAY-17 08:04:44 -04:00

                                              CUTOVER         SESSION ABORTED 28-MAY-17 08:04:37 -04:00

                                              CLEANUP         NOT STARTED

               

               

               

               

              File System Synchronization Used in this Patching Cycle: None

               

               

               

               

               

               

               

               

              File System Information:

              ========================

               

              File System: SINGLE NODE INSTANCE

               

              Node Name   Node Type   Current Base                   Other Base                     Non Editioned Base             Inst Top

              ----------- ----------- ------------------------------ ------------------------------ ------------------------------ ------------------------------

              ebs         master      /u01/install/VISION/fs2        /u01/install/VISION/fs1        /u01/install/VISION/fs_ne      /u01/install/VISION/fs2/inst/a

              pps/EBSDB_ebs

               

               

              Custom File System Synchronization script: /u01/install/VISION/fs_ne/EBSapps/appl/ad/custom/adop_sync.drv

               

              Patch Stage Top: /u01/install/VISION/fs_ne/EBSapps/patch

               

              Online Patching Log base: /u01/install/VISION/fs_ne/EBSapps/log/adop/13

               

              =========================================================================

              =             Patches Applied in the Current Patching Cycle

              =========================================================================

               

              Node Name    Node Type Patch         Status         Run ID File System Patch Base    File System Applied Base  Adpatch Parameters   Session Type

              ------------ --------- ------------- ----------- --------- ------------------------- ------------------------- -------------------- ---------------

              ebs          master    25178222      R                  -1 ROLLED_BACK               /u01/install/VISION/fs1                        ONLINE

               

               

              =========================================================================

              =             ADADMIN actions in the Current Patching Cycle

              =========================================================================

               

               

              ====================================================================

              = There is no Online Patching Cycle in progress at this time.

              = Report sections dependent upon Patch Edition will have no output.

              ====================================================================

               

              =========================================================================

              =                 Summary of the Last 10 ADOP Cycles

              =========================================================================

               

              Session ID Node Name  Node Type Started    Elapsed(Prepare) Elapsed(Cutover) Elapsed(Cleanup) Elapsed(Apply) Status       Patches Applied      Session Type

              ---------- ---------- --------- ---------- ---------------- ---------------- ---------------- --------------- ------------ -------------------- ---------------

                      13 ebs        master    25-MAY-17 0:02:46          0:00:00          0:00:00          0:00:00         COMPLETE     25178222             ONLINE

                                    master 0:00:00         COMPLETE     CLONE                CLONE

                                    master    28-MAY-17 0:00:00          0:00:00          0:00:00          0:00:00         NOT-STARTED  CLONE                CLONE

               

                      12 ebs        master    23-MAY-17 0:03:10          0:17:40          0:00:24          0:00:00         COMPLETE     --                   --

               

                      11 ebs        master    09-OCT-14 0:22:28          1:05:42          0:28:13          0:00:00         COMPLETE     --                   --

               

                      10 ebs        master    09-OCT-14 0:53:56          1:24:05          0:01:05          0:06:01         COMPLETE     19445058             ONLINE

                                    master 0:09:23         COMPLETE     19581770             ONLINE

                                    master 0:46:56         COMPLETE     CONFIG_CLONE         CONFIG_CLONE

                                    master               0:00:00          0:00:00          0:00:00          0:00:00         COMPLETE     CONFIG_CLONE         CONFIG_CLONE

               

                       9 slc00tdd   master 07-JUL-14  0:07:23          0:20:31          0:01:05          0:00:00         COMPLETE     --                   --

               

                       8 slc00tdd   master 07-JUL-14  0:10:06          0:14:34          0:01:01          0:00:05         COMPLETE     18886213             ONLINE

                                    master 0:04:36         COMPLETE     CONFIG_CLONE         CONFIG_CLONE

                                    slave                0:00:00          0:00:00          0:00:00          0:00:00         COMPLETE     CONFIG_CLONE         CONFIG_CLONE

               

                       7 slc00tdd   master 30-JUN-14  0:06:57          16:28:44         0:03:38          0:00:00         COMPLETE     -- --

               

                       6 slc00tdd   master 30-JUN-14  0:00:00          0:00:00          0:00:36          0:00:07         COMPLETE     18824534             HOTPATCH

               

                       5 slc00tdd   master 30-JUN-14  0:30:06          0:22:00          0:04:51          0:23:20         COMPLETE     CONFIG_CLONE         CONFIG_CLONE

                                    master 0:46:01         COMPLETE     6390830              ONLINE

                                    slave                0:00:00          0:00:00          0:00:00          0:00:00         COMPLETE     CONFIG_CLONE         CONFIG_CLONE

              • 5. Re: How to fix failed adop session
                Asmirnov-Oracle

                So I did execute

                 

                adop phase=abort,cleanup cleanup_mode=full

                 

                now I have this, can I try phase=prepare and then phase=apply now?

                 

                Thank you

                 

                 

                Node Name  

                Node Type   Phase       Status      Started                    Finished                 Elapsed

                --------------- --------------- --------------- --------------- ------------------------------ ------------------------------ ------------

                ebs         master      PREPARE     SESSION ABORTED 28-MAY-17 23:10:50 -04:00  29-MAY-17 00:00:28 -04:00  0:49:38
                                            APPLY       SESSION ABORTED
                                            FINALIZE    SESSION ABORTED
                                            CUTOVER     SESSION ABORTED
                                            CLEANUP     COMPLETED   29-MAY-17 00:09:03 -04:00  29-MAY-17 00:09:35 -04:00  0:00:32
                • 6. Re: How to fix failed adop session
                  #MVivek

                  Yes.. please proceed.

                   

                  If any error keep posted.

                   

                  Thanks

                  Vivek