3 Replies Latest reply on Jun 30, 2016 6:33 AM by Poorvika

    adop fs_clone fails while creating FMW_HOME

    Poorvika

      Hello All,

       

      I'm facing below issue while running fs_clone.

      Kindly help me out in fixing this.

       

       

      ================================

       

      START: Creating FMW Home.

      Running /bsib/app02/fs1/EBSapps/comn/adopclone_bsi003/FMW/pasteBinary.sh -javaHome /bsib/app02/fs2/EBSapps/comn/util/jdk -al /bsib/a

      pp02/fs1/EBSapps/comn/adopclone_bsi003/FMW/FMW_Home.jar -tl /bsib/app02/fs2/FMW_Home -invPtrLoc /var/opt/oracle/oraInst.loc -ldl /bs

      ib/app02/fs1/inst/apps/BSIB_bsi003/admin/log/clone/fmwT2PApply -silent true -debug true -executeSysPrereqs false

      Script Executed in 934705 milliseconds, returning status 255

      ERROR: Script failed, exit code 255

      ===============================

       

       

      <Jun 27, 2016 4:58:56 AM EDT> <Error> <Deployer> <BEA-149205> <Failed to initialize the application 'oacore [Version=1.0.0]' due to error java.lang.NullPointerException.

      java.lang.NullPointerException

              at java.io.File.<init>(File.java:277)

              at oracle.apps.ad.util.AppStartupPropertyListener.preStart(AppStartupPropertyListener.java:89)

              at weblogic.application.internal.flow.BaseLifecycleFlow$PreStartAction.run(BaseLifecycleFlow.java:290)

              at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)

              at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)

              Truncated. see log file for complete stacktrace

      Caused By: java.lang.NullPointerException

              at java.io.File.<init>(File.java:277)

              at oracle.apps.ad.util.AppStartupPropertyListener.preStart(AppStartupPropertyListener.java:89)

              at weblogic.application.internal.flow.BaseLifecycleFlow$PreStartAction.run(BaseLifecycleFlow.java:290)

              at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)

              at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)

              Truncated. see log file for complete stacktrace

      >

       

      Thanks,

      Nishita

        • 1. Re: adop fs_clone fails while creating FMW_HOME
          Pravin Takpire

          can you please check entries of /etc/hosts and check if these are correct and have all entries (including DB host and app hosts)

          regards

          Pravin

          • 2. Re: adop fs_clone fails while creating FMW_HOME
            Poorvika

            Pravin,

             

            /erc/hosts looks good.

            • 3. Re: adop fs_clone fails while creating FMW_HOME
              Poorvika

              Hello All,

               

              Issue is fixed after correcting inventory. Here is the list of all the issues and their fixes

              Issues and fixes

               

              1) [ERROR]: Domain might be locked by some other WLS user process

              Released the configuration from Web logic console

               

               

              2)

              UNEXPECTED]Error occurred while CLONE Patch File System from Run File System using command: "perl /**/app02/fs1/EBSapps/appl/

              ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl -contextfile=/bsib/app02/fs1/inst/apps/**/appl/admin/**.

              xml -patchcontextfile=/**/app02/fs2/inst/apps/**/appl/admin/**.xml -promptmsg=hide -console=off -mode=create -se

              ssionid=17 -timestamp=20160626_052637 -outdir=/**/app02/fs_ne/EBSapps/log/adop/17/fs_clone_20160626_052637/**".

                  Releasing the managed servers ports...

                  Log: /**/app02/fs_ne/EBSapps/log/adop/17/fs_clone_20160626_052637/**/txkCloneAcquirePort.log

                  [UNEXPECTED]fs_clone phase completed with errors/warnings. Please check logfiles

               

               

              START: Creating FMW Home.

              Running /abc1/app02/fs1/EBSapps/comn/adopclone_**/FMW/pasteBinary.sh -javaHome /abc1/app02/fs2/EBSapps/comn/util/jdk -al /abc1/a

              pp02/fs1/EBSapps/comn/adopclone_abc1/FMW/FMW_Home.jar -tl /abc1/app02/fs2/FMW_Home -invPtrLoc /var/opt/oracle/oraInst.loc -ldl /babc1/app02/fs1/inst/apps/BSI_abc1/admin/log/clone/fmwT2PApply -silent true -debug true -executeSysPrereqs false

              Script Executed in 4356909 milliseconds, returning status 255

              ERROR: Script failed, exit code 255

               

              SEVERE : Jun 27, 2016 3:50:26 AM - ERROR - CLONE-20218 Cloning is not successful.

              SEVERE : Jun 27, 2016 3:50:26 AM - CAUSE - CLONE-20218 An internal operation failed.

              SEVERE : Jun 27, 2016 3:50:26 AM - ACTION - CLONE-20218 Provide the clone log and error file for investigation.

              1. java.lang.NullPointerException

                      at oracle.as.install.engine.modules.install.util.InstallUtils.setAdditionalOracleHomeProperty(InstallUtils.java:333)

                      at oracle.as.clone.util.OracleHomeDepRefUtil.setAdditionalOracleHomePropertyUsingInstallUtils(OracleHomeDepRefUtil.java:493)

                      at oracle.as.clone.util.OracleHomeDepRefUtil.setOHProperties(OracleHomeDepRefUtil.java:474)

                      at oracle.as.clone.util.OracleHomeDepRefUtil.setOracleHomeProperties(OracleHomeDepRefUtil.java:424)

                      at oracle.as.clone.mwhome.cloner.OracleHomeApplyCloner.updateOracleHomeInfo(OracleHomeApplyCloner.java:1904)

                      at oracle.as.clone.mwhome.cloner.NormalOracleHomeApplyCloner.doPostApplyClone(NormalOracleHomeApplyCloner.java:127)

               

              Fixes tried for this :

               

              a) To avoid timeout errors , I had set the variable

                  export TIMEDPROCESS_TIMEOUT= -114

               

              b) Found that fs_clone was failing in create FMW_HOME stage. It had an issue copying FMW_Home from run to patch.

                  Removed some huge log files under $FMW_HOME

               

              c) OraInventory issue -- > Added patch fs entries in orainventory

               

              3) After fixing the inventory issue, fs_clone passed the above error stage but then it failed with another error.

                 Eventhough adop -status showed fs_clone completed on bsi003, there were some errors reported while bringing up patch fs adminserver

                 In the previous run, FMW_home was removed from patch fs. So , adop couldn't find the env file and it got failed

               

                 Node bsi003:

                     - Fs_clone Status: COMPLETED

                  [PROCEDURE] [START 2016/06/29 00:21:16] Checking if node "abc1" is abandoned

                  [STATEMENT] abandon_sid : 0

                  [PROCEDURE] [END   2016/06/29 00:21:16] Checking if node "abc2" is abandoned

                  Node db129:

                     - Fs_clone Status: NOT STARTED

               

              The below error was mentioned in the log file.

               

                       [STATEMENT] adop will not do fs_clone on completed nodes since FORCE option is not set

                  [UNEXPECTED]Error occurred while executing "sh /abc1/app02/fs2/inst/apps/***/admin/scripts/adadminsrvctl.sh start -nopromptmsg -silent forcepatchfs"

               

               

              4) when adop phase=fs_clone is mentioned , fs_clone will be started from previous failed session. Since there is no patch fs, adop failed again when trying to locate patch fs files.

               

              When "force=yes" option is used, it starts fs_clone from scratch. It removes the complete FMW_HOME from patch fs and recreates it ( something similar to adcfgclone)

               

              Node Name       Node Type Phase Status Started Finished Elapsed

              --------------- ---------- --------------- --------------- -------------------- -------------------- ------------

              abc1 master FS_CLONE COMPLETED       2016/06/26 03:47:13 2016/06/29 04:13:43  72:26:30

               

              abc2 slave      CONFIG_CLONE COMPLETED       2016/06/26 03:47:14 2016/06/29 05:16:49  73:29:35



              Happy troubleshooting