6 Replies Latest reply on Jul 11, 2018 3:53 PM by Vikram VK

    EPM FoundationService startup issue

    SaravananJ

      Hi,

      We recently applied below list of Opatchs to the Oracle EPM 11.1.2.4 environment.

       

      Patch 26496794: Oracle Hyperion Shared Services Patch Set Update: 11.1.2.4.006

      Patch 26751072 Patch Set Update 11.1.2.4.007 for Oracle Hyperion Shared Services Release

      Patch 27523845: Patch Set Update: 11.1.2.4.207 for Oracle Hyperion Financial Management

      Patch 27290291 Patch Set Update: 11.1.2.4.709 for Oracle Hyperion Financial Reporting

      Patch 27397226 Hyperion Essbase Server Patch Set Update 11.1.2.4.023

      Patch 27397214: PATCHSET UPDATE: HYPERION ESSBASE RTC 11.1.2.4.023

      Patch 27397211: PATCHSET UPDATE: HYPERION ESSBASE CLIENT MSI 11.1.2.4.023

      Patch 27397238: PATCHSET UPDATE: HYPERION ESSBASE ADMINISTRATION SERVICES SERVER 11.1.2.4.023

      Patch 27397232: PATCHSET UPDATE: HYPERION ANALYTIC PROVIDER SERVICES 11.1.2.4.023

      Patch 27780138: Patch Set Update: 11.1.2.4.013 for Oracle Hyperion Calculation Manager

      Patch 25312033: FDMEE PATCHSET UPDATE 11.1.2.4.220

      SU Patch [GFWX]: WLS PATCH SET UPDATE 10.3.6.0.180417 (Patch 27395085)

      Patch 28103100 Patch Set Update: 11.1.2.4.008 for Hyperion Planning

       

       

      Now when i try to startup services, the admin serivce starts fine, but Foundation service startsup in admin mode. Because both SHAREDSERVICE and WORKSPACE deployments fail due to below error.

       

      <05-Jul-2018 12:36:48 o'clock BST> <Error> <Deployer> <BEA-149205> <Failed to initialize the application 'SHAREDSERVICES [Version=11.1.2.0]' due to error weblogic.management.DeploymentException: [J2EE:160149]Error while processing library references. Unresolved application library references, defined in weblogic-application.xml: [Extension-Name: epm-applistener-libraries, exact-match: false]..

      weblogic.management.DeploymentException: [J2EE:160149]Error while processing library references. Unresolved application library references, defined in weblogic-application.xml: [Extension-Name: epm-applistener-libraries, exact-match: false].

      at weblogic.application.internal.flow.CheckLibraryReferenceFlow.prepare(CheckLibraryReferenceFlow.java:26)

      at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:648)

      at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)

      at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:191)

      at weblogic.application.internal.EarDeployment.prepare(EarDeployment.java:59)

      Truncated. see log file for complete stacktrace

      >

      <05-Jul-2018 12:36:49 o'clock BST> <Error> <Deployer> <BEA-149205> <Failed to initialize the application 'WORKSPACE [Version=11.1.2.0]' due to error weblogic.management.DeploymentException: [J2EE:160149]Error while processing library references. Unresolved application library references, defined in weblogic-application.xml: [Extension-Name: epm-applistener-libraries, exact-match: false]..

      weblogic.management.DeploymentException: [J2EE:160149]Error while processing library references. Unresolved application library references, defined in weblogic-application.xml: [Extension-Name: epm-applistener-libraries, exact-match: false].

      at weblogic.application.internal.flow.CheckLibraryReferenceFlow.prepare(CheckLibraryReferenceFlow.java:26)

      at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:648)

      at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)

      at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:191)

      at weblogic.application.internal.EarDeployment.prepare(EarDeployment.java:59)

      Truncated. see log file for complete stacktrace

      >

       

       

      I even tried to revoke the patches, re-install all components, reapply patch and re-configure. Still ending up with same error. Please advice if anyone faced similar issue.

       

      Regards

      Saravanan Jagadeesan

        • 1. Re: EPM FoundationService startup issue
          Cord234

          It looks like you're having an issue with your Weblogic config.xml file.  See related: Unable to Connect to Essbase Administration Services (EAS), Error "Unresolved application library references" (Doc ID 2374501.1)

          • 2. Re: EPM FoundationService startup issue
            Vikram VK

            So basically issue is you re-deployed the Shared Services via Configuration Utility and it is failing however it was suggested to do only when you have websphere.

             

            https://updates.oracle.com/Orion/Services/download?type=readme&aru=21540008#_Toc491169426

            1. For WebSphere Application Server only: Redeploy the restored web application. See "Deploying EPM System Products to WebSphere Application Server" in the Oracle Enterprise Performance Management System Installation and Configuration Guide for instructions.

             

            Any particular reason why it was done?

             

            Also, did you followed Cord234 suggestion?

             

            Thanks.

            • 3. Re: EPM FoundationService startup issue
              SaravananJ

              Thanks @Vikram and @Cord234.

               

              I followed Cord234 referenced oracle doc, but no luck. The config.xml paths seemed fine.

              Also I did redeploy the shared services after I faced the issue, thinking redeploy will fix it. but it did not.

               

              However I had now resolved the issue by following steps.

              1) Did a rollback of all Opatches (both EPM ones and oracle_common).

              2) Re-install the EPM components by running install tool.

              3) Re-apply all the patches in proper sequence (both EPM and Oracle_common).

              4) Removed the domain folder  (/user_projects/domain/EPMSystem).

              5) Reconfigure and Re-deploy Foundation service followed by all other service.

               

              Though this is a much elaborate process, it did fix any corruption that may have either happened at config.xml or web-application.xml or libraries.

               

              I doubt the weblogic patch could have caused the issue. Because while applying the weblogic patch, though I had shut down all EPM services, there were few Payara glassfish application running (which is related to HFM EPM Maestro). I have a feeling that might have corrupted the weblogic binaries. By doing a full re-install -> re-configure -> re-deploy, I got rid of the corruption.

               

              Thanks much!

              Saravanan

              • 4. Re: EPM FoundationService startup issue
                Vikram VK

                I too faced similar kind of issues when applied the weblogic security patch SU Patch [GFWX]: WLS PATCH SET UPDATE 10.3.6.0.180417 (Patch 27395085)

                 

                I am glad to see that you have resolved the issue.

                 

                BTW, can you check whether GFWX patch is active because it need jdk6u191. You can validate via weblogic console.

                 

                 

                Thanks.

                • 5. Re: EPM FoundationService startup issue
                  SaravananJ

                  HI Vikram,

                   

                  I am not sure about the java dependency. I have JDK 1.6.0.35 and Jrocket 1.6.0.37 on my servers. Also below is what is see from console. So far seem to work fine for me

                   

                  • 6. Re: EPM FoundationService startup issue
                    Vikram VK

                    If everything is working fine then no need to take any action. This is just a FYI.

                     

                    April 2018 Critical Patch Update: Additional Information about the Oracle WebLogic Server Vulnerability CVE-2018-2628 (Doc ID 2395745.1)

                     

                    If you see this doc it says clearly that if you are in jdk6 then you need jdk6u191.

                     

                     

                    If you want to find out weblogic security risk is mitigated then you need to do the Vulnerability scan then this might pop up.

                     

                    Thanks for sharing screenshot.