FDMEE Servers are not load balancing

4223228

    Hi FDMEE Experts / Hyperion Infra Experts

     

    We have 4 load balancing servers which hosts FDMEE 11.1.2.4. However they dont seem to be properly load balancing the requests.Here are the 2 problems we have with load balancing

     

    1. FDMEE batch loads are always routed to only one particular FDMEE server.

     

    2. FDMEE user loads are routed to all 4 servers .however the user loads are always very high on 2 servers and the loads on remaining 2 servers are always very low.

     

    Please provide your thoughts on what could be causing this problem as well provide your ideas/comments/directions to troubleshoot this problem.

     

    Appreciate your help!

      • 1. Re: FDMEE Servers are not load balancing
        WPaffhausen

        Hello,

         

        I believe I have already answered your question in the other channel.  But it all depends on what patchlevel you are at and how you configured/setup your Load Balancing.  Assuming you setup your load-balancing correctly there are patches that correct some of the behavior you are seeing.  I would suggest you are fully patched for both WebLogic and FDMEE.

         

        There has been FDMEE patches within the last 2+ months that address Batch Automation and Load Balancing between servers.

         

        Thank you,

        Wayne

        1 位用户发现它有用
        • 2. Re: FDMEE Servers are not load balancing
          Madhusudhan. M

          If you are on patch level 11.1.2.4.220, there is a PSE for this issue.

           

          Defect Number

          Defect Fixed

          • 28907648 

          JOBS IN A BATCH ARE NOT LOAD BALANCED ALL JOBS RUN ON SINGLE NODE

          Release 11.1.2.4.220 Patch Set Exception (PSE): 28909964 

           

          Thanks,

          Mady

          • 3. Re: FDMEE Servers are not load balancing
            4223228

            Hi Wayne / Mady

             

            Thank you for your response and suggestions.

             

            Regarding FDMEE:

             

            We are on FDMEE 11.1.2.4.210.012 and we are having this issue since 2017 and I am not sure what version or patch level of FDMEE we were in at that point of time.But this is an ongoing issue with no luck.Would it still be applicable to upgrade to the .220 release with the PSE? Also does this release and Patch set is specific to fix load balancing with FDMEE batch loads? Because we have load balancing issue with the User load as well.

             

            Regarding the Load Balancing Configuration  :

             

            One change that I was made aware that we did try to resolve this problem was, to change the load balancing algorithm in Weblogic from ROund Robin to Weight based with no luck.Besides this do you suggest any other config or setup particularly to check.Could you please refer me to any configuration guide that I can refer to ensure the load balancing is setup correctly?

            • 4. Re: FDMEE Servers are not load balancing
              Madhusudhan. M

              I found like this issue resolved in 220 with PSE. Better you log ticket with oracle if you need more clarity on this.

               

              Thanks

              Madhusudhan

               

              Please mark helpful or correct if you find suitable information.

              1 位用户发现它有用
              • 5. Re: FDMEE Servers are not load balancing
                iArchSolutions-Joe

                The issue should be corrected with that .220 PSE fix.  Make sure you apply the base .220 PSU first then the newer PSE's. As for confirmation that the load balancing is setup correctly, re-check your mod_wl_ohs.conf file from OHS to ensure all four nodes are listed in the FDMEE sections.  You can also check inside of WebLogic under the "Cluster section" to ensure you see all four nodes listed for the ERPIntegrator cluster.  But as WPaffhausen and Madhusudhan. M mentioned, likely the patch is your ticket!  We had the same issue last week at another client.

                 

                Joe Malewicki

                Certified Cloud Architect

                joe.malewicki@iarchsolutions.com

                203-880-4268

                • 6. Re: FDMEE Servers are not load balancing
                  4223228

                  Thanks for your comments Joe!.Upon checking the load balancing setup , all the 4 FDMEE nodes appear in the Weblogic Console however DO NOT appear in the config file mentioned above in any of the foundation server.

                   

                  Please advise will that have an impact?If so how do we correct it?

                  • 7. Re: FDMEE Servers are not load balancing
                    iArchSolutions-Joe

                    Ahhh the plot thickens!  OK so piece of cake.  There are a few scenarios on how to update this depending upon how you built your system.  If you have multiple OHS servers and when you originally configure the "web server" task during the configuration routine, you might have selected to use a "shared file location" for the OHS config files.  If so it'll be a UNC path you provided (i.e \\server-name\share-name).  If you only have a single OHS server then the config files will be located under the httpConfig folder structure buried down under your Oracle EPM Instance (<drive>:\Oracle\Middleware\user_projects\<instance-name>\etc....

                     

                    You need to find out first where these files are located.  Then backup those config files (mod_wl_ohs.conf, httpd.conf, epm.conf, etc...). there will be a few of them you want to copy safely.  Grab them all for safe keeping ion their original state.

                     

                    Backup your SS database/schema (always backup first!) as well in a COLD state.

                     

                    Re-run the web server config task using the configtool.bat|sh.  You should have everything shutdown while you do this across all the servers to prevent Shared Services DB table locks.  YOu'll expand Foundation and select ONLY the Configure web server task.  Nothing else. Keep all your defaults as you step through that process. If you used a shard location in your original web server config I mentioned about then you are done updating.  If you did two separate OPHS nodes on two servers WITHOUT a shared location, you need to run the web server config on that second OHS box as well.

                     

                    Once done you should take a look at your mod_wl_ohs.conf file. it should update with all of the other nodes in your deployment now.  You'll see the different locationMatch sections for each product and each product will have x number of nodes added.

                     

                    Let me know how that works!  But again - I'd also get that patched up!  Any questions on any of this, just ask.

                     

                    Joe Malewicki

                    Certified Cloud Architect

                    joe.malewicki@iarchsolutions.com

                    203-880-4268

                     

                    ***Please don’t forget to mark comments as helpful or answered so others in the community can better resolve the same issues if it is encountered.***

                    • 8. Re: FDMEE Servers are not load balancing
                      iArchSolutions-Joe

                      Any luck after the web server (OHS) config?

                       

                      Joe Malewicki

                      Certified Cloud Architect

                      joe.malewicki@iarchsolutions.com

                      203-880-4268

                       

                      ***Please don’t forget to mark comments as helpful or answered so others in the community can better resolve the same issues if it is encountered.***

                      • 9. Re: FDMEE Servers are not load balancing
                        4223228

                        Hi Joe,

                         

                        Thanks much for the followup.Just to re confirm regarding the mod_wl_ohs.conf file, should this be checked in FDMEE Server or Foundation Server? I did find this config file in the Foundation server but didn't find the FDMEE nodes listed in the config file.Please confirm.We have 3 foundation servers but I wasn't part of the initial setup so not sure whether a shared file location was selected.Is there a way to find where the config files will be?

                         

                        We do see the FDMEE nodes listed in the OHS config file in QA environment.However  the batch load issue (concentrated to one server) exist  in QA as well.So we are going towards the path of patching FDMEE to 11.1.2.4.220 with the PSE.

                         

                        Thanks

                        Manoj

                        • 10. Re: FDMEE Servers are not load balancing
                          iArchSolutions-Joe

                          Sorry for the delay in replying - the mod_wl_ohs.conf file will live on the server(s) hosting the Oracle HTTP Server (OHS) instance(s).  You an check this blog I just threw up for another user and see where to find the right version(s) of the file:

                           

                          https://www.iarchsolutions.com/how-to-enable-debug-mode-for-oracle-http-server-ohs-within-for-a-weblogic-managed-server/

                           

                          You should try the re-run web server config and see if those entries show up.  the FDMEE patch won't add the sections.  and doing it manually means something didn't update that file as it soul've been so that's odd as well.

                           

                          Joe Malewicki

                          Certified Cloud Architect

                          joe.malewicki@iarchsolutions.com

                          203-880-4268

                           

                          ***Please don’t forget to mark comments as helpful or answered so others in the community can better resolve the same issues if it is encountered.***