3 Replies Latest reply: Jul 30, 2014 5:10 AM by User10124166-Oracle RSS

    Assign operation taking 30 seconds in OSB Process Flow

    Manesh Abhimanyu

      Hi OTNians,

       

      We have a OSB Proxy service which does some basic operations like calling a third party web service business service.

       

      But prior to the start of business service call, there are TWO assign operations:

       

      1. Assign [ $body/* ] to [ OrgBody ]

       

      2. Assign [ 'file:////u02/Oracle/Middleware/filedrop/Errorfiles' ] to [ ErrorFolder ]

       

      In the Process flow of the Proxy service, we have the above two assign statements and then the business service. Quite simple!

       

      By enabling tracing we found that the above two assign operation is taking for than 15 seconds sometimes. (This high flow time occurs 5-6 times a day basically during peak hours)

       

      Please provide some pointers how we can debug further or any known issue with the assign operation. (or related performance issue)

       

      Trace Logs:

       

      ####<Jul 16, 2014 10:06:31 AM SGT> <Info> <OSB Kernel> <Stocks1WEB02> <stockex_server1> <[ACTIVE] ExecuteThread: '225' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <ee4e081806644bdd:-8efa9bb:14510826e24:-8000-0000000002814bd3> <1405476391490> <BEA-398200> <

      [OSB Tracing] Inbound request was received.

       

      Service Ref = MMJ/StocksSummary/GetStocksSummary/GetStocksSummary

      URI = /MMJ/GetStocksSummary

      Message ID = XXXX73667714891957--8efxxxb.1473cddc855.xxx0

       

       

      ####<Jul 16, 2014 10:06:47 AM SGT> <Info> <OSB Kernel> <Stocks1WEB02> <stockex_server1> <[ACTIVE] ExecuteThread: '225' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <ee4e081806644bdd:-8efa9bb:14510826e24:-8000-0000000002814bd3> <1405476391496> <BEA-398202> <

      [OSB Tracing] Outbound request was sent.

       

      Service Ref = RRS/BSESummary/GetBSESummary/BizBSEGetDStocks

      URI = http://bombaystockexchange.mmj.com/getBSESummary

       

      Thanks..!

        • 1. Re: Assign operation taking 30 seconds in OSB Process Flow
          Ankit kalanoria

          Above tracing doesn't only mean that assign action is taking time, it could also mean that there is starvation of kernel thread (socket muxer threads) in weblogic. This could happen under high load. When request pipeline completes in route, it register a muxer thread for callback and copies context variable there and then releases original request thread. If under high load, these socket muxer threads are not available then there could be unnecessary delay between context variable exchange as looks high possibility in your case.

           

          What we need to do is separate thread pool for muxer threads and osb req-resp pipeline threads. You need to:

          • Create custom workmanager and assign it to proxy and business service using dispatch policy. Defining parameters for work manager is tricky part and involves study of load and thread dump etc. I always get confused defining best tuned workmanager.
          • Or simply, its time to increase your hardware capacity. If your organization is planning further scalability in future then it could be time.

           

          Above is the direction and solution which I will recommend. As far as assign action is concerned, nothing looks suspicious on that. Still you can try "$body/*[1]". And also put a log statement after assign and before route. It will help verifying if it is issue of assign or thread starvation.

           

          Hope this helps.

           

          Regards,

          Ankit

          • 2. Re: Assign operation taking 30 seconds in OSB Process Flow
            NevinCleetus-Oracle

            Hi,

            As Ankit mentioned, I doubt the issue is with the assign action. The above assign actions should not cause such a delay. As the performance degradation is intermittent and occurs only during peak load, it is less likely due to assign action. It would be ideal to take few thread dumps during heavy load and analyze it.

             

            Also, did you collect the statistics before and after the assign operations?  The trace message tells it is collected after sending the outbound message.

            • 3. Re: Assign operation taking 30 seconds in OSB Process Flow
              User10124166-Oracle

              Any further assistance needed on this?

              Did you collect the stats. before and after the assign operation?

               

              Thanks,

              Sandeep