13 Replies Latest reply: Apr 8, 2012 11:16 PM by Naveen1215 RSS

    Regarding recovery in One way Bpel Process

    Naveen1215
      Hi,
      I have the following scenario,

      1) Bpel Process A receives the request message from the consumer and validates it and sends an ack to the consumer and puts the messages into a Queue using JMS.
      2) BPEL Process B(One way BPEL) polls the messages in the Queue and then calls a stored procedure using database adapter, In this process if the stored procedure is down, then I use my fault policies to retry for 3 times. The issue that i am facing is, when the target system is down all my message are lost. How can i ensure that all my messages are delivered to the target system once the target system is up. I don't want to use a sync bpel since the performance is the major issue. The major criteria is once the target system is up, all the messages that could not be deliverd should be delivered first and then the incoming messages must be delivered. The order of delivery is important.
      Thanks,
      Naveen
        • 1. Re: Regarding recovery in One way Bpel Process
          Arik
          Hi Naveen,

          Few questions: For how long does the DB is down? How often it happens? What is the frequency of the customer's requests?

          One solution I can think about is, if the db invoke fails, catch it, and then you can put the messages into a Queue again.
          But I didn't realy get into the whole picture.

          Arik
          • 2. Re: Regarding recovery in One way Bpel Process
            Naveen1215
            Hi Arik,

            Actually when the target system is down, there are two factors that need to be taken into consideration, order of delivery and guaranteed delivery. So, if we catch the messages and put them in a JMS queue, when the target system is up the message in the error queue needs to be manually moved in weblogic server to the queue from which the bpel process is polling. By that time the source system which is unaware that the target system is down will be sending te messages. So the order of delivery is lost.

            Thanks,
            Naveen.
            • 3. Re: Regarding recovery in One way Bpel Process
              Arik
              Can the Ack' you send back to the source be with something like an error status? If so, you can add a small scheck to see if the target system is down, before continue.
              That way you will receive the request again.
              • 4. Re: Regarding recovery in One way Bpel Process
                RobertMetcalf
                Hi,

                From Naveen question the ACK is sent when the message is initially placed in the queue.
                We will need to implement a similar setup.
                Once the requesting application receives an ACK it should be 100% certain that the message will be reliable delivered and no longer has to worry about it.
                The AIA/SOA/Weblogic infrastructure must be setup to either reliable deliver the message or if it fails generate an error and save the message in case it needs to be resubmitted.
                Our DBA's tell us they may from time to time take systems offline for patching over a number of hours. It would be nice in this case for SOA to save the messages and deliver them when the maintenance period is over. This was something that attracted us to using queues.

                Robert
                • 5. Re: Regarding recovery in One way Bpel Process
                  Sridhar-SOA
                  Naveen,

                  1. To ensure delivery of messages to the target system in case of db/procedure inavailibility we can use BPEL Engines Recovery Configuration ( in the case the type of message would be Invoke).
                  2. To ensure that the messages are delivered in an order , we need to add ws-rm policy where we can set INOrder Delivery Assurance.

                  Regards,
                  Sridhar.
                  • 6. Re: Regarding recovery in One way Bpel Process
                    Naveen1215
                    Hi,

                    Thanks for your replies.
                    sridhar- If we use BPEL Engines Recovery Configuration, then it requires manual intervention. Is there any chance that we can make it automated.

                    Thanks,
                    Naveen.
                    • 7. Re: Regarding recovery in One way Bpel Process
                      Sridhar-SOA
                      Naveen,

                      Yes Recovery can be Automated. Its very simple and easy. Below is a link from Oracle Documentation that explains in detail about the same.

                      http://docs.oracle.com/cd/E17904_01/integration.1111/e10226/bp_config.htm

                      Sections 11.2,11.3 in the link above will guide you doing a automated recovery.

                      Another blog about the same by Marc - http://orasoa.blogspot.in/2010/12/soa-11g-automatic-recovery.html


                      (Note: Be aware of the negative consequences of using this automated recovery as the faulted instances will keep on being submitted for recovery. So use MaxRecoverAttempt property to limit the number of times a instance need to be tried for recovery)

                      Regards,
                      Sridhar.
                      • 8. Re: Regarding recovery in One way Bpel Process
                        Naveen1215
                        Hi sridhar,

                        Thanks for Your reply.

                        Here i am facing two issues with automated recovery setting in em console.

                        1) When the target system is up, The new incoming messages are delivered before the recovered message are delivered. The "inorder delivery " is not achieved.
                        2) This property is not project specific.When we apply this setting in console it applies to all the projects. Is there any way we can make this setting project specific.

                        Thanks,
                        Naveen
                        • 9. Re: Regarding recovery in One way Bpel Process
                          Neeraj Sehgal
                          Hi Naveen,

                          I'll try to put forward a solution. Happy to improve it by others.

                          Looking at your use-case, you have the following requirements
                          - Sequential processing of the messages (Unit-of-Order on JMS would have rescued you if you wouldn't have any concern about the performance)
                          - Less impact on Performance (that's why you do not want to use Sync process)

                          The solution could be
                          1.) Use a Mediator in front of your BPEL process B. This mediator will poll the messages from the JMS queue.
                          2.) Enable FIFO Resequencing on Mediator based on the group to which the multiple messages belong (identify this intelligently). e.g. if multiple PO updates are expected PO number can be a group ID. This will ensure that messages are fetched from the JMS and maintained in the Mediator Resequencer table.
                          3.) Call your BPEL process B from this Mediator synchronously so that it processes the next message of the group (say for a PO number) only after first message of that group is successfully completed.
                          4.) Mediator by default will call the BPEL process B independent of each group (e.g. PO1 has 3 messages and PO2 has 5 messages grouped in Mediator store, it will parallely create 2 instances one for PO1 and one for PO2). Hence optimizing the performance.
                          5.) However, I'm not sure that if the target system is not available or the message fails, would it resubmit the message or not? I think there should be some out of the box option for that. However, if it is not, automatic recovery should come to rescue as the case described in above thread wouldn't be encountered because it locks all the remaining messages of a group unless the first one is processed

                          You can arrive at a different variant as well based on the requirements.

                          For more info on resequencing see below
                          http://docs.oracle.com/cd/E17904_01/integration.1111/e10224/med_resequencer.htm

                          Regards,
                          Neeraj Sehgal
                          • 10. Re: Regarding recovery in One way Bpel Process
                            Naveen1215
                            Hi,

                            Thanks for your reply.

                            Can you give me more information on where we can set the unit of order on JMS.

                            Thanks,
                            Naveen
                            • 11. Re: Regarding recovery in One way Bpel Process
                              Neeraj Sehgal
                              Hi Naveen,

                              More information on Unit-Of-Order can be found at
                              http://docs.oracle.com/cd/E12840_01/wls/docs103/jms/uoo.html

                              How to configure Unit of order is explained here
                              http://docs.oracle.com/cd/E12840_01/wls/docs103/ConsoleHelp/taskhelp/jms_modules/connection_factories/ConfigureUOOParams.html

                              For this to work with SOA, you'll have to create a synchronous connection to JMS queue which is explained as under.
                              Consuming jms queue synchronously in SOA11g

                              Regards,
                              Neeraj Sehgal
                              • 12. Re: Regarding recovery in One way Bpel Process
                                Naveen1215
                                Hi Neeraj,

                                I was trying using the mediators(FIFO). I could see that its showing some impact on the performance. In ideal scenario, The 2nd message cannot be processed until the 1st message is processed. In ideal scenario, i want multiple messages to be processed at a single time.When the target system is down , once the target system comes up all the messages must be recoverable and must follow an inorder delivery.


                                Thanks,
                                Naveen
                                • 13. Re: Regarding recovery in One way Bpel Process
                                  Naveen1215
                                  Hi,

                                  Can anyone please suggest a solution for this.

                                  Thanks,
                                  Naveen