SOA Messaging Reliability through JMS and Oracle Service Bus

Version 9

    by Sebastian Lik-Keung Ma


    This article describes the design of a reliable messaging solution for SOA integration projects. It uses concepts like canonical schemas, durable POJO (Plain Old Java Object) messages, publish-subscribe and error handling within the Oracle SOA, Oracle Service Bus and WebLogic JMS infrastructure.

     

    The JMS Message

     

    For consistency, the type of message we are publishing will be based on a canonical schema. Typically, you would have a JDeveloper project that stores the canonical schemas used by all your custom applications. These schemas could be version-controlled, e.g. by Subversion, as well as deployed to MDS as a SOA bundle. Sample JDeveloper project containing canonical schemas shown below.

     

    image001.jpg

    Figure 1. JDeveloper project containing canonical schemas

     

    In this article, we will use the employeeWorkSchedule schema as our message type.

     

    <?xml version = '1.0' encoding = 'UTF-8'?>
    <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
     xmlns:cmn="http://scm.com/soacommon/xsd/common"
                xmlns:empws="http://scm.com/soacommon/xsd/employeeWorkSchedule"
     targetNamespace="http://scm.com/soacommon/xsd/employeeWorkSchedule"
     elementFormDefault="qualified">
    
        <xsd:import schemaLocation="common_v1_0.xsd" namespace="http://scm.com/soacommon/xsd/common" />
    
        <xsd:element name="employeeWorkSchedule" type="empws:tEmployeeWorkSchedule"/>
        <xsd:annotation>
            <xsd:documentation>Definition of an Employee Work Schedule message</xsd:documentation>
        </xsd:annotation>
        <xsd:complexType name="tEmployeeWorkSchedule">
            <xsd:sequence>
                <xsd:element ref="empws:employeeWorkScheduleInfo"/>
                <xsd:element ref="cmn:messageProperties"/>
                <xsd:element ref="cmn:standardResponse"/>
                <xsd:element ref="cmn:standardFault"/>
            </xsd:sequence>
    </xsd:complexType>
    
    
    
    
    

     

    Publishing JMS message from ADF Java application

     

    We publish our message to JMS from ADF applications. To simplify the process of using Java to construct message objects based on our XSD canonical schema, we use the JAXB (Java Architecture for XML Binding)

    tool from JDeveloper.

     

    As shown in Figure 2a below, select the XSD file, right-click and select “Generate JAXB 2.0 Content Model”.

     

    image002.jpg

    Figure 2a. JDeveloper JAXB generator


    image003.jpg

    Figure 2b. JDeveloper JAXB generator

     

    The generated POJO classes can be stored and version-controlled in a separate JDeveloper project. See Figure 2b where the output source goes to a directory different from the schema file. This project will produce the JAR file to be referenced by other ADF applications. Figure 2b shows the generated POJO classes.

     

    image004.png

    Figure 2c. Generated POJO classes

     

    The ADF application that wants publish the canonical message will include the JAR file in its project properties. See Figure 3 below.

     

    image005.jpg

    Figure 3. Referenced library JAR file containing the JAXB generated POJO classes

     

    The Java codes in an ADF application can now construct the JMS message based on strongly-typed POJO classes, e.g. TEmployeeWorkSchedule and TEmployeeWorkScheduleInfo below.

     

     

    import com.scm.soacommon.xsd.common.TMessageProperties;
    import com.scm.soacommon.xsd.employeeworkschedule.ObjectFactory;
    import com.scm.soacommon.xsd.employeeworkschedule.TEmployeeWorkSchedule;
    import com.scm.soacommon.xsd.employeeworkschedule.TEmployeeWorkScheduleInfo;
    import javax.xml.bind.JAXBContext;
    import javax.xml.bind.Marshaller;
    import javax.xml.bind.Unmarshaller;
    ...
    public class EmployeesImpl extends EntityImpl {
        ...
        private TEmployeeWorkSchedule constructEvent() {
            String msgId = UUID.randomUUID().toString();
            com.scm.soacommon.xsd.common.ObjectFactory cmnFactory =
                new com.scm.soacommon.xsd.common.ObjectFactory();
            ObjectFactory wsFactory = new ObjectFactory();
    
            TEmployeeWorkSchedule ws = wsFactory.createTEmployeeWorkSchedule();
            TEmployeeWorkScheduleInfo info = wsFactory.createTEmployeeWorkScheduleInfo();
            info.setEmployeeNo(this.getEmployeeId().toString());
    
            try {
                info.setScheduleDate(
                    wsFactory.createTEmployeeWorkScheduleInfoScheduleDate(
                        DatatypeFactory.newInstance().newXMLGregorianCalendar(
                            new GregorianCalendar(
                                2015,
                                1,
                                1))));
            } catch (DatatypeConfigurationException e) {
                logger.logWarning(e);
            }
            info.setShiftCodeCurrent(wsFactory.createTEmployeeWorkScheduleInfoShiftCodeCurrent("33333"));
            info.setShiftCodePrevious(wsFactory.createTEmployeeWorkScheduleInfoShiftCodePrevious("22222"));
            info.setWorkPatternName(wsFactory.createTEmployeeWorkScheduleInfoWorkPatternName("554321"));
            ws.setEmployeeWorkScheduleInfo(info);
    
            TMessageProperties msgProps = cmnFactory.createTMessageProperties();
            msgProps.setMessageId(cmnFactory.createTMessagePropertiesMessageId(msgId));
            msgProps.setAppName(cmnFactory.createTMessagePropertiesAppName("ADF2JMS"));
            msgProps.setSchemaName(cmnFactory.createTMessagePropertiesSchemaName(ws.getClass().getName()));
            ws.setMessageProperties(msgProps);
    
            return ws;
        }
    
    
    
    
    
    
    

     

    WebLogic JMS Topic

     

    JMS Topic not only supports publish-subscribe messaging model, it also provides better reliability via message durability. This means that JMS only removes the message from its topic when all durable subscribers have received the message. As long one subscriber is down or unavailable, the message will remain in the topic or until the message expires. The expiration property (Expiration Time for Incomplete UOW Messages) can be configured in WebLogic Admin console. The default value of -1 indicates the message never expires.

     

    image006.jpg

    Figure 4. To set message expiration value, click on the Advanced link.

     

    image007.jpg

    Figure 5. Default message expiration value is -1, i.e. message never expires.

     

    Setting a default Durable Subscriber

     

    Messages, once consumed by all subscribers, will disappear from the JMS topic. For tracing and debugging purposes, you can create an additional subscriber in WebLogic Admin console. This subscriber will collect the consumed messages within WebLogic. See Figure 6 below.

     

    image008.jpg

    Figure 6. Default durable subscriber created.

     

    To examine a message, click on the Show Messages button.

     

    A sample message detail is shown below.

     

    image009.jpg

    Figure 7. Message detail

     

    Note that the additional key-value property is set by the message publisher.

     

    Design of SOA Messaging Reliability through JMS and OSB

     

    The overall design of a reliable messaging solution in SOA is demonstrated in the diagram below.

     

    image010.png

    Figure 8. Overall design of a reliable messaging solution in SOA


    1. ADF application publishes the message to its JMS Topic. It is important to comply that messages published within the boundary of all custom applications be based on canonical schemas.


      Hint
      : The message publisher can set custom key-value message property so that subscribers can subscribe to message of interest only. See sample codes below.

       

      try {
           ObjectFactory wsFactory = new ObjectFactory();
           TEmployeeWorkSchedule ws = wsFactory.createTEmployeeWorkSchedule();
           ...
           TextMessage textMsg = session.createTextMessage();
           ...
           // adds a property for JMS message selector
           textMsg.setStringProperty("schema", ws.getClass().getName());     sender.send(textMsg);
      } catch (JMSException jmse) {
          logger.logWarning(jmse);
      }
      
      
      
      
      


    2. An OSB Proxy Service subscribes to the JMS Topic. Retry threshold is set in JMS Redelivery Limit.


      Hint
      : As long as the message publisher sets custom header properties for a message type, the Proxy Service can selectively subscribe to messages of interest only. This method is applicable if you have other Proxy Services that subscribe to the same JMS Topic for other message types.

    3. On receiving the message, the Proxy Services transforms canonical message to the 3rd Party message, then routes it to an OSB Business Service.
    4. The OSB Business Service then invokes the a 3rd Party Web Service.

    5. The Error Handler will be invoked for each failed invocation.

      Hint: Contrary to programming analogy, error in OSB is raised not after all retries have failed. It is important to understand that an error is raised to the Error Handler for each failure, including for each failed invocation by the Business Service.

    6. The Error Handler will determine the course of action to take. If retry threshold has exceeded, the Error Handler will re-publish the original ADF message to another JMS Topic to be “retried” by another Proxy Service.

    7. The number of JMS Topic-retry is dependent on individual application requirements.

    8. The final Proxy will determine the last action to take, i.e. when it has exhausted all retry attempts. Here, the final Proxy’s Error Handler invokes a SOA Composite. This composite simply contains a BPEL Logger that logs error messages to (ODL) Oracle Diagnostic Logging. ODL logs can be retrieved from Oracle Enterprise Manager and/or soa_server log files.


    OSB Configuration


    To configure OSB in Oracle SOA Suite 11g, you can use either the OSB web console or Oracle Enterprise Pack for Eclipse (OEPE). In our development environment, we use OEPE. To enable OSB Proxy Service as a JMS durable subscriber, the typical OSB configurations are described below as self-explanatory screenshots.

     

    image011.jpg

    Figure 9. Conguration page - Service Type set to Messaging Service

    image012.jpg

    Figure 10. Message Type Configuration page – Request Message Type set to XML with known canonical schema


    image013.jpg

    Figure 11. Transport Configuration page – Get All Headers set to Yes

     

    Figure 11, above, shows Get All Headers set to Yes. This is to ensure that all message headers, including custom message key-value properties, are received from the JMS Topic.

     

    image014.jpg

    Figure 12. JMS Transport Configuration page – Message Selector and Durable Subscription settings

     

    Figure 12, above, shows the setting of the Durable Subscriber to receive message of interest only via Message Selector.

     

    Intricacies of Retries and Error Handling

     

    This section details the scopes of the routing components inside the OSB Proxy Service. As shown in Figure 13 below, we have a “Route to Business Service” on one side and an “Error Handler” on the other.

     

    image015.png

    Figure 13. OSB Proxy Service - Message Flow


    Route scope

     

    In the Routing node, you will set the OSB Business Service and operation to invoke.

     

    image016.png

    Figure 14 Routing node – Business Service


    We highlight the various actions inside the Routing node shown in Figure 13 above.

     

    1. Assigns original incoming message body to a variable – body_source. The value of this variable will be used in Error Handler scope

      image017.png

      Figure 15. Assigns original message body to variable

    2. Assigns custom message property value to a variable – jms_property_schema. The value of this variable contains the schema name used by subsequent JMS Subscriber’s message selector

      image018.jpg

      Figure 16. Assigns the value of a custom message property to variable

    3. Transforms incoming canonical message to outgoing 3rd party message. XQuery is used for the transformation. The outgoing message body will be replaced by the result of the transformation.

      image019.jpg

      Figure 17. Replace outgoing message with transformed message

     

    Error Handler scope


    Our Error Handler performs error-handling action only if invocation of Business Service failures exceeds a threshold.

      1. Checks if retry count exceeds threshold. There are three known retry mechanisms in OSB:

        Note: WebLogic Server supports the JMSXDeliveryCount message property, which specifies the number of message delivery attempts, where the first attempt is 1, the next attempt is 2, and so on. WebLogic Server makes a best effort to persist the delivery count, so that the delivery count does not reset back to 1 after a server reboot. (From WebLogic Admin Console documentation for JMS Redelivery Limit)

         

        With this note in mind, we are using the third mechanism for our solution. The “If” condition checks if retries exceeded a threshold of 3, or if errorCode is BEA-380001.

        image020.jpg

        Figure 18. Checking if JMSXDeliveryCount exceeded threshold


        If the “If” condition is true, the Error Handler will publish a message to another JMS Topic for further retries.

        • Setting retry count on Business Service: In this mechanism, the Proxy will send the request to Business Service and wait for a response. Business Service will retry the destination service in case of failures depending on the retry count and delay. When Business Service retries it will NOT send a value of retry count in any way to the destination service. The number of retries made by the Business Service will also not be known by the Proxy service. In this case, you do not have knowledge of the number of tries it took.
        • Setting retry count and retry delay as routing options in Proxy while calling the Business Service: This mechanism is similar to that above; Proxy will send request only once to Business Service, and Business Service will retry. The only difference is that Proxy will set the Business Service’s Retry Count and Delay dynamically instead of configured values in the Business Service configuration. In this case, $outbound variable will have the Retry Count variable and the value would be the one set at design time. This value will not change at any time during the processing of Proxy. That is, if you had set the retry count as 5 in the routing option, $outbound will have a value of retry count as 5. Even if Business Service got a success response in only 2 retries, the value will not change in $outbound of Proxy.
        • Adding a JMS queue in the flow before the Proxy Service and setting the retry count and delay on the JMS queue: In this mechanism, the JMS server controls the retry count and the Proxy itself is invoked multiple times according to setting of retry on JMS Queue.
      2. Replaces outgoing message with original incoming message. The Error Handler will send the original failed message to a retry JMS Topic. However, due to the stateless nature of OSB, the original message from the Route scope is unavailable. As a workaround, we have stored the original message in step a.to a variable $body_source. Now we need only to replace the outgoing message with the original message content stored in $body_source

        image021.jpg

        Figure 19. Replace outgoing message with original content

         

      3. Sets custom key-value property to message header. Similary, if we want subsequent Subscriber to be able to receive only message of interest, here we set the custom key-value property for use as message selector.

        image022.jpg

        Figure 20. Set custom key-value message property in Transport Header

    References

     

    [1] Richards, Mark, Richard Monson-Haefel and David A. Chappell, Java Message Service (O’Reilly, 2009, 2nd ed.).

    [2] Walmsley, Priscilla, XQuery (O’Reilly, 2007).

    [3] Oracle Community FAQ  “OSB Logging re-try count value” (https://community.oracle.com/thread/2166596).

     

    About the Author

     

    Sebastian Lik-Keung Ma ( Sebastian Ma) has been a Software Engineer and Craftsman in Singapore and Germany, specializing in distributed software applications written in C++, Java and C#. He currently works as a SOA Practitioner and Software Development Manager in Singapore.

     


    Note: This article has been reviewed by the relevant Oracle product team and found to be in compliance with standards and practices for the use of Oracle products.