This article describes a concrete customer case in a simplified variant; an on-premise SAP system is to be integrated with the SaaS CRM solution Salesforce.com (SFDC).
By Sven Bernhardt, Alexander Däubler, and Cornelia Spanner
Integrating distributed systems with each other has been a challenge for many years. In the context of digital transformation, companies are using more and more Software as a Service (SaaS) applications to address standard processes, so that they can focus on improving and evolving the core business. This means even more complexity in integration, since cloud and on-premise systems need to be integrated.
There are three typical integration scenarios:
- On-premise to on-premise: Integration is done on-premise, based on, for example, Oracle SOA Suite
- On-premise to cloud: Integration is done in the cloud (e.g., using ICS) or on an on-premise integration platform
- Cloud to cloud: Integration is done in the cloud
The arising integration challenges can be addressed by either hybrid cloud integration or a full Integration Platform as a Service (iPaaS) approach, depending on a company’s center of gravity:
A hybrid approach (left) should be chosen if both cloud and on-premise applications must be integrated.
In a full iPaaS scenario (right), most of the integrations need to be done between cloud-based applications, because the center of gravity has moved to the cloud. The need for integrations with on-premise applications is low, because there are usually only a few legacy applications still hosted on-premise.
Oracle Integration Cloud Service (ICS) is a complete and strategic cloud-based integration platform that can be used in the context of a hybrid approach as well as when taking a full iPaaS approach. Using ICS, the definition of integrations comes to a new level. Pre-built integrations available through Oracle Cloud Marketplace that implement common integration scenarios can be used as a basis for new integrations and adapted to specific user needs. For data mappings, recommendations are shown based on knowledge derived from the same mappings done by other users. In conjunction with a very clearly structured and intuitive UI, which also enables non-technical users to define integrations between systems and applications, new integrations can be created rapidly and efficiently.
One key differentiator of Oracle ICS with respect to its competitors is that cloud-to-on-premise integrations are done using a lightweight agent concept. The so-called “connectivity agent” has to be hosted and deployed on premise, ensuring communication between ICS and the on-premise applications, without the need to open firewall ports.
This article describes a concrete customer case in a simplified variant; an on-premise SAP system is to be integrated with the SaaS CRM solution Salesforce.com (SFDC). This use case will be explained, followed by an architectural overview and a deeper look into the concrete implementation steps in Oracle ICS.
Business Use Case: Closed-loop Order Management
In the German market, SAP is the leader in on-premise ERP solutions, and is key to integration projects in that country. Recently, SFDC has been gaining market share—and thus the need to integrate SFDC and SAP will also increase tremendously.
One of our customers had precisely such a scenario: during its lifecycle, an order had to be synchronized between SAP and SFDC. The figure below depicts the general principle of this closed-loop order scenario.
The business flow works as follows: when an order is created in SFDC it needs to be transmitted to the on-premise SAP system for further processing; whenever the status of an SAP order changes, the new status value needs to be synchronized with SFDC.
From a business perspective, this scenario is straightforward and offers a great benefit for the customer regarding consistency and comprehensibility of the order lifecycle.
Challenges and Solution Architecture
Integrating different systems is usually complex, because the APIs of the applications to be integrated are often quite technical and use complex data models that are also challenging to translate. In addition, the integration of cloud with on-premise applications makes things even worse, since the communication between those applications has to pass a company’s firewall. Security exceptions, such as opening application-specific ports in the firewall, have had to be defined to deal with this issue—usually preceded by exhausting and long-term discussions with the internal security department.
The good news: ICS helps overcome those challenges and decrease complexity significantly.
ICS provides a rich palette of different connectors that connect to any system easily, without a need to know about the technical details of the targeted systems. In addition, support for handling complex data models and the needed translations is provided (e.g., through pre-defined data mappings available through the Cloud marketplace or recommendations given while implementing the mappings).
By introducing the concept of the connectivity agent, Oracle ICS puts an end to connectivity challenges. The agent is hosted on the on-premise side behind a company’s firewall and registers with ICS.
Communication is done between ICS and the on-premise agent using HTTPS, where the agent always initiates interaction with ICS. There is no direct communication between ICS and the on-premise systems.
The monitoring of the agent can be done via the ICS UI, so there is no need to set up additional monitoring capabilities on the on-premise side:
From an architectural point of view, using Oracle ICS in conjunction with the connectivity agent is the combination to be used for integrating SFDC with on-premise SAP. The figure below shows the targeted solution architecture for the previously described use case:
The following sections describe how to create and configure the components depicted in this figure and show how to create the integration from SFDC to SAP, and vice versa, using Oracle ICS.
Create SAP Order (SFDC to SAP Integration)
The first step is to integrate SFDC to SAP so that SAP can be informed about newly created SFDC orders.
Configure SFDC Connection
For sending SOAP messages to a defined endpoint over HTTP(S), SFDC uses the so-called Outbound Messaging functionality, as depicted in the figure below:
First, the attributes to be included in the message are defined. These attributes are needed for generating the outbound WSDL, which is required for defining the integration later on.
To send a message automatically when a certain event occurs in SFDC, a workflow rule needs to be created in SFDC. Each rule must define when (evaluation criteria) and under which conditions (rule criteria) the rule will trigger a specific action. For the SFDC to SAP integration, the criteria are pretty simple: when a new order is created, an outbound message is sent to ICS.
A sample order for this showcase could look as follows (note that the SAP Status field is empty for a recently created order that has not yet been processed in SAP):
Create Salesforce Connection
The integration requires a Salesforce connection to exchange messages with SFDC. In general, ICS supports both inbound and outbound communication. Assuming that the “Salesforce Custom Policy” security policy is chosen, the configuration of the connection in ICS is straightforward, requiring only the following input:
- Enterprise WSDL: Mandatory for any client that wants to use the SFDC web services, it describes all operations and objects available for an organization; it can be downloaded from the SFDC web app
- Username: SFDC user account name
- Password: Password for this account
The completed configuration looks like this:
If the connection test succeeds, it can be saved and is then ready to use.
Configure SAP Connection
Create Connectivity Agent
Exchanging data with the on-premise SAP system requires a second connection. As mentioned earlier, ICS uses a connectivity agent for handling interactions with on-premise systems, so the agent must be set up before the connection can be defined.
Before installing the agent, two prerequisites must be performed within the ICS instance: create a new agent group and download the agent installer.
An agent group is necessary for monitoring and managing an agent. Currently, only one agent is supported per agent group. The agent installer can be downloaded from the agent group configuration page:
For the showcase, a plain Oracle Linux VM is used. The only installation prerequisite is an applicable Oracle JDK. The connectivity agent installation is kicked off by a single command:
./cloud-connectivity-agent-installer.bsx -h=https://integration-deopitzicstrial.integration.us2.oraclecloud.com:443 -email@example.com -p=welcome1 —ad=OPITZ_ICS_AGENT -au=weblogic -ap=welcome1
The installer basically performs the following steps:
- Install and configure the agent
- Start up the agent
- Register the agent with the ICS agent group created earlier
If the installation was successful, the monitoring for the agent group will show that there is a connectivity agent assigned to it:
Create SAP Connection
Now that the connectivity agent is up and running, the SAP connection for the showcase can be created. The customization is a bit more complex than in the SFDC case, but still rather simple.
User and password settings for the SAP system must be provided. The connection properties section contains a couple of SAP specific fields that need to be set as shown below:
The JNDI Name field is important and depends on the on-premise agent configuration. Before an SAP connection can be executed successfully, the SAP adapter on the on-premise side must be configured properly:
- SAP Java connection libraries must be added to the agent’s classpath on the VM (as described here:
- The outbound connection pool, with the JNDI name as used in the ICS connection, has to be configured so that the parameters match the SAP installation
- Deployment target for the on-premise SAP adapter must defined correctly
Now the connection can be tested successfully and is ready for further usage.
With both connections configured, the integration can be implemented: it is triggered from SFDC, maps the incoming order message into an SAP request, and leverages the SAP connection to create an order.
When starting the definition of a new integration, the first step is to select an appropriate integration pattern. At the moment, ICS offers four different patterns:
The integration maps incoming SFDC data to invoke an SAP service, so the “Map My Data” pattern is the correct one in this case.
Defining an integration starts with creating the source and target connections. The Salesforce connection created earlier must be dragged to the source part of the integration flow. A proper name and description for the endpoint have to be defined. In addition to the properties defined in the connection (e.g., Enterprise WSDL and user credentials), some additional information specific to this integration needs to be set:
- Outbound Message WSDL: Created in SFDC, as described earlier, defining the message payload the integration expects:
- Response settings: “Send a response” should be unchecked so that no response is sent to SFDC
Now the SAP connection must be dragged to the target part of the integration. The wizard offers different categories for accessing SAP data (BAPI, RFC or IDOC). In the current example, for reasons of simplification, a custom SAP table where orders are stored is used. As an external interface for creating such orders, an RFC method called Z_CREATE_ORDER has been created, so this method needs to be chosen from the list:
As a next step, the attribute mapping between the two connections needs to be defined. The mapping is quite simple, as the target request contains just three fields: two are mapped based on input from the source payload, and the status field is always statically set to “NEW”:
Now the integration itself is finished except for the business identifiers that can be defined to consistently track specific instances for this integration:
After at least one tracking field has been defined, the whole integration, which looks similar to this, can be saved:
The SAP call is synchronous (i.e., it returns a response). As this is just a technical acknowledgment, there is no need to forward any information to SFDC—a response is not expected at all. When the progress value of the integration is set to 100%, the integration is ready to be activated. The activation is the last step before the integration can actually be used.
Now the integration is ready for usage and can be easily tested by creating a new order in SFDC. Using the monitoring capabilities of ICS, any instance that has been executed can be tracked. The integration overview shows the most recent integrations that have been executed; the primary tracking identifier is also depicted here:
When one such instance is selected, additional information is shown:
In this example everything has worked perfectly, so the integration from SFDC to SAP is completed. Anew record has been created in the SAP table:
Now that the order integration from SFDC to SAP works, the way back needs to be implemented for forwarding order status updates from SAP to SFDC (thus closing the order management loop).
Update SFDC Order Status (SAP to SFDC Integration)
Configure Inbound SOAP Connection
The SAP adapter basically supports inbound communication by receiving so-called IDOC documents. An IDOC represents a standard SAP data object used for data exchange between SAP and other systems. Since the showcase is simplified and a custom table in SAP is used, using IDOCs is not an option for triggering the integration. Instead, the integration is <Q: missing word?>exposing a SOAP interface that can be called manually via, for example, SoapUI.
To create a SOAP connection, a WSDL needs to be provided. For the showcase, a simple SOAP interface with a single operation is used as a trigger for the integration. After saving and successfully testing the connection, the integration can be defined in the next step.
For this integration, the “Map My Data” pattern has again been used for the implementation. An alternative would be the “Orchestration” pattern, but, in general, an orchestration pattern is used if there are requirements regarding more advanced use cases, such as content-based routing. For the example described here, the simpler “Map My Data” pattern is sufficient.
The first step is to drag the SOAP connection to the source part of the integration.
Second, the SAP connection has to be dragged to the target part and the SAP method (RFC, BAPI or IDOC) needs to be specified. Because it is required to read data from an SAP table, the suitable module is an RFC called RFC_READ_TABLE. The integration is querying the SAP table for new records with an updated status value of “Modified”. An example might look like this:
The data retrieved from SAP needs to be synchronized to SFDC, which can be achieved by using the enrichment call functionality provided by ICS. Enrichment calls are normally used to do additional calls within the integration in order to enrich either the integrations request or response payload with further data. In this case, the feature is used for writing data to SFDC.
The Salesforce connection needs to be placed on the response of the integration flow:
Afterwards the operation type and the business object to change must be defined. CRUD update is the correct operation and Order the correct object type:
The remaining steps are identical to the first integration. Set business identifiers for tracking, save the integration, and activate it:
If the SAP table contains a record with status value “Modified”, this record is read from SAP and used for updating the respective order in SFDC:
The use case described in this article shows that integrating on-premise with cloud applications is not that hard. While we used a simplified scenario here, in the complete scenario, the most complex thing would be the data mapping of a full SFDC to SAP order and vice versa. The intention of this article was to demonstrate the key capabilities of Oracle ICS:
- Rich connectivity capabilities
- Various options for defining integrations
- Easily enabled on-premise to cloud integration using the connectivity agent
Positioned as an iPaaS, Oracle ICS provides rich options for defining integrations between systems, as demonstrated while describing the basic implementation steps. Basically, there are various alternatives for implementing the integration between SAP and SFDC.
It is impressive to see how ICS has evolved over the last months and also how already available features will improve and grow in the coming months. Here Oracle has a clear, consistent vision. Through the Oracle marketplace as well as contributions from partners, the platform will become even more valuable for customers in the future.
About the Authors
Sven Bernhardt is a leading SOA/BPM architect and works as a solution architect for OPITZ CONSULTING Deutschland GmbH, a German Oracle Platinum Partner. In his role, he follows his passion for designing and building future-oriented, robust enterprise applications based on pioneering technologies. Sven is involved in diverse, large SOA and BPM implementations, dealing with challenges in the areas of business process automation and enterprise application integration. He also has significant experience as an SOA/BPM coach, trainer, developer, and architect. He develops and prepares implementation best practices and showcases and is responsible for knowledge building with respect to different Oracle technologies in the middleware area. Sven is an Oracle ACE and a frequent speaker at numerous IT conferences.
Alexander Däubler works as a Managing Consultant for OPITZ CONSULTING Deutschland GmbH and is based in Munich. He has long-term experience in SOA and BPM and has been involved in various integration projects as both developer and architect. Currently, he concentrates on supporting customers that want to move their integrations from on-premise to the cloud.
Cornelia Spanner has worked as an SOA developer for OPITZ CONSULTING Deutschland GmbH since 2012. She focuses on implementing and designing SOA-based integrations and architectures. Currently she is working on an integration scenario for a major automotive supplier in Germany. She specializes in everything concerning the cloud business.