Skip navigation

Some of the old stuff from my previous whitepaper and some new insights about Oracle FMW on Exalogic. Lost to consider!



Exalogic as a converged engineered system has various layers to consider when we want to apply all the possiblle enhancements on every possible layer in the Oracle FMW Stack


In this whitepaper we will start with the following components:

  • WebLogic Server
  • Coherence
  • Database traffic ( JDBC)
  • Message Traffic ( HTTP, REST, SOAP,JMS)
  • Traffic Director







Exalogic Enhancement

Tickbox enabled

Set in WLS console


I/O Enhancement




Domain Env file


Exalogic Enhancement

# Enable Java Exalogic optimizations
-Xlargepages:exitOnFailure=false -Doracle.xdkjava.exalogic.optimization=true

Domain Env file


ListenAdress IPOIB

WLS AdminServer ListenAdress

WLS Console



  1. KernelMBean.AddWorkManagerThreadsByCpuCount mbean

FMW Console



Admin Channel

t3, http

WLS Console


Client Channels

t3, http

WLS Console



jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=sdp)(HOST=<>)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=<DB Service>)))



JDBC optimization

  • oracle.jdbc.enableJavaNetFastPath=true

FMW Console



-Dtangosol.coherence.wka1=(IPoIB adress localhost)

-Dtangosol.coherence.wka2=(IPoIB adress1)

-Dtangosol.coherence.localhost=(IPoIB adress2)

In-JVM coherence and separate Coherence cluster




FMW Console

SOA Infra

SOA BPEL Coherence Dehydration







Domain Env File

SOA Infra

SOA Coherence Optimizations






Domain Env File


Message Compression


WLS Conole


Message Compression


WLS Console


TLOG and persistency

JDBC Store ( Over IpoIB)

WLS Console


Lockless requests

  1. ServerMBean.useConcurrentQueueForRequestManager=true

FMW Console


JSP Factory caching




WebLogic Admin channel over Infiniband private net


The private InfiniBand fabric including WebLogic Clusters and Coherence clusters, requires a set of IP addresses for all WebLogic Managed Server-to-Administration Server traffic and for cluster communication. These IP addresses are associated with the BOND0 interface.


Within Exalogic it is possible to use a private vnet that can be reserved for administrative purposes such as accessing the AdminServer and its consoles, internal cluster communication and so on.

This can be easily achieved by setting all listenadresses to the IP that is bound to Infiniband

The same we do for all managed servers and their listen adresses

Trouble is, that we can’t access the AdminServer url over IPOIB, For this we create separate channels

Also for the managed servers, which have clients access we create channels to connect to the outside world 


There are several channels to identify:

  • Several channels for EoIB interfaces, such as:
    • AdminServer Network Channel
    • HTTP and T3 Network Channels

These are needed for internal resources like AdminServers or managed servers internal communications to communicate with the outside world.

You must create two network channels for the Administration Server. The network channels are necessary for routing HTTP traffic and T3 traffic (TCP-based protocol used by WebLogic Server) coming in from an external data center via Ethernet over InfiniBand (EoIB). You must create the following network channels:

HTTP Client channel for AdminServer

  1. Log in the WLS console and Click Lock & Edit in the Change Center.
  2. In the left pane of the Console, expand Environment, and then Servers.
  3. In the Servers tab, click AdminServer(admin).
  4. Select Protocols → Channels. Click New.
  5. Enter AdminHTTPClient as the name of the new network channel and select http as the protocol, then click Next.
  6. Enter the following information in the Network Channel Addressing page:
    • Listen address: < the EOIB IP adress >This address is the floating IP assigned to the Administration Server using the BOND<n> interface.
    • Listen port: 7001
  7. Click Next, and select Enabled on the Network Channel Properties page. and Click Finish.
  8. Activate these changes,


T3 Client channels

Repeat the same steps as above but choose T3 as a protocol. Repeat these steps also for all managed servers


When started, you see the several interfaces which are listened on

Sep 5, 2014 4:49:15 PM CEST> <Notice> <Server> <BEA-002613> <Channel "Default" is now listening on for protocols iiop, t3, ldap, snmp, http.>

<Sep 5, 2014 4:49:15 PM CEST> <Notice> <Server> <BEA-002613> <Channel "AdminChannel" is now listening on for protocols t3, http.>

And you still can use the normal AdminServer adress to logon, either hostname or Admin VIP configured.

When you configure Oracle Traffic Director, this should be configured using the IPoIB, in every other setup you choose the EoIB, but then you miss all IB enhancements.

  • Expand Farm_domain_name, SOA, and then right click soa-infra (server_name) (either one of the servers).
  • Click SOA Administration, and then BPEL Properties.
  • Select More BPEL Configuration properties.
  • Enter CacheEnabled for the property QualityOfService property.
  • Select Apply.
  • In the Domain Structure window, expand the Environment node.
  • Click Servers.
  • Click the name of the server (WLS_SOA1 or WLS_SOA2, which are represented as hyperlinks) in the Name column of the table.
  • The settings page for the selected server appears.
  • Click Lock & Edit.
  • Click the Server Start tab.
  • In the Arguments field, add the following for all SOA managed servers


-Xlargepages:exitOnFailure=false -Doracle.xdkjava.exalogic.optimization=true





Note that all the values are examples and can differ from your situation. Always measure, analyse, test and decide!

Using Embedded Coherence ( Several FMW products, SOA suite, IDM)


With optimizations in SOA Suite the process state can now reside in Oracle Coherence - which is an in-memory data grid. Coherence uses direct memory acces for replicating data.

We configure coherence using Unicast, therefor we should disable all multicast entries, located in the setDomainEnv

Futher more we should set all WKA adresses to use the IPoIB interface, by setting the virtual hostname that is set for the IP adress. This is typically the VIP adress used for SOA internal traffic ( attached to IB interface)




SOAHOST1-PRIV-V1 is the virtual host name that maps to the virtual IP where WLS_SOA1 is listening (in SOAHOST1), and so on.


Configuring Coherence Caches for Dehydrations

Dehydrations take place in the SOA repository database, but you can use Coherence to do offload this kind of work. If you configure the BPEL engine with the CacheEnabled property, the engine runs many fewer reads against database. Because many reads require locks and version checks, eliminating them improves BPEL engine performance. Configuring Oracle Coherence for dehydration requires the following steps:

Enabling the property

To enable the cache enabled property in SOA, follow theses steps:

  1. Log in to Oracle Enterprise Manager Fusion Middleware Control
  2. Expand Farm_domain_name, SOA, and then right click soa-infra (server_name) (either one of the servers).
  3. Click SOA Administration, and then BPEL Properties.
  4. Select More BPEL Configuration properties.
  5. Enter CacheEnabled for the property QualityOfService property.
  6. Select Apply.



Next, set the required server properties for using in-process coherence cache for dehydration.

To enable the cache enabled property in SOA:

  1. In the Domain Structure window, expand the Environment node.
  2. Click Servers.
  3. Click the name of the server (WLS_SOA1 or WLS_SOA2, which are represented as hyperlinks) in the Name column of the table.
  4. The settings page for the selected server appears.
  5. Click Lock & Edit.
  6. Click the Server Start tab.
  7. In the Arguments field, add the following for all SOA managed servers


-Xlargepages:exitOnFailure=false -Doracle.xdkjava.exalogic.optimization=true

Update SOA JVM settings

To optimize behavior with local storage caches and improve performance, update the SOA servers' start parameters to also include the following flags:

All tuning settings together in setDomainEnv or UserEnv file

Maybe itś obvious, but every setting you optimize must have a reason, so do not set all these setting just like that. Do it one by one with a group of parameters, and test it with it. The outcome of your analysis will give you a guideline of what is applicable for you and what is not. To be set in the setDomainEnv, or better use a separate parameter file and reference it to the setDomainEnv.

  • Java Optimizations:



       → Infiniband enabling! IMB driver mist be enabled for that. Only for Linux x86- 64 bit platforms.
-Dweblogic.transaction.blocking.rollback=true<location of your trust store>"


  • BPEL Optimizations. The main parameter is the bpel.cache.localStorage; set it to true and then tune the other paramters to your needs, like cache sizes and worker threads.



Note that all the values are examples and can differ from your situation. Always measure, analyse, test and decide!

Cluster replications

This applies only for stateful applications such as B2B Console, or SOA/BPM Composer

To enable session replication enhancements for the SOA_Cluster:

  1. Shut down the servers in the SOA_Cluster.
  2. To set replication ports for a managed server, such as WLS_SOA1:
    1. Under Domain Structure, click Environment and Servers. Click on the SOA managed server, Configuration tab, Cluster tab.
    2. In the Replication Ports field, enter a range of ports for configuring multiple replication channels. For example, replication channels for managed servers in SOA_Cluster can listen on ports starting from 8006 to 8009. To specify this range of ports, enter 8006-8009.


Create a custom network channel for each managed server in the cluster : 

Log on to the Admin Console In the Servers table, click the SOA managed server instance.

  1. Select Protocols, and then Channels, and click New.
  2. Listen Address: SOAHOST1-PRIV-V1 ( (IPoIB Adress )
  3. Listen port: 8006 Remove the default external listen port.
  4. Click Next, and in the Network Channel Properties page, select Enabled and Outbound Enabled, then click Finish.
  5. Under the Network Channels, select ReplicationChannel, the network channel you created for the WLS_SOA1 managed server.


  • In the Replication Channel field, ensure that ReplicationChannel is set as the name of the channel to be used for replication traffic.
  • In the Advanced section, select the Enable One Way RMI for Replication option., and click Save.
  • Activate the changes and restart the managed servers.
  • Add the system property to the start parameters for the WebLogic servers
  • Configure multiple services to share the same store if they will commonly be invoked within the same transaction.
  • Do no use the XA driver for the JDBC Store because the JMS infrastructure of WebLogic itselve is fully equipped for XA
  • JMS often executes direct database operations to invoke its store services within the same transaction,so using a JDBC data source with Logging Last Resource (LLR) might be a good option; it will do two-phase commit, in a single local transaction DB operation, this improves the overall transaction performance.


  • MessageCompressionOptions: Along with the option above, choose one of the compression option in. Note that compression is done by gzip, so you can choose for the best compression:
    • LZF


JMS also makes use of the domain wide Exalogic optimizations by using concurrent queues for pending messages. A buffer queue is used for incoming requests which increases throughput aby scheduling these requests and prevent locks. 


Other Considerations

These will discuss some other considerations you could think of, either by determining if it’s applicable to your situation, either it may come out of some load or stresstest scenario and analysis.

Hugepages and Transparent HugePages

Default in Exalogic bare metal compute nodes. Hugepages allocation can be good for applications which have to allocate large memory blocks.

In your Exalogic VM infstrastructure design, decide which VM will be used for such type of applications like WebCenter Portal, and if they need it. In some situation it can be necessary to reserve a computenode for these environments instead of doing dynamic allocation, and you might consider to use Transparent HugePages for these.

Oracle Traffic Director enhancements


In almost every Enterprise Deployment Guide from a Fusion Middleware Product, Oracle uses the Oracle WebTier as a controlling layer for you HTTP calls.

In Exalogic, a software loadbalancer is added, as you might know, called Oracle Traffic Director. Depending on the approach of how you design your Cloud infrastructure with one of these components

  • Network topology
  • Segregations of duties and and task rolled out over several departments, or done by one department overall? Who will take care of what?( Cloud Infrastructure team, Middleware Team, all by one DevOps team)
  • Other solutions than provided by Oracle ( 3rd party WebTiers, hardware loadbalancers)
  • Failover groups
  • Origin Servers
  • Various listeners (HTTP, TCP)

SOA Suite

The SOA servers listen on both EoIB and IPoIB interfaces for in and external traffic. Internal could be:

  • Cluster traffic
  • Internal apllication traffic ( RMI, T3 ) An example could be an configured route for mediator traffic, or business services to SOA composites.
  • External traffic for in and outbound services.


The default channel uses an IPoIB listen address. This is for optimized server-to-server as well as for Coherence dehydration and deployment optimizations.

Besides the IPoIB interfaces, the SOA Managed Servers also listen on EoIB virtual IP addresses so that they can be accessed externally for the following purposes:

  • RMI/JMX/T3 traffic
  • HTTP traffic used for remote deployment.
  • External JMS producers and consumers.
  • Can use separate WebLogic channels for T3 and HTTP for traffic isolation.



OSB can follow the same pattern as SOA Suite. OSB should be the central bus for routing, transforming and exchanging messagetaffic between different front and backend applications

OSB also uses coherence for result caching. We now use the in JVM mechanism, which means internal coherence libraries will be used.

There for, Coherence WKA addresses need to be configured in the server start parameters. These WKA addresses should be configured with the IPoIB address. These addresses should be accessible only internal within the OSB WebLogic domain by all managed and AdminServer.



1  Oracle SOA Suite Basic tuning

This section has been divided in several subsections to identify each layer which participates in the entire stack, and will go through step by step to identify the best optimized settings.

The subsections are


  • Operating System/Storage /Network settings
  • Java Virtual Machine settings
  • Oracle WebLogic generic settings
  • Oracle SOA Suite 11g settings:
    • SOA Infrastructure engines
    • SOA Services



It is very important that individual performance analysis are as similar to each other as possible, to ensure that it is actually meaningful to compare and draw conclusions from the results.

Ensure that the environment is warmed up and has reached its steady state. Measurements and anlysis must be taken during the steady state.

Performance can differ markedly between cold and warm environments because of issues like the impact of optimizations that are done after a number of repetitions of a particular piece of code, the optimizations done by the database after a number of queries have been executed, the settling of objects in the various parts of the JVM heap over time.


Not only consider CPU and Memory % but:

  • Memory stats (inc GC)
  • Connection pool stats
  • JMS queue stats
  • Transaction stats
  • Data in and output


These areas are typically to look at in these types of environments:


  • CPU
  • Memory
  • Locks
  • Bandwidth
  • Storage
  • Resource pools


Always consider before implementing :

  • What am I changing
  • Has the bottleneck gone away?
  • Is there a new bottleneck?


Way of working:

  • Bottom up
    • Work from the CPU upwards
  • Treat SOA Suite as any other Java application at O/S level
    • Key suspects such as Garbage Collection and locking   






Improvement Areas : Performance


Files/directories recommendations

  • Writing logs to local storage will direct bottlenecks from network connections to shared storage.
  • BUT transaction logs and JMS persistence stores must remain in shared storage, so that failover can work



Java VM Settings

The JVM heap should be at least 6G for the SOA managed servers. Allow 50% overhead process memory from heap size. For 6G heap allow the java process 10G of memory. Monitor/tune heap size, TLA size etc. during load testing.


AdminServer tuning

The AdminServer plays an important role in the processing of SOA transaction flows. The following components need attention:


AdminServer JVM Arguments

For JVM, use the following:

  • Set the XgcPrio to pausetime
  • Set the transaction rollback blocking to true
  • Disable Managed Server notifications



Additional Fusion Middleware EM console and DMS settings:

  • Set the discovery cache age to 28800000
  • Set the discovery wait time to 30000
  • Set the cache results to true
  • Increase the dumpinterval to 10800 with a maximum of 75 MB
  • Enable Large Repository
  • Increase the collector configuration settings for DMS:
    • Use an interval of 300 secs with a remove cycle of 3
    • And use an interval of 120 secs with a remove cycle of 2, set this to default
  • Restrict the amount of displayed instance information to 12 hours


Implement SOA ping

Apply Patch 17571359 which provides a HTTP URL for load balancers that returns SOA server status in the format: http://[host]:[port]/soa-infra/services/isSoaServerReady. It will return response code 503 (Service Unavailable) when SOA Server is not ready to accept requests and 200(OK) when SOA Platform is running and accepting requests.


JDBC related settings

  • Connection Pool

       JDBC connection pool should be set so that it doesn’t need to grow after it is initialized. This can be done by:

    • Set initial capacity the same as maximum capacity.
    • Be aware that datasources always have enough free connections
    • Statement caching can eliminate potential performance impacts caused by repeated cursor creation and repeated statement parsing and creation. Statement caching also reduces the performance impact of communication between the application server and the database server
    • Disable unnecessary connection testing and profiling.


  • Large Payload handling

Disable wrap datatype in case of large SOA payloads being inserted through JCA.


SOA Suite transactional improvements

During a longer period of time, it becomes clear which transactions run longer than initially expected. One should always monitor long running transactions to detect possible problem areas.

Based on that analysis, the following needs attention:


SOA Database:                                             Distributed lock timeout

WebLogic domain wide transacations:     JTA – identify increasing amount of application errors and rollbacks

SOA Engine beans:                                      Increase Timeout

BPEL:                                                            Wait time for synchronous invocations



In-Flight processes

Make use of this option when using synchronous operations, but limit persistency.

  • InMemoryOptimisation (transient processes)
  • Reduce audit threshold
  • Reduce completion persist level
  • Increase audit trail size threshold
  • Increase large document threshold


SOA Suite BPEL engine

  • Minimize the amount of audit on transactions
  • Increase the dispatcher threads to a value ( after gaining performance statitistics)
  • Persist one-way invocations in a asynchronous manner


SOA Suite Mediator engine

  • Prefer to use DVM lookups for XSL transformations ( this is to reduce I/O operations)
  • Use and tune deferred routing rule mechanism



Improvement areas: Stability


Instance Purging/archiving

Scripts are available for instance archiving and purging. A better option is to use of database partitions which can be managed by point in time to archive and drop instance data.

See for details. Oracle DBA’s should be able to assist in moving to partitioned database from current non-partitioned one.


MDS update/refresh

  • Whenever there is a change in endpoint from end systems, the MDS has to be updated. End point info should be in config plan, which allows new values at the time of deployment.
  • If you change endpoint info in MDS post deployment, then you can clear MDS cache using EM from the MDSAppRuntime
  • MDS should be cleared on a regular basis – old metadata labels(> 3 months ) should be cleaned


Instance Patching

Instance patchingis useful when

  • A new version of process can be deployed and running instances can either continue on old version
  • Or migrate to new version either automatically (if versions are compatible) .


Zero Downtime Patching

Apply patch to Admin server first. If the patch does not contain WebLogic cluster fixes that would cause an inconsistency in the cluster the patch may be applied without shutting down the whole cluster.

Since OPatch works on the SOA and oracle_common directories in middleware home the cluster will also need at least two middleware home directories. Shutdown all the members using one of the middleware home directories, apply the patch(es) and then restart the server instances. Repeat those steps on the members using the other middleware home.


Oracle Service Bus Basic Tuning

This section has been divided in several subsections to identify each layer which participates in the entire stack, and will go through step by step to identify the best optimized settings. In fact it is the same approach as in the Oracle SOA Suite, although from a historical perspective, OSB in tighter integrated with WebLogic.


The subsections are:

  • Java Virtual Machine settings
  • Oracle WebLogic generic settings
  • Oracle Service Bus 11g settings:
    • Service Bus Infrastructure engines
    • OSB Proxy and Business Services

Key areas are:

  • JVM
  • JDBC
  • JMS
  • Caching /Coherence


Improvement areas: Performance Tuning


JVM tuning

Always keep in mind that OSB is typically focusing on throughput, short live objects so try to size the JVM not too small or too big. GC should not take longer than 1 sec.

Size of JVM is usually 4 to 6 GB.


Logging on WebLogic

Disable access logging for managed and AdminServer, leave that to Oracle HTTP Server.



Consider if persistency is needed for any JMS. If it’s not, disable it. This should be known by development team but operations team can advise and discuss.

On non-Exalogic hardware, use file based persistency.


Router Runtime Cache

To reduce the amount of time spent on compiling the proxy pipeline set this value to 80% of the amount of deployed proxy services.



Service Result caching

Use this options in certain OSB Business services, in combination with the underlying coherence framework.

Consider to use either in JVM coherence or configure coherence in its own JVM. For the last option you need more resources ( CPU, RAM, Storage)


Static data

Cache the static data that need to be configured in your application. You need to consider:

  • Amount of data need to be cached
  • Which caching api to use – recommendation is to use Coherence



JCA Adapter tuning

Consider every polling interval used ad some adapter (FTP, file, DB) if applicable



For parallel processing use this parameter. The most optimum value should be determined on the amount of messages per hour, usually this parameter is 6 or 8.


Monitoring and alerting

Keep this for the most services at global level, only for key services, define a separate monitoring approach.


For several proxy and business services which arre marked as critical, configure and use pipeline and / or SLA alerting


Improvement areas: Stability


Logging, monitoring, tracing

  • Log as minimum as necessary
  • Monitor important areas with an SLA alert
  • Never enable tracing in production


Purging alerts

OSB SLA and pipeline alerts need to be purged on a regular base. Create a custom WLDF archive for the alerts and use the following purging strategy:




Purge time

Pipeline and SLA Alerts

14 days ( 336 hours )

Every day at 00:00