1  Oracle SOA Suite Basic tuning

This section has been divided in several subsections to identify each layer which participates in the entire stack, and will go through step by step to identify the best optimized settings.

The subsections are

 

  • Operating System/Storage /Network settings
  • Java Virtual Machine settings
  • Oracle WebLogic generic settings
  • Oracle SOA Suite 11g settings:
    • SOA Infrastructure engines
    • SOA Services

 

 

It is very important that individual performance analysis are as similar to each other as possible, to ensure that it is actually meaningful to compare and draw conclusions from the results.

Ensure that the environment is warmed up and has reached its steady state. Measurements and anlysis must be taken during the steady state.

Performance can differ markedly between cold and warm environments because of issues like the impact of optimizations that are done after a number of repetitions of a particular piece of code, the optimizations done by the database after a number of queries have been executed, the settling of objects in the various parts of the JVM heap over time.

 

Not only consider CPU and Memory % but:

  • Memory stats (inc GC)
  • Connection pool stats
  • JMS queue stats
  • Transaction stats
  • Data in and output

 

These areas are typically to look at in these types of environments:

 

  • CPU
  • Memory
  • Locks
  • Bandwidth
  • Storage
  • Resource pools

 

Always consider before implementing :

  • What am I changing
  • Has the bottleneck gone away?
  • Is there a new bottleneck?

 

Way of working:

  • Bottom up
    • Work from the CPU upwards
  • Treat SOA Suite as any other Java application at O/S level
    • Key suspects such as Garbage Collection and locking   

 

 

 

 

 

Improvement Areas : Performance

 

Files/directories recommendations

  • Writing logs to local storage will direct bottlenecks from network connections to shared storage.
  • BUT transaction logs and JMS persistence stores must remain in shared storage, so that failover can work

 

 

Java VM Settings

The JVM heap should be at least 6G for the SOA managed servers. Allow 50% overhead process memory from heap size. For 6G heap allow the java process 10G of memory. Monitor/tune heap size, TLA size etc. during load testing.

 

AdminServer tuning

The AdminServer plays an important role in the processing of SOA transaction flows. The following components need attention:

 

AdminServer JVM Arguments

For JVM, use the following:

  • Set the XgcPrio to pausetime
  • Set the transaction rollback blocking to true
  • Disable Managed Server notifications

 

 

Additional Fusion Middleware EM console and DMS settings:

  • Set the discovery cache age to 28800000
  • Set the discovery wait time to 30000
  • Set the cache results to true
  • Increase the dumpinterval to 10800 with a maximum of 75 MB
  • Enable Large Repository
  • Increase the collector configuration settings for DMS:
    • Use an interval of 300 secs with a remove cycle of 3
    • And use an interval of 120 secs with a remove cycle of 2, set this to default
  • Restrict the amount of displayed instance information to 12 hours

 

Implement SOA ping

Apply Patch 17571359 which provides a HTTP URL for load balancers that returns SOA server status in the format: http://[host]:[port]/soa-infra/services/isSoaServerReady. It will return response code 503 (Service Unavailable) when SOA Server is not ready to accept requests and 200(OK) when SOA Platform is running and accepting requests.

 

JDBC related settings

  • Connection Pool

       JDBC connection pool should be set so that it doesn’t need to grow after it is initialized. This can be done by:

    • Set initial capacity the same as maximum capacity.
    • Be aware that datasources always have enough free connections
    • Statement caching can eliminate potential performance impacts caused by repeated cursor creation and repeated statement parsing and creation. Statement caching also reduces the performance impact of communication between the application server and the database server
    • Disable unnecessary connection testing and profiling.

 

  • Large Payload handling

Disable wrap datatype in case of large SOA payloads being inserted through JCA.

 

SOA Suite transactional improvements

During a longer period of time, it becomes clear which transactions run longer than initially expected. One should always monitor long running transactions to detect possible problem areas.

Based on that analysis, the following needs attention:

 

SOA Database:                                             Distributed lock timeout

WebLogic domain wide transacations:     JTA – identify increasing amount of application errors and rollbacks

SOA Engine beans:                                      Increase Timeout

BPEL:                                                            Wait time for synchronous invocations

 

 

In-Flight processes

Make use of this option when using synchronous operations, but limit persistency.

  • InMemoryOptimisation (transient processes)
  • Reduce audit threshold
  • Reduce completion persist level
  • Increase audit trail size threshold
  • Increase large document threshold

 

SOA Suite BPEL engine

  • Minimize the amount of audit on transactions
  • Increase the dispatcher threads to a value ( after gaining performance statitistics)
  • Persist one-way invocations in a asynchronous manner

                    

SOA Suite Mediator engine

  • Prefer to use DVM lookups for XSL transformations ( this is to reduce I/O operations)
  • Use and tune deferred routing rule mechanism

 

 

Improvement areas: Stability

 

Instance Purging/archiving

Scripts are available for instance archiving and purging. A better option is to use of database partitions which can be managed by point in time to archive and drop instance data.

See http://docs.oracle.com/cd/E28280_01/admin.1111/e10226/soaadmin_partition.htm for details. Oracle DBA’s should be able to assist in moving to partitioned database from current non-partitioned one.

 

MDS update/refresh

  • Whenever there is a change in endpoint from end systems, the MDS has to be updated. End point info should be in config plan, which allows new values at the time of deployment.
  • If you change endpoint info in MDS post deployment, then you can clear MDS cache using EM from the MDSAppRuntime
  • MDS should be cleared on a regular basis – old metadata labels(> 3 months ) should be cleaned

 

Instance Patching

Instance patchingis useful when

  • A new version of process can be deployed and running instances can either continue on old version
  • Or migrate to new version either automatically (if versions are compatible) .

 

Zero Downtime Patching

Apply patch to Admin server first. If the patch does not contain WebLogic cluster fixes that would cause an inconsistency in the cluster the patch may be applied without shutting down the whole cluster.

Since OPatch works on the SOA and oracle_common directories in middleware home the cluster will also need at least two middleware home directories. Shutdown all the members using one of the middleware home directories, apply the patch(es) and then restart the server instances. Repeat those steps on the members using the other middleware home.

 

Oracle Service Bus Basic Tuning

This section has been divided in several subsections to identify each layer which participates in the entire stack, and will go through step by step to identify the best optimized settings. In fact it is the same approach as in the Oracle SOA Suite, although from a historical perspective, OSB in tighter integrated with WebLogic.

 

The subsections are:

  • Java Virtual Machine settings
  • Oracle WebLogic generic settings
  • Oracle Service Bus 11g settings:
    • Service Bus Infrastructure engines
    • OSB Proxy and Business Services

Key areas are:

  • JVM
  • JDBC
  • JMS
  • Caching /Coherence

 

Improvement areas: Performance Tuning

 

JVM tuning

Always keep in mind that OSB is typically focusing on throughput, short live objects so try to size the JVM not too small or too big. GC should not take longer than 1 sec.

Size of JVM is usually 4 to 6 GB.

 

Logging on WebLogic

Disable access logging for managed and AdminServer, leave that to Oracle HTTP Server.

 

JMS

Consider if persistency is needed for any JMS. If it’s not, disable it. This should be known by development team but operations team can advise and discuss.

On non-Exalogic hardware, use file based persistency.

 

Router Runtime Cache

To reduce the amount of time spent on compiling the proxy pipeline set this value to 80% of the amount of deployed proxy services.

 

 

Service Result caching

Use this options in certain OSB Business services, in combination with the underlying coherence framework.

Consider to use either in JVM coherence or configure coherence in its own JVM. For the last option you need more resources ( CPU, RAM, Storage)

 

Static data

Cache the static data that need to be configured in your application. You need to consider:

  • Amount of data need to be cached
  • Which caching api to use – recommendation is to use Coherence

   

 

JCA Adapter tuning

Consider every polling interval used ad some adapter (FTP, file, DB) if applicable

 

activationInstances

For parallel processing use this parameter. The most optimum value should be determined on the amount of messages per hour, usually this parameter is 6 or 8.

 

Monitoring and alerting

Keep this for the most services at global level, only for key services, define a separate monitoring approach.

 

For several proxy and business services which arre marked as critical, configure and use pipeline and / or SLA alerting

 

Improvement areas: Stability

 

Logging, monitoring, tracing

  • Log as minimum as necessary
  • Monitor important areas with an SLA alert
  • Never enable tracing in production

 

Purging alerts

OSB SLA and pipeline alerts need to be purged on a regular base. Create a custom WLDF archive for the alerts and use the following purging strategy:

 

Type

Retention

Purge time

Pipeline and SLA Alerts

14 days ( 336 hours )

Every day at 00:00