This discussion is archived
1 Reply Latest reply: Nov 16, 2012 9:37 AM by 899771 RSS

Oracle BPM 11.1.1.5 Performance issues

899771 Newbie
Currently Being Moderated
Hi,

I have 2 Node Cluster of Oracle SOA 11.1.1.5 installed. Have a separate cluster for SOA/BPM, BAM, OSB, WSM. Here are the OS and Java versions

Java Vendor: HP
Java Version: 1.6.0.14
OS: HP-UX
OS Version: B.11.31

Running into an issue where Oracle BPM is performing very slow the task forms takes forever to come up and performing any actions takes too long to proceed som time it keeps timing out. I have over 100 SOA Process running plus some BPM Process but BMP Workspace application is running too slow as have customized tasks forms. But the same works a bit better in other instance which is non-clustered. Any idea what can be done on the server level to get around the performance issues. So far have modified Audit Levels and reduced soa-infra bpm log have the following settings in setDomainEnv.sh for USER_MEM_ARGS=-Xms2000m -Xmx6000m -XX:PermSize=1000m

But not help. Any idea what else to look into to get around these bpm performance issues. Here is what I have in setSOADomainEnv.sh

# 8395254: add -da:org.apache.xmlbeans... in EXTRA_JAVA_PROPERTIES
EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES} -da:org.apache.xmlbeans..."

XENGINE_DIR="${SOA_ORACLE_HOME}/soa/thirdparty/edifecs/XEngine"
DEFAULT_MEM_ARGS="-Xms512m -Xmx1024m"
PORT_MEM_ARGS="-Xms768m -Xmx1536m"

if [ "${JAVA_VENDOR}" != "Oracle" ] ; then
  DEFAULT_MEM_ARGS="${DEFAULT_MEM_ARGS} -XX:PermSize=128m -XX:MaxPermSize=512m"
  PORT_MEM_ARGS="${PORT_MEM_ARGS} -XX:PermSize=256m -XX:MaxPermSize=512m"
fi

#========================================================
# setup LD_LIBRARY_PATH if directory is present...
#========================================================
if [ -d ${XENGINE_DIR}/bin ]; then
   LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${XENGINE_DIR}/bin"
   export LD_LIBRARY_PATH
fi

#========================================================
# setup platform specific environment variables
#========================================================
case ${PLATFORM_TYPE} in
  #-----------------------------------------------------
  # AIX
  #-----------------------------------------------------
  AIX)

    if [ -d ${XENGINE_DIR}/bin ]; then
       LIBPATH="${LIBPATH}:${XENGINE_DIR}/bin"
       export LIBPATH
    fi

    USER_MEM_ARGS=${PORT_MEM_ARGS}
    export USER_MEM_ARGS

    # Fix for 7828060
    POST_CLASSPATH=${POST_CLASSPATH}:${SOA_ORACLE_HOME}/soa/modules/soa-ibm-addon.jar

    # Fix for 7520915 and 8264518 and 8305217
    EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES} -Djavax.xml.datatype.DatatypeFactory=org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl -Djava.endorsed.dirs=${SOA_ORACLE_HOME}/bam/modules/org.apache.xalan_2.7.1"
    export EXTRA_JAVA_PROPERTIES

    ;;

  #-----------------------------------------------------
  # HPUX
  #-----------------------------------------------------
  HP-UX)

    if [ -d ${XENGINE_DIR}/bin ]; then
       SHLIB_PATH="${SHLIB_PATH}:${XENGINE_DIR}/bin"
       export SHLIB_PATH
    fi

    LD_LIBRARY_PATH="${XENGINE_DIR}/bin:${LD_LIBRARY_PATH}"
    export LD_LIBRARY_PATH

    USER_MEM_ARGS="-d64 ${PORT_MEM_ARGS}"
    export USER_MEM_ARGS

    ;;
And here is what I have in setDomainEnv.sh
XMS_SUN_64BIT="256"
export XMS_SUN_64BIT
XMS_SUN_32BIT="256"
export XMS_SUN_32BIT
XMX_SUN_64BIT="512"
export XMX_SUN_64BIT
XMX_SUN_32BIT="512"
export XMX_SUN_32BIT
XMS_JROCKIT_64BIT="256"
export XMS_JROCKIT_64BIT
XMS_JROCKIT_32BIT="256"
export XMS_JROCKIT_32BIT
XMX_JROCKIT_64BIT="512"
export XMX_JROCKIT_64BIT
XMX_JROCKIT_32BIT="512"
export XMX_JROCKIT_32BIT


if [ "${JAVA_VENDOR}" = "Sun" ] ; then
     WLS_MEM_ARGS_64BIT="-Xms256m -Xmx512m"
     export WLS_MEM_ARGS_64BIT
     WLS_MEM_ARGS_32BIT="-Xms256m -Xmx512m"
     export WLS_MEM_ARGS_32BIT
else
     WLS_MEM_ARGS_64BIT="-Xms512m -Xmx512m"
     export WLS_MEM_ARGS_64BIT
     WLS_MEM_ARGS_32BIT="-Xms512m -Xmx512m"
     export WLS_MEM_ARGS_32BIT
fi

if [ "${JAVA_VENDOR}" = "Oracle" ] ; then
     CUSTOM_MEM_ARGS_64BIT="-Xms${XMS_JROCKIT_64BIT}m -Xmx${XMX_JROCKIT_64BIT}m"
     export CUSTOM_MEM_ARGS_64BIT
     CUSTOM_MEM_ARGS_32BIT="-Xms${XMS_JROCKIT_32BIT}m -Xmx${XMX_JROCKIT_32BIT}m"
     export CUSTOM_MEM_ARGS_32BIT
else
     CUSTOM_MEM_ARGS_64BIT="-Xms${XMS_SUN_64BIT}m -Xmx${XMX_SUN_64BIT}m"
     export CUSTOM_MEM_ARGS_64BIT
     CUSTOM_MEM_ARGS_32BIT="-Xms${XMS_SUN_32BIT}m -Xmx${XMX_SUN_32BIT}m"
     export CUSTOM_MEM_ARGS_32BIT
fi


MEM_ARGS_64BIT="${CUSTOM_MEM_ARGS_64BIT}"
export MEM_ARGS_64BIT

MEM_ARGS_32BIT="${CUSTOM_MEM_ARGS_32BIT}"
export MEM_ARGS_32BIT

if [ "${JAVA_USE_64BIT}" = "true" ] ; then
     MEM_ARGS="${MEM_ARGS_64BIT}"
     export MEM_ARGS
else
     MEM_ARGS="${MEM_ARGS_32BIT}"
     export MEM_ARGS
fi

MEM_PERM_SIZE_64BIT="-XX:PermSize=128m"
export MEM_PERM_SIZE_64BIT

MEM_PERM_SIZE_32BIT="-XX:PermSize=128m"
export MEM_PERM_SIZE_32BIT

if [ "${JAVA_USE_64BIT}" = "true" ] ; then
     MEM_PERM_SIZE="${MEM_PERM_SIZE_64BIT}"
     export MEM_PERM_SIZE
else
     MEM_PERM_SIZE="${MEM_PERM_SIZE_32BIT}"
     export MEM_PERM_SIZE
fi

MEM_MAX_PERM_SIZE_64BIT="-XX:MaxPermSize=512m"
export MEM_MAX_PERM_SIZE_64BIT

MEM_MAX_PERM_SIZE_32BIT="-XX:MaxPermSize=512m"
export MEM_MAX_PERM_SIZE_32BIT

if [ "${JAVA_USE_64BIT}" = "true" ] ; then
     MEM_MAX_PERM_SIZE="${MEM_MAX_PERM_SIZE_64BIT}"
     export MEM_MAX_PERM_SIZE
else
     MEM_MAX_PERM_SIZE="${MEM_MAX_PERM_SIZE_32BIT}"
     export MEM_MAX_PERM_SIZE
fi

if [ "${JAVA_VENDOR}" = "Sun" ] ; then
     if [ "${PRODUCTION_MODE}" = "" ] ; then
          MEM_DEV_ARGS="-XX:CompileThreshold=8000 ${MEM_PERM_SIZE} "
          export MEM_DEV_ARGS
     fi
fi

# Had to have a separate test here BECAUSE of immediate variable expansion on windows

if [ "${JAVA_VENDOR}" = "Sun" ] ; then
     MEM_ARGS="${MEM_ARGS} ${MEM_DEV_ARGS} ${MEM_MAX_PERM_SIZE}"
     export MEM_ARGS
fi

if [ "${JAVA_VENDOR}" = "HP" ] ; then
     MEM_ARGS="${MEM_ARGS} ${MEM_MAX_PERM_SIZE}"
     export MEM_ARGS
fi

if [ "${JAVA_VENDOR}" = "Apple" ] ; then
     MEM_ARGS="${MEM_ARGS} ${MEM_MAX_PERM_SIZE}"
     export MEM_ARGS
fi

if [ "${debugFlag}" = "true" ] ; then
     JAVA_OPTIONS="${JAVA_OPTIONS} -da:org.apache.xmlbeans... "
     export JAVA_OPTIONS
fi

export USER_MEM_ARGS="-Xms4g -Xmx6g -XX:PermSize=2g -XX:+UseParallelGC -XX:+UseParallelOldGC"
Here is the output of the top command
Load averages: 0.08, 0.06, 0.06
315 processes: 205 sleeping, 110 running
Cpu states:
CPU   LOAD   USER   NICE    SYS   IDLE  BLOCK  SWAIT   INTR   SSYS
 0    0.06   4.0%   0.2%   1.0%  94.8%   0.0%   0.0%   0.0%   0.0%
 2    0.07   3.6%   5.0%   0.2%  91.2%   0.0%   0.0%   0.0%   0.0%
 4    0.07   2.0%   0.2%   0.0%  97.8%   0.0%   0.0%   0.0%   0.0%
 6    0.07   2.4%   0.2%   0.8%  96.6%   0.0%   0.0%   0.0%   0.0%
 8    0.09   1.2%   0.2%  10.9%  87.7%   0.0%   0.0%   0.0%   0.0%
10    0.12   3.0%   0.0%  11.1%  85.9%   0.0%   0.0%   0.0%   0.0%
12    0.08   3.0%   0.2%   6.6%  90.3%   0.0%   0.0%   0.0%   0.0%
14    0.09   4.2%   1.2%   0.8%  93.8%   0.0%   0.0%   0.0%   0.0%
---   ----  -----  -----  -----  -----  -----  -----  -----  -----
avg   0.08   3.0%   1.0%   3.8%  92.2%   0.0%   0.0%   0.0%   0.0%

System Page Size: 4Kbytes
Memory: 38663044K (38379156K) real, 149420048K (148978096K) virtual, 26349848K f
ree  Page# 1/63

CPU TTY  PID USERNAME PRI NI   SIZE    RES STATE    TIME %WCPU  %CPU COMMAND
 4   ?  3926 root     152 20   213M 70040K run    694:47 10.05 10.04 cimprovagt
 4   ?  6855 user1  152 20  7704M  1125M run      4:36  9.31  9.29 java
 0   ?  6126 user2  152 20  2790M  1863M run     22:57  4.16  4.15 java
Here is the Memory on the box

Memory: 98132 MB (95.83 GB)


Thanks
  • 1. Re: Oracle BPM 11.1.1.5 Performance issues
    899771 Newbie
    Currently Being Moderated
    After changing JVM settigns for soa cluster it's a bit better but still slow so wondering what other tweaks can be done on the JVM side. Here is what is for the SOA Cluster

    USER_MEM_ARGS="-server -Xms12928m -Xmx12928m -XX:PermSize=3072m -Xmn3232m -XX:+SXTElimination -XX:+UseParallelGC -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:-TraceClassLoading -XX:-TraceClassUnloading"


    We are running multiple instances on the same boxes and the total RAM on the machine is 95 GB on each box which is being shared across 4 cluster environments. For now other 3 cluster environments have the SOA Cluster JVM setting as

    USER_MEM_ARGS="-server -Xms4096m -Xmx4096m -XX:PermSize=1024m -Xmn1152m -XX:+SXTElimination -XX:+UseParallelGC -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:-TraceClassLoading -XX:-TraceClassUnloading"


    Any help on what else I can tweak or set in JVM to get a better performance.


    Thanks

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points