This discussion is archived
10 Replies Latest reply: Dec 17, 2012 4:15 AM by Ganesh.. RSS

The WL server causes the use of CPU to reach to 100%

Ferez Newbie
Currently Being Moderated
Hi

I am using WebLogic Server Version 10.3.4.0. I got 3 applications deployed in this server, two of which developed by ADF technology and the other one by GWT. I start the server with 10GB of RAM assigned to it and after a while (sometimes one day, some other times 5 hours) the weblogic server causes the use of the CPU to reach to 100% and as a result the server sort of doesn't response anymore. The number of users is not very large, maybe 10~20 users for each application. I have no idea what is the cause of this, configuration of wl server or one of the applications. How could I find out? Do you have any idea?

Anyway, my system info is as follows:
CPU: Intel(R) Xeon(R) CPU 5160 @ 3.00GHz with 4 cores
RAM: 32GB
OS: Red Hat Enterprise Linux Server release 5.7, x86_64, 2.6.18

Thanks,
Ferez
  • 1. Re: The WL server causes the use of CPU to reach to 100%
    Kalyan Pasupuleti-Oracle Expert
    Currently Being Moderated
    Hi,


    1. Get the top output and find the PID associated with your userID that started WLS that is using up the CPU.

    2. Take several thread dumps of the WebLogic Server via kill -3 <PID>

    3. Convert the PID number from the first step to a hex value.

    (The JVM for Linux implements Java threads as native threads, which results in each thread being a separate Linux process.)

    4. Search the thread dumps for the corresponding value after "nid=" that match the hex number from the previous step.

    This will give you the thread that is causing the High CPU issue.

    5. Determine what exactly happening that particular class.


    Regards,
    Kal
  • 2. Re: The WL server causes the use of CPU to reach to 100%
    Pierluigi Vernetto Pro
    Currently Being Moderated
    mmm you might find it simpler to use a profiling tool - even JConsole, better still yourkit - to identify hot methods
  • 3. Re: The WL server causes the use of CPU to reach to 100%
    147443 Newbie
    Currently Being Moderated
    Also to add to the list :

    - Your sever may have 10G RAM but you need to find out how much memory is dedicated to each managed server.
    ps -ef |grep -i weblogic

    - Please check managed server log and find out how often Java Garbage collection is run

    - What Java vendor you are using - JRockIt, Sun or ....

    ps -ef |grep -i java
    which java
    java -version
  • 4. Re: The WL server causes the use of CPU to reach to 100%
    Ganesh.. Explorer
    Currently Being Moderated
    Hi,
    You can follow this link
    http://gskblogs.blogspot.in/2012/09/thread-analysis-in-weblogic-on-linux.html
    It will give you the exact lines of code in which the cpu utilization is high.
  • 5. Re: The WL server causes the use of CPU to reach to 100%
    Ferez Newbie
    Currently Being Moderated
    Thank you all for the replies, they were very helpful and I actually used every single one of them to find out the following:

    Part of the "ps –ef | java" command’s output shows that I am using jrockit-jdk1.6.0_20-R28.1.0-4.0.1 and -Xms10240m, -Xmx10240m options are set.

    Using "top –H –p 3889" command (3889 is the process id obtained from the previous "ps –ef | java" command) shows that 4 "GC Worker Thread" process is run for each CPU core and they take 100% of the CPU time.

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    3895 root 25 0 13.1g 11g 21m R 99.9 38.1 15:54.95 (GC Worker Thre
    3896 root 25 0 13.1g 11g 21m R 99.9 38.1 15:50.55 (GC Worker Thre
    3893 root 25 0 13.1g 11g 21m R 98.6 38.1 15:42.09 (GC Worker Thre
    3894 root 19 0 13.1g 11g 21m R 98.6 38.1 15:43.10 (GC Worker Thre
    3899 root 20 4 13.1g 11g 21m S 1.3 38.1 7:07.59 (VM Periodic Ta
    3889 root 18 0 13.1g 11g 21m S 0.0 38.1 0:00.00 java
    3890 root 16 0 13.1g 11g 21m S 0.0 38.1 0:04.96 java
    3891 root 16 0 13.1g 11g 21m S 0.0 38.1 0:00.12 (Signal Handler
    3892 root 15 0 13.1g 11g 21m S 0.0 38.1 0:17.36 (OC Main Thread


    I used JRockit Mission Control to see what happens. It turns out that after a while the 10GB memory gets occupied, GC starts to work to make room for the app but it could free up memory just about 1GB and the memory gets occupied soon and GC comes into play again and this process gets repeated again and again and it keeps the CPU usage near 100%. What is wrong? Is it because of memory leaks? How could I find out which application (and which part in the application) causes this? I am stuck and I really appreciate your help.

    The following is part of Thread Stack Dump of the server:

    This page displays the current stacks for each thread.

    ===== FULL THREAD DUMP ===============

    Sun Nov 11 12:27:49 2012

    Oracle JRockit(R) R28.1.0-123-138454-1.6.0_20-20101014-1350-linux-x86_64

    "Main Thread" id=1 idx=0x4 tid=3890 prio=5 alive, waiting, native_blocked

    -- Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x22985f770[fat lock]

    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)

    at java/lang/Object.wait(J)V(Native Method)

    at java/lang/Object.wait(Object.java:485)

    at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:979)

    ^-- Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x22985f770[fat lock]

    at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:488)

    at weblogic/Server.main(Server.java:71)

    at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)

    -- end of trace

    "(Signal Handler)" id=2 idx=0x8 tid=3891 prio=5 alive, native_blocked, daemon

    "(OC Main Thread)" id=3 idx=0xc tid=3892 prio=5 alive, native_blocked, daemon

    "(GC Worker Thread 1)" id=? idx=0x10 tid=3893 prio=5 alive, daemon

    "(GC Worker Thread 2)" id=? idx=0x14 tid=3894 prio=5 alive, daemon

    "(GC Worker Thread 3)" id=? idx=0x18 tid=3895 prio=5 alive, daemon

    "(GC Worker Thread 4)" id=? idx=0x1c tid=3896 prio=5 alive, daemon

    "(Code Generation Thread 1)" id=4 idx=0x20 tid=3897 prio=5 alive, native_waiting, daemon

    "(Code Optimization Thread 1)" id=5 idx=0x24 tid=3898 prio=5 alive, native_waiting, daemon

    "(VM Periodic Task)" id=6 idx=0x28 tid=3899 prio=10 alive, in native, daemon

    "Finalizer" id=7 idx=0x2c tid=3900 prio=8 alive, native_blocked, daemon

    at jrockit/memory/Finalizer.waitForFinalizees(J[Ljava/lang/Object;)I(Native Method)
             
                    at jrockit/memory/Finalizer.access$700(Finalizer.java:12)
             
                    at jrockit/memory/Finalizer$4.run(Finalizer.java:189)
             
                    at java/lang/Thread.run(Thread.java:619)
             
                    at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
             
                    -- end of trace
             
                "Reference Handler" id=8 idx=0x30 tid=3901 prio=10 alive, native_blocked, daemon
             
                    at java/lang/ref/Reference.waitForActivatedQueue(J)Ljava/lang/ref/Reference;(Native Method)
             
                    at java/lang/ref/Reference.access$100(Reference.java:11)
             
                    at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:82)
             
                    at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
             
                    -- end of trace
             
                "(Sensor Event Thread)" id=9 idx=0x34 tid=3902 prio=5 alive, native_blocked, daemon
             
                "VM JFR Buffer Thread" id=10 idx=0x38 tid=3903 prio=5 alive, in native, daemon
             
                "Timer-0" id=13 idx=0x3c tid=3906 prio=5 alive, waiting, native_blocked, daemon
             
                    -- Waiting for notification on: java/util/TaskQueue@0x221261da8[fat lock]

    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)

    at java/lang/Object.wait(J)V(Native Method)

    at java/lang/Object.wait(Object.java:485)

    at java/util/TimerThread.mainLoop(Timer.java:483)

    ^-- Lock released while waiting: java/util/TaskQueue@0x221261da8[fat lock]

    at java/util/TimerThread.run(Timer.java:462)

    at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)

    -- end of trace

    "Timer-1" id=14 idx=0x40 tid=3907 prio=5 alive, waiting, native_blocked, daemon

    -- Waiting for notification on: java/util/TaskQueue@0x221262160[fat lock]

    at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)

    at java/lang/Object.wait(J)V(Native Method)

    at java/util/TimerThread.mainLoop(Timer.java:509)

    ^-- Lock released while waiting: java/util/TaskQueue@0x221262160[fat lock]

    at java/util/TimerThread.run(Timer.java:462)

    at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)

    -- end of trace

    "[STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" id=15 idx=0x44 tid=3908 prio=1 alive, native_blocked, daemon

    at jrockit/net/SocketNativeIO.readBytesPinned(Ljava/io/FileDescriptor;[BIII)I(Native Method)
             
                    at jrockit/net/SocketNativeIO.socketRead(SocketNativeIO.java:32)[inlined]

    at java/net/SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I(SocketInputStream.java)[inlined]

    at java/net/SocketInputStream.read(SocketInputStream.java:129)[optimized]

    at oracle/net/nt/MetricsEnabledInputStream.read(TcpNTAdapter.java:718)[optimized]

    at oracle/net/ns/Packet.receive(Packet.java:295)[optimized]

    at oracle/net/ns/DataPacket.receive(DataPacket.java:106)

    at oracle/net/ns/NetInputStream.getNextPacket(NetInputStream.java:317)[optimized]

    at oracle/net/ns/NetInputStream.read(NetInputStream.java:262)[optimized]

    at oracle/net/ns/NetInputStream.read(NetInputStream.java:187)[inlined]

    at oracle/net/ns/NetInputStream.read(NetInputStream.java:104)[optimized]

    at oracle/jdbc/driver/T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:126)[inlined]

    at oracle/jdbc/driver/T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:82)[optimized]

    at oracle/jdbc/driver/T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1177)[optimized]

    at oracle/jdbc/driver/T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1153)[inlined]

    at oracle/jdbc/driver/T4CTTIfun.receive(T4CTTIfun.java:312)[optimized]

    at oracle/jdbc/driver/T4CTTIfun.doRPC(T4CTTIfun.java:204)[optimized]

    at oracle/jdbc/driver/T4C8Oall.doOALL(T4C8Oall.java:540)[inlined]

    at oracle/jdbc/driver/T4CPreparedStatement.doOall8(T4CPreparedStatement.java:217)[inlined]

    at oracle/jdbc/driver/T4CPreparedStatement.fetch(T4CPreparedStatement.java:1168)[optimized]

    at oracle/jdbc/driver/OracleResultSetImpl.close_or_fetch_from_next(OracleResultSetImpl.java:628)[inlined]

    at oracle/jdbc/driver/OracleResultSetImpl.next(OracleResultSetImpl.java:490)[optimized]

    ^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x285c6cfe0[recursive]

    ^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x285c6cfe0[thin lock]

    at weblogic/jdbc/wrapper/ResultSet_oracle_jdbc_driver_OracleResultSetImpl.next()Z(Unknown Source)[optimized]

    at oracle/jbo/server/QueryCollection.hasNextInResultSet(QueryCollection.java:4611)[inlined]

    at oracle/jbo/server/ViewObjectImpl.hasNextForCollection(ViewObjectImpl.java:6899)[optimized]

    at oracle/jbo/server/QueryCollection.hasNext(QueryCollection.java:4579)[inlined]

    at oracle/jbo/server/QueryCollection.populateRow(QueryCollection.java:3553)[optimized]

    at oracle/jbo/server/QueryCollection.fetch(QueryCollection.java:3387)[optimized]

    ^-- Holding lock: oracle/jbo/JboSyncLock@0x2ca5f7460[recursive]

    at oracle/jbo/server/QueryCollection.get(QueryCollection.java:2188)[optimized]

    at oracle/jbo/server/ViewRowSetImpl.getRow(ViewRowSetImpl.java:5016)[optimized]

    at oracle/jbo/server/ViewRowSetImpl.getRow(ViewRowSetImpl.java:3242)

    at oracle/jbo/server/ViewObjectImpl.activateCurrentRow(ViewObjectImpl.java:18786)

    at oracle/jbo/server/ViewRowSetIteratorImpl.activateIteratorState(ViewRowSetIteratorImpl.java:3947)[optimized]

    ^-- Holding lock: oracle/jbo/JboSyncLock@0x2ca5f7460[recursive]

    at oracle/jbo/server/ViewRowSetImpl.activateIteratorState(ViewRowSetImpl.java:7235)

    at oracle/jbo/server/ViewRowSetImpl.doCreateAndInitRow(ViewRowSetImpl.java:2465)

    ^-- Holding lock: oracle/jbo/JboSyncLock@0x2ca5f7460[thin lock]

    at oracle/jbo/server/ViewRowSetImpl.createRow(ViewRowSetImpl.java:2455)

    at oracle/jbo/server/ViewObjectImpl.createRow(ViewObjectImpl.java:10631)

    at org/nict/oms11g/view/backingbeans/exit/ReturnCartableBean.processRow(ReturnCartableBean.java:435)

    at org/nict/oms11g/view/backingbeans/exit/ReturnCartableBean.notUsed(ReturnCartableBean.java:102)

    at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)

    at jrockit/vm/Reflect.invokeMethod(Ljava/lang/Object;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
             
                    at sun/reflect/NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
             
                    at sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             
                    at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[optimized]

    at java/lang/reflect/Method.invoke(Method.java:597)[optimized]

    at com/sun/el/parser/AstValue.invoke(Lcom/sun/el/lang/EvaluationContext;[Ljava/lang/Class;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
             
                    at com/sun/el/MethodExpressionImpl.invoke(Ljavax/el/ELContext;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
             
                    at org/apache/myfaces/trinidadinternal/taglib/util/MethodExpressionMethodBinding.invoke(MethodExpressionMethodBinding.java:53)
             
                    at org/apache/myfaces/trinidad/component/UIXComponentBase.broadcastToMethodBinding(UIXComponentBase.java:1256)
             
                    at org/apache/myfaces/trinidad/component/UIXCommand.broadcast(UIXCommand.java:183)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent$1.run(ContextSwitchingComponent.java:92)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent._processPhase(ContextSwitchingComponent.java:361)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent.broadcast(ContextSwitchingComponent.java:96)
             
                    at oracle/adf/view/rich/component/fragment/UIXInclude.broadcast(UIXInclude.java:102)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent$1.run(ContextSwitchingComponent.java:92)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent._processPhase(ContextSwitchingComponent.java:361)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent.broadcast(ContextSwitchingComponent.java:96)
             
                    at oracle/adf/view/rich/component/fragment/UIXInclude.broadcast(UIXInclude.java:96)
             
                    at oracle/adf/view/rich/component/fragment/UIXRegion.broadcast(UIXRegion.java:148)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent$1.run(ContextSwitchingComponent.java:92)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent._processPhase(ContextSwitchingComponent.java:361)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent.broadcast(ContextSwitchingComponent.java:96)
             
                    at oracle/adf/view/rich/component/fragment/UIXInclude.broadcast(UIXInclude.java:102)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent$1.run(ContextSwitchingComponent.java:92)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent._processPhase(ContextSwitchingComponent.java:361)
             
                    at oracle/adf/view/rich/component/fragment/ContextSwitchingComponent.broadcast(ContextSwitchingComponent.java:96)
             
                    at oracle/adf/view/rich/component/fragment/UIXInclude.broadcast(UIXInclude.java:96)
             
                    at oracle/adfinternal/view/faces/lifecycle/LifecycleImpl.broadcastEvents(LifecycleImpl.java:879)
             
                    at oracle/adfinternal/view/faces/lifecycle/LifecycleImpl._executePhase(LifecycleImpl.java:312)[optimized]

    at oracle/adfinternal/view/faces/lifecycle/LifecycleImpl.execute(LifecycleImpl.java:185)

    at javax/faces/webapp/FacesServlet.service(FacesServlet.java:265)

    at weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)

    at weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)[inlined]

    at weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:300)[optimized]

    at weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26)[optimized]

    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

    at oracle/adf/model/servlet/ADFBindingFilter.doFilter(ADFBindingFilter.java:205)[optimized]

    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

    at oracle/adfinternal/view/faces/webapp/rich/RegistrationFilter.doFilter(RegistrationFilter.java:106)

    at org/apache/myfaces/trinidadinternal/webapp/TrinidadFilterImpl$FilterListChain.doFilter(TrinidadFilterImpl.java:446)

    at oracle/adfinternal/view/faces/activedata/AdsFilter.doFilter(AdsFilter.java:60)

    at org/apache/myfaces/trinidadinternal/webapp/TrinidadFilterImpl$FilterListChain.doFilter(TrinidadFilterImpl.java:446)

    at org/apache/myfaces/trinidadinternal/webapp/TrinidadFilterImpl._doFilterImpl(TrinidadFilterImpl.java:271)

    at org/apache/myfaces/trinidadinternal/webapp/TrinidadFilterImpl.doFilter(TrinidadFilterImpl.java:177)

    at org/apache/myfaces/trinidad/webapp/TrinidadFilter.doFilter(TrinidadFilter.java:92)

    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

    at oracle/jheadstart/controller/jsf/AuthenticationFilter.doFilter(AuthenticationFilter.java:282)

    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

    at oracle/security/jps/ee/http/JpsAbsFilter$1.run(JpsAbsFilter.java:111)

    at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)[inlined]

    at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:313)[inlined]

    at oracle/security/jps/ee/util/JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:413)[inlined]

    at oracle/security/jps/ee/http/JpsAbsFilter.runJaasMode(JpsAbsFilter.java:94)[inlined]

    at oracle/security/jps/ee/http/JpsAbsFilter.doFilter(JpsAbsFilter.java:161)[optimized]

    at oracle/security/jps/ee/http/JpsFilter.doFilter(JpsFilter.java:71)

    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

    at oracle/dms/servlet/DMSServletFilter.doFilter(DMSServletFilter.java:136)[optimized]

    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

    at weblogic/servlet/internal/RequestEventsFilter.doFilter(RequestEventsFilter.java:27)

    at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[inlined]

    at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3715)[optimized]

    at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3681)

    at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)[inlined]

    at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:120)[optimized]

    at weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2277)

    at weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:2183)

    at weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1454)[optimized]

    at weblogic/work/ExecuteThread.execute(ExecuteThread.java:207)[optimized]

    at weblogic/work/ExecuteThread.run(ExecuteThread.java:176)

    at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)

    -- end of trace

    ...
  • 6. Re: The WL server causes the use of CPU to reach to 100%
    Ganesh.. Explorer
    Currently Being Moderated
    Hi,
    The stacktrace shows that the thread 3908 is stuck waiting for the reply from the database server.
    A couple of things :
    1. Check the database and network performance.
    2. Which GC strategy are you using?
    You can use Jrockit memory leak detector to check any memory leaks in your code.
  • 7. Re: The WL server causes the use of CPU to reach to 100%
    Ferez Newbie
    Currently Being Moderated
    Hi

    1- It seems that the database and network performance is fine.

    2- Here is my VM and GC related info as is shown in the WLS console

    --->Java Virtual Machine Memory Utilization Statistics

    Heap Size Current: 10737418240
    Heap Free Current: 2967018664
    Heap Free Percent: 27
    Heap Size Max: 10737418240
    Total Physical Memory: 33752182784

    --->Garbage Collection Statistics

    GC Algorithm: Dynamic GC, current strategy: Generational Parallel Mark & Sweep, generational=true, sweep=parallel, mark=parallel
    Total GC Count: 6
    Last GC Start: 12/11/12 10:22:54 AM IRST
    Last GC End: 12/11/12 10:22:55 AM IRST
    Total GC Time: 1349
    Handles Compaction: true
    Concurrent: false
    Generational: true
    Incremental: false
    Parallel: true

    --->Java Virtual Machine Processor Utilization Statistics

    Total Number Of Threads: 66
    Number Of Daemon Threads: 60
    Number Of Processors: 4
    All Processors Average Load: 0.25
    JVM Processor Load: 0.25
    --------------------------------------------------------

    Going through thread 3908 stack trace shows that line 435 in ReturnCartableBean.java in my application is sort of responsible of this. Am I right? I guess it from the following section in the dump:

    at org/nict/oms11g/view/backingbeans/exit/ReturnCartableBean.processRow(ReturnCartableBean.java:435)

    This line in my application is:
    oracle.jbo.Row newRow = myViewObject.createRow();

    What is wrong with this statement? Is this problematic? It seems that it works fine at first but it causes problems after a while.

    Cheers,
    Ferez
  • 8. Re: The WL server causes the use of CPU to reach to 100%
    Ganesh.. Explorer
    Currently Being Moderated
    Hi,

    at oracle/jbo/server/ViewObjectImpl.createRow(ViewObjectImpl.java:10631)

    at org/nict/oms11g/view/backingbeans/exit/ReturnCartableBean.processRow(ReturnCartableBean.java:435)

    You are right buddy, this is the line where the thread is getting stuck. And there is nothing wrong with the statement you gave.
    If you follow the stacktrace, it is waiting for T4CPreparedStatement.fetch over network.
    In general case scenario, ViewObject.createRow should not cause stuck threads. I will recommend you to check for application performance tuning, mainly AppModule tuning. Check this link :
    http://docs.oracle.com/cd/E15523_01/web.1111/b31974/bcampool.htm
  • 9. Re: The WL server causes the use of CPU to reach to 100%
    Ferez Newbie
    Currently Being Moderated
    Hi

    Thank you very much Ganesh, I am going to check the AM tuning as you suggested but one more question. How could you get the fact that T4CPreparedStatement.fetch is the cause of problem? I mean when I follow the stacktrace to the top (starting from ViewObjectImpl.createRow) I could see lines such as:
    ^-- Holding lock: oracle/jbo/JboSyncLock@0x2ca5f7460[thin lock]
    or this one
    ^-- Holding lock: oracle/jdbc/driver/T4CConnection@0x285c6cfe0[thin lock]

    but for me there is nothing wrong with this line
    at oracle/jdbc/driver/T4CPreparedStatement.fetch(T4CPreparedStatement.java:1168)[optimized]

    How do you know what is the cause of problem?

    Regards,
    Ferez
  • 10. Re: The WL server causes the use of CPU to reach to 100%
    Ganesh.. Explorer
    Currently Being Moderated
    Hi,

    From the stacktrace it is visible that the thread is stuck in createRow() method. This method is trying to fetch some data from DB with T4CPreparedStatement.fetch() this call. Going deeper, you can say the thread is stuck while fetching some data which is not something very often. So this means, there is seriously something wrong with either the connection or the tuning of application or database.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points