0 Replies Latest reply: Aug 20, 2007 9:47 AM by 666705 RSS

    weblogic server crashes

    666705
      Hi..My production server crashes oftenly. Here is the thread dump.Can someone help me in analysing it?
      "ExecuteThread: '5' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x00132eb0 nid=0x42 in Object.wait() [0xdd9ff000..0xdd9ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9ef8970> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe9ef8970> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '4' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x00132d08 nid=0x41 in Object.wait() [0xddaff000..0xddaffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9ef89f0> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe9ef89f0> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '3' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x00154228 nid=0x40 in Object.wait() [0xddbff000..0xddbffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9ef8a70> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe9ef8a70> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '2' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x00122c50 nid=0x3f in Object.wait() [0xddcff000..0xddcffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9ef8af0> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe9ef8af0> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '1' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x005159c0 nid=0x3e in Object.wait() [0xdddff000..0xdddffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9ef8b70> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe9ef8b70> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '0' for queue: 'JMS.TimerClientPool'" daemon prio=5 tid=0x00a13f18 nid=0x3d in Object.wait() [0xddeff000..0xddeffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9ef8bf0> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe9ef8bf0> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0x00856728 nid=0x3c waiting for monitor entry [0xddfff000..0xddfffc28]
           at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)
           - waiting to lock <0xe9efad00> (a java.lang.String)
           at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
           at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

      "ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0x00d5ac28 nid=0x3b in Object.wait() [0xde0ff000..0xde0ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9ef8cf0> (a [Lweblogic.socket.PosixSocketInfo$FdStruct;)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:95)
           - locked <0xe9ef8cf0> (a [Lweblogic.socket.PosixSocketInfo$FdStruct;)
           - locked <0xe9efad00> (a java.lang.String)
           at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
           at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

      "ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0x0034f960 nid=0x3a waiting for monitor entry [0xde1ff000..0xde1ffc28]
           at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)
           - waiting to lock <0xe9efad00> (a java.lang.String)
           at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
           at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

      "SSLListenThread.Default" prio=5 tid=0x003e86f8 nid=0x39 runnable [0xde2ff000..0xde2ffc28]
           at java.net.PlainSocketImpl.socketAccept(Native Method)
           at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
           - locked <0xe9f00e98> (a java.net.PlainSocketImpl)
           at java.net.ServerSocket.implAccept(ServerSocket.java:448)
           at javax.net.ssl.impl.SSLServerSocketImpl.accept(Unknown Source)
           at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:735)
           at weblogic.t3.srvr.ListenThread.run(ListenThread.java:301)

      "ListenThread.Default" prio=5 tid=0x00402740 nid=0x38 runnable [0xde3ff000..0xde3ffc28]
           at java.net.PlainSocketImpl.socketAccept(Native Method)
           at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
           - locked <0xe9f00f90> (a java.net.PlainSocketImpl)
           at java.net.ServerSocket.implAccept(ServerSocket.java:448)
           at java.net.ServerSocket.accept(ServerSocket.java:419)
           at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:26)
           at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:735)
           at weblogic.t3.srvr.ListenThread.run(ListenThread.java:301)

      "Thread-7" daemon prio=5 tid=0x00152088 nid=0x37 in Object.wait() [0xde4ff000..0xde4ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9e8c4a8> (a java.util.TaskQueue)
           at java.util.TimerThread.mainLoop(Timer.java:429)
           - locked <0xe9e8c4a8> (a java.util.TaskQueue)
           at java.util.TimerThread.run(Timer.java:382)

      "ExecuteThread: '0' for queue: 'JMS.TimerTreePool'" daemon prio=5 tid=0x007fcf50 nid=0x36 in Object.wait() [0xdeeff000..0xdeeffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9e54370> (a weblogic.jms.backend.BETimerTree)
           at weblogic.jms.backend.BETimerTree.execute(BETimerTree.java:146)
           - locked <0xe9e54370> (a weblogic.jms.backend.BETimerTree)
           at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

      "Thread-6" prio=5 tid=0x004a4ef0 nid=0x35 in Object.wait() [0xdf2ff000..0xdf2ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9df51d0> (a java.util.TaskQueue)
           at java.util.TimerThread.mainLoop(Timer.java:429)
           - locked <0xe9df51d0> (a java.util.TaskQueue)
           at java.util.TimerThread.run(Timer.java:382)

      "weblogic.health.CoreHealthMonitor" daemon prio=5 tid=0x00f0b4d8 nid=0x34 waiting on condition [0xdf4ff000..0xdf4ffc28]
           at java.lang.Thread.sleep(Native Method)
           at weblogic.t3.srvr.CoreHealthMonitorThread.run(CoreHealthMonitorThread.java:127)

      "Thread-5" prio=5 tid=0x00a74580 nid=0x33 in Object.wait() [0xdf5ff000..0xdf5ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe9ad2c40> (a java.util.TaskQueue)
           at java.util.TimerThread.mainLoop(Timer.java:429)
           - locked <0xe9ad2c40> (a java.util.TaskQueue)
           at java.util.TimerThread.run(Timer.java:382)

      "VDE Transaction Processor Thread" prio=2 tid=0x001a7f40 nid=0x31 in Object.wait() [0xdf6ff000..0xdf6ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f0130> (a com.octetstring.vde.backend.standard.TransactionProcessor)
           at java.lang.Object.wait(Object.java:429)
           at com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:365)
           - locked <0xe99f0130> (a com.octetstring.vde.backend.standard.TransactionProcessor)
           at com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:212)

      "ExecuteThread: '2' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x00d36f80 nid=0x30 in Object.wait() [0xdf7ff000..0xdf7ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f01b0> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f01b0> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '1' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x00d36cc8 nid=0x2f in Object.wait() [0xdf8ff000..0xdf8ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f0230> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f0230> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '0' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x00d36b20 nid=0x2e in Object.wait() [0xdf9ff000..0xdf9ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f02b0> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f02b0> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '1' for queue: 'weblogic.admin.HTTP'" daemon prio=5 tid=0x00994ab8 nid=0x2d in Object.wait() [0xdfaff000..0xdfaffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f0330> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f0330> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '0' for queue: 'weblogic.admin.HTTP'" daemon prio=5 tid=0x00d36978 nid=0x2c in Object.wait() [0xdfbff000..0xdfbffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f03b0> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f03b0> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "weblogic.security.SpinnerRandomSource" daemon prio=5 tid=0x00f3bf20 nid=0x2b in Object.wait() [0xdfcff000..0xdfcffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f0478> (a java.lang.Object)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.security.SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:60)
           - locked <0xe99f0478> (a java.lang.Object)
           at java.lang.Thread.run(Thread.java:534)

      "weblogic.time.TimeEventGenerator" daemon prio=9 tid=0x0077f270 nid=0x2a in Object.wait() [0xdfdff000..0xdfdffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f04e8> (a weblogic.time.common.internal.TimeTable)
           at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:272)
           - locked <0xe99f04e8> (a weblogic.time.common.internal.TimeTable)
           at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:118)
           at java.lang.Thread.run(Thread.java:534)

      "ExecuteThread: '4' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x008873a8 nid=0x29 in Object.wait() [0xdfeff000..0xdfeffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f0560> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f0560> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '3' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x008867b0 nid=0x28 in Object.wait() [0xdffff000..0xdffffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f05e0> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f05e0> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '2' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x00885bb8 nid=0x27 in Object.wait() [0xe00ff000..0xe00ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f0660> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f0660> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '1' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x008849b8 nid=0x26 in Object.wait() [0xe01ff000..0xe01ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f06e0> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f06e0> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '0' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x00883dc0 nid=0x25 in Object.wait() [0xe02ff000..0xe02ffc28]
           at java.lang.Object.wait(Native Method)
           - waiting on <0xe99f0760> (a weblogic.kernel.ExecuteThread)
           at java.lang.Object.wait(Object.java:429)
           at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:153)
           - locked <0xe99f0760> (a weblogic.kernel.ExecuteThread)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:172)

      "ExecuteThread: '24' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x009340a8 nid=0x24 runnable [0xe03fe000..0xe03ffc28]
           at java.net.SocketInputStream.socketRead0(Native Method)
           at java.net.SocketInputStream.read(SocketInputStream.java:129)
           at oracle.net.ns.Packet.receive(Unknown Source)
           at oracle.net.ns.DataPacket.receive(Unknown Source)
           at oracle.net.ano.CryptoDataPacket.receive(CryptoDataPacket)
           at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:971)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:941)
           at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:432)
           at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:181)
           at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows(T4CPreparedStatement.java:543)
           at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:8661)
           - locked <0xeac764e0> (a oracle.jdbc.driver.T4CPreparedStatement)
           - locked <0xeac6f578> (a oracle.jdbc.driver.T4CConnection)
           at weblogic.jdbc.wrapper.PreparedStatement.executeBatch(PreparedStatement.java:137)
           at net.sf.hibernate.impl.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:50)
           at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:105)
           at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2103)
           at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2062)
           at net.sf.hibernate.impl.SessionImpl.flush(SessionImpl.java:2005)
           at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:57)
           at com.marsh.itg.csi.ui.HibernateTile.perform(HibernateTile.java:67)
           at org.apache.struts.tiles.TilesRequestProcessor.processTilesDefinition(TilesRequestProcessor.java:239)
           at org.apache.struts.tiles.TilesRequestProcessor.processForwardConfig(TilesRequestProcessor.java:303)
           at org.apache.struts.action.RequestProcessor.processActionForward(RequestProcessor.java:401)
           at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:279)
           at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)
           at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:502)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
           at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:996)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:315)
           at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6452)
           at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
           at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
           at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3661)
           at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2630)
           at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

      "ExecuteThread: '23' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x009334b0 nid=0x23 runnable [0xe04fe000..0xe04ffc28]
           at java.net.SocketInputStream.socketRead0(Native Method)
           at java.net.SocketInputStream.read(SocketInputStream.java:129)
           at oracle.net.ns.Packet.receive(Unknown Source)
           at oracle.net.ns.DataPacket.receive(Unknown Source)
           at oracle.net.ano.CryptoDataPacket.receive(CryptoDataPacket)
           at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:971)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:941)
           at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:432)
           at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:181)
           at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows(T4CPreparedStatement.java:543)
           at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:8661)
           - locked <0xeabbb740> (a oracle.jdbc.driver.T4CPreparedStatement)
           - locked <0xeab7f888> (a oracle.jdbc.driver.T4CConnection)
           at weblogic.jdbc.wrapper.PreparedStatement.executeBatch(PreparedStatement.java:137)
           at net.sf.hibernate.impl.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:50)
           at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:105)
           at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2103)
           at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2062)
           at net.sf.hibernate.impl.SessionImpl.flush(SessionImpl.java:2005)
           at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:57)
           at com.marsh.itg.csi.hibernate.HibernateRunner.run(HibernateRunner.java:39)
           at com.marsh.itg.csi.ui.HibernateTile.run(HibernateTile.java:111)
           at com.marsh.itg.csi.ui.plan.AnswerForm.reset(AnswerForm.java:134)
           at org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:811)
           at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:254)
           at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)
           at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:520)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
           at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:996)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:315)
           at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6452)
           at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
           at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
           at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3661)
           at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2630)
           at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

      "ExecuteThread: '22' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x00b0c230 nid=0x22 runnable [0xe05fe000..0xe05ffc28]
           at java.net.SocketInputStream.socketRead0(Native Method)
           at java.net.SocketInputStream.read(SocketInputStream.java:129)
           at oracle.net.ns.Packet.receive(Unknown Source)
           at oracle.net.ns.DataPacket.receive(Unknown Source)
           at oracle.net.ano.CryptoDataPacket.receive(CryptoDataPacket)
           at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:971)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:941)
           at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:432)
           at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:181)
           at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows(T4CPreparedStatement.java:543)
           at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:8661)
           - locked <0xeaced490> (a oracle.jdbc.driver.T4CPreparedStatement)
           - locked <0xeab10f08> (a oracle.jdbc.driver.T4CConnection)
           at weblogic.jdbc.wrapper.PreparedStatement.executeBatch(PreparedStatement.java:137)
           at net.sf.hibernate.impl.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:50)
           at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:105)
           at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2103)
           at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2062)
           at net.sf.hibernate.impl.SessionImpl.flush(SessionImpl.java:2005)
           at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:57)
           at com.marsh.itg.csi.ui.HibernateTile.perform(HibernateTile.java:67)
           at org.apache.struts.tiles.TilesRequestProcessor.processTilesDefinition(TilesRequestProcessor.java:239)
           at org.apache.struts.tiles.TilesRequestProcessor.processForwardConfig(TilesRequestProcessor.java:303)
           at org.apache.struts.action.RequestProcessor.processActionForward(RequestProcessor.java:401)
           at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:279)
           at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)
           at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:502)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
           at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:996)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:315)
           at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6452)
           at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
           at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
           at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3661)
           at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2630)
           at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

      "ExecuteThread: '21' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x00932268 nid=0x21 runnable [0xe06fe000..0xe06ffc28]
           at java.net.SocketInputStream.socketRead0(Native Method)
           at java.net.SocketInputStream.read(SocketInputStream.java:129)
           at oracle.net.ns.Packet.receive(Unknown Source)
           at oracle.net.ns.DataPacket.receive(Unknown Source)
           at oracle.net.ano.CryptoDataPacket.receive(CryptoDataPacket)
           at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:971)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:941)
           at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:432)
           at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:181)
           at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows(T4CPreparedStatement.java:543)
           at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:8694)
           - locked <0xeaa31870> (a oracle.jdbc.driver.T4CPreparedStatement)
           - locked <0xea9ab0a8> (a oracle.jdbc.driver.T4CConnection)
           at weblogic.jdbc.wrapper.PreparedStatement.executeBatch(PreparedStatement.java:137)
           at net.sf.hibernate.impl.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:50)
           at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:105)
           at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2103)
           at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2061)
           at net.sf.hibernate.impl.SessionImpl.flush(SessionImpl.java:2005)
           at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:57)
           at com.marsh.itg.csi.ui.HibernateAction.submit(HibernateAction.java:70)
           at com.marsh.itg.csi.ui.CancellableAction.execute(CancellableAction.java:28)
           at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:480)
           at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
           at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)
           at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:520)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
           at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:996)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:315)
           at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6452)
           at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
           at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
           at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3661)
           at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2630)
           at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
           at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

      "ExecuteThread: '20' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x00930ed0 nid=0x20 runnable [0xe07fe000..0xe07ffc28]
           at java.net.SocketInputStream.socketRead0(Native Method)
           at java.net.SocketInputStream.read(SocketInputStream.java:129)
           at oracle.net.ns.Packet.receive(Unknown Source)
           at oracle.net.ns.DataPacket.receive(Unknown Source)
           at oracle.net.ano.CryptoDataPacket.receive(CryptoDataPacket)
           at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.net.ns.NetInputStream.read(Unknown Source)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:971)
           at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:941)
           at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:432)
           at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:181)
           at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows(T4CPreparedStatement.java:543)
           at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:8694)
           - locked <0xea9129d8> (a oracle.jdbc.driver.T4CPreparedStatement)
           - locked <0xea8fc828> (a oracle.jdbc.driver.T4CConnection)
           at weblogic.jdbc.wrapper.PreparedStatement.executeBatch(PreparedStatement.java:137)
           at net.sf.hibernate.impl.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:50)
           at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:105)
           at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2103)
           at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2061)
           at net.sf.hibernate.impl.SessionImpl.flush(SessionImpl.java:2005)
           at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:57)
           at com.marsh.itg.csi.ui.HibernateAction.submit(HibernateAction.java:70)
           at com.marsh.itg.csi.ui.CancellableAction.execute(CancellableAction.java:28)
           at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:480)
           at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
           at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1420)
           at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:520)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
           at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:996)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:419)
           at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:315)
           at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6452)
           at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
           at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:118)
           at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3661)
           at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2