2 Replies Latest reply: Apr 17, 2012 8:04 AM by 931157 RSS

    Deadlock in Oracle.sql.ARRAY type

    931157
      Hi,
      We've come across the deadlock situation below when running multiple J2EE MDB instances that are trying to write to the DB:

      [deadlocked thread] [ACTIVE] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)':
      --------------------------------------------------------------------------------------------------
      Thread '[ACTIVE] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'' is waiting to acquire lock 'oracle.jdbc.driver.T4CConnection@90106ee' that is held by thread '[ACTIVE] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)''

      Stack trace:
      ------------
      oracle.sql.ARRAY.toBytes(ARRAY.java:673)
      oracle.jdbc.driver.OraclePreparedStatement.setArrayCritical(OraclePreparedStatement.java:5985)
      oracle.jdbc.driver.OraclePreparedStatement.setARRAYInternal(OraclePreparedStatement.java:5944)
      oracle.jdbc.driver.OraclePreparedStatement.setObjectCritical(OraclePreparedStatement.java:8782)
      oracle.jdbc.driver.OraclePreparedStatement.setObjectInternal(OraclePreparedStatement.java:8278)
      oracle.jdbc.driver.OraclePreparedStatement.setObject(OraclePreparedStatement.java:8868)
      oracle.jdbc.driver.OraclePreparedStatementWrapper.setObject(OraclePreparedStatementWrapper.java:240)
      weblogic.jdbc.wrapper.PreparedStatement.setObject(PreparedStatement.java:287)
      org.springframework.jdbc.core.StatementCreatorUtils.setValue(StatementCreatorUtils.java:356)
      org.springframework.jdbc.core.StatementCreatorUtils.setParameterValueInternal(StatementCreatorUtils.java:216)
      org.springframework.jdbc.core.StatementCreatorUtils.setParameterValue(StatementCreatorUtils.java:127)
      org.springframework.jdbc.core.PreparedStatementCreatorFactory$PreparedStatementCreatorImpl.setValues(PreparedStatementCreatorFactory.java:298)
      org.springframework.jdbc.object.BatchSqlUpdate$1.setValues(BatchSqlUpdate.java:192)
      org.springframework.jdbc.core.JdbcTemplate$4.doInPreparedStatement(JdbcTemplate.java:892)
      org.springframework.jdbc.core.JdbcTemplate$4.doInPreparedStatement(JdbcTemplate.java:1)
      org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:586)
      org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:614)
      org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:883)
      org.springframework.jdbc.object.BatchSqlUpdate.flush(BatchSqlUpdate.java:184)
      com.csfb.fao.rds.rfi.common.dao.storedprocs.SaveEarlyExceptionBatchStoredProc.execute(SaveEarlyExceptionBatchStoredProc.java:93)
      com.csfb.fao.rds.rfi.common.dao.EarlyExceptionDAOImpl.saveEarlyExceptionBatch(EarlyExceptionDAOImpl.java:34)
      com.csfb.fao.rds.rfi.application.rulesengine.RulesEngine.saveEarlyExceptions(RulesEngine.java:302)
      com.csfb.fao.rds.rfi.application.rulesengine.RulesEngine.executeRules(RulesEngine.java:209)
      com.csfb.fao.rds.rfi.application.rulesengine.RulesEngine.onMessage(RulesEngine.java:97)
      com.csfb.fao.rds.feeds.process.BaseWorkerMDB.onMessage(BaseWorkerMDB.java:518)
      weblogic.ejb.container.internal.MDListener.execute(MDListener.java:466)
      weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:371)
      weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:327)
      weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4547)
      weblogic.jms.client.JMSSession.execute(JMSSession.java:4233)
      weblogic.jms.client.JMSSession.executeMessage(JMSSession.java:3709)
      weblogic.jms.client.JMSSession.access$000(JMSSession.java:114)
      weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:5058)
      weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
      weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
      weblogic.work.ExecuteThread.run(ExecuteThread.java:173)


      [deadlocked thread] [ACTIVE] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)':
      --------------------------------------------------------------------------------------------------
      Thread '[ACTIVE] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'' is waiting to acquire lock 'oracle.jdbc.driver.T4CConnection@b48b568' that is held by thread '[ACTIVE] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)''

      Stack trace:
      ------------
      oracle.sql.ARRAY.toBytes(ARRAY.java:673)
      oracle.jdbc.driver.OraclePreparedStatement.setArrayCritical(OraclePreparedStatement.java:5985)
      oracle.jdbc.driver.OraclePreparedStatement.setARRAYInternal(OraclePreparedStatement.java:5944)
      oracle.jdbc.driver.OraclePreparedStatement.setObjectCritical(OraclePreparedStatement.java:8782)
      oracle.jdbc.driver.OraclePreparedStatement.setObjectInternal(OraclePreparedStatement.java:8278)
      oracle.jdbc.driver.OraclePreparedStatement.setObject(OraclePreparedStatement.java:8868)
      oracle.jdbc.driver.OraclePreparedStatementWrapper.setObject(OraclePreparedStatementWrapper.java:240)
      weblogic.jdbc.wrapper.PreparedStatement.setObject(PreparedStatement.java:287)
      org.springframework.jdbc.core.StatementCreatorUtils.setValue(StatementCreatorUtils.java:356)
      org.springframework.jdbc.core.StatementCreatorUtils.setParameterValueInternal(StatementCreatorUtils.java:216)
      org.springframework.jdbc.core.StatementCreatorUtils.setParameterValue(StatementCreatorUtils.java:127)
      org.springframework.jdbc.core.PreparedStatementCreatorFactory$PreparedStatementCreatorImpl.setValues(PreparedStatementCreatorFactory.java:298)
      org.springframework.jdbc.object.BatchSqlUpdate$1.setValues(BatchSqlUpdate.java:192)
      org.springframework.jdbc.core.JdbcTemplate$4.doInPreparedStatement(JdbcTemplate.java:892)
      org.springframework.jdbc.core.JdbcTemplate$4.doInPreparedStatement(JdbcTemplate.java:1)
      org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:586)
      org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:614)
      org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:883)
      org.springframework.jdbc.object.BatchSqlUpdate.flush(BatchSqlUpdate.java:184)
      com.csfb.fao.rds.rfi.common.dao.storedprocs.SaveEarlyExceptionBatchStoredProc.execute(SaveEarlyExceptionBatchStoredProc.java:93)
      com.csfb.fao.rds.rfi.common.dao.EarlyExceptionDAOImpl.saveEarlyExceptionBatch(EarlyExceptionDAOImpl.java:34)
      com.csfb.fao.rds.rfi.application.rulesengine.RulesEngine.saveEarlyExceptions(RulesEngine.java:302)
      com.csfb.fao.rds.rfi.application.rulesengine.RulesEngine.executeRules(RulesEngine.java:209)
      com.csfb.fao.rds.rfi.application.rulesengine.RulesEngine.onMessage(RulesEngine.java:97)
      com.csfb.fao.rds.feeds.process.BaseWorkerMDB.onMessage(BaseWorkerMDB.java:518)
      weblogic.ejb.container.internal.MDListener.execute(MDListener.java:466)
      weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:371)
      weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:327)
      weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4547)
      weblogic.jms.client.JMSSession.execute(JMSSession.java:4233)
      weblogic.jms.client.JMSSession.executeMessage(JMSSession.java:3709)
      weblogic.jms.client.JMSSession.access$000(JMSSession.java:114)
      weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:5058)
      weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
      weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
      weblogic.work.ExecuteThread.run(ExecuteThread.java:173)




      Looking at the ARRAY.toBytes() method:
      public byte[] toBytes()
      throws SQLException
      {
      synchronized (getInternalConnection())
      {
      return this.descriptor.toBytes(this, this.enableBuffering);
      }
      }

      ..., it synchronizes on the following method (getInternalConnection() -> getPhysicalConnection()):

      oracle.jdbc.internal.OracleConnection getPhysicalConnection()
      {
      if (this.physicalConnection == null)
      {
      try
      {
      this.physicalConnection = ((oracle.jdbc.internal.OracleConnection)new OracleDriver().defaultConnection());
      }
      catch (SQLException localSQLException)
      {
      }

      }

      return this.physicalConnection;
      }


      defaultConnection() does the following:
      public Connection defaultConnection()
      throws SQLException
      {
      if ((defaultConn == null) || (defaultConn.isClosed()))
      {
      synchronized (OracleDriver.class)
      {
      if ((defaultConn == null) || (defaultConn.isClosed()))
      {
      defaultConn = connect("jdbc:oracle:kprb:", new Properties());
      }
      }
      }

      return defaultConn;
      }

      So there's synchronizations on the connection instance and OracleDriver.class object... I can't see how this can deadlock. To get to the point of needing the lock on OracleDriver.class, the thread would already have the lock on the connection instance.... clearly I'm missing something.

      Thanks

      Edited by: 928154 on 17-Apr-2012 03:42
        • 1. Re: Deadlock in Oracle.sql.ARRAY type
          gimbal2
          Welcome to the forum. If you want help, at least try to think where to post a question and look for a forum that matches the topic. Lets examine what you have:

          - its Weblogic, so if you would ask a non-programming related question anywhere it would be in the Weblogic forum
          - HOWEVER, if you check the top of the stacktrace, you'll see that the problem stems from the JDBC driver, so a JDBC related forum would be a closer match

          For future reference, Weblogic specific questions should go here: https://forums.oracle.com/forums/category.jspa?categoryID=193
          and JDBC/OJDBC driver related questions should go here: Java Database Connectivity (JDBC)

          Final tip: use \
           tags to post code so it is readable.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
          • 2. Re: Deadlock in Oracle.sql.ARRAY type
            931157
            Thanks have reposted in jdbc fourm
            Deadlock in Oracle.sql.ARRAY type