1 Reply Latest reply: Feb 29, 2012 4:21 PM by 431897 RSS

    Hang on OpsDacRead? ODP.NET version incompatibility?

    866991
      I'm having issues with a thread in our program hitting OpsDacRead() and hanging. It is happening with a number of queries, one simply a 'select * from gv$database'. Some queries are running just fine. Some will work if their parameters are set to allow a small result set, but if they go longer than 30 seconds or so will never return. (Or at least not for 5 hours, the longest I've waited before cancelling one of the queries).

      It started after the product I am working on upgraded the Oracle Client 10.2.0.4. Previously 10.2.0.2.20 was in use, and the queries all worked back then. Even at present, I am only seeing this problem at a particular site, where the only unique feature is that they are using a Oracle 11.1.0.7 database, more recent than any of our other clients. They were working fine before the upgrade, though. At the same time as the upgrade from 10.2.0.2.20, the server running our program was upgraded from Windows Server 2003 x86 to Server 2008 x64. Our code has not changed in the areas surrounding the Oracle calls, except that the proxy making the call is using the newer Oracle.DataAccess.dll, and is now 64-bit.

      I have tried installing Oracle clients 11.1.0.7 and 11.2 (x64, and later x86) on the server running our product at the client site, but none of the versions worked any better than 10.2.0.4 had. I finally installed 10.2.0.2, since it had worked for us before, and compiled one of our internal-use query-running apps to x86. That allowed that specific app to work, but when I tried to duplicate that with the Oracle proxy in our main product it still had issues with hanging on OpsDacRead.

      I am consistently seeing this series of commands in the trace files when we encounter the hang:

      TIME:2011/06/04-22:21:54:136 TID:2618 OpsSqlPrepare2(): SQL: <query text>
      TIME:2011/06/04-22:21:54:136 TID:2618 (ENTRY) OpsPrmAllocOpoPrmCtx()
      TIME:2011/06/04-22:21:54:136 TID:2618 (EXIT) OpsPrmAllocOpoPrmCtx(): RetCode=0 Line=71
      TIME:2011/06/04-22:21:54:136 TID:2618 (ENTRY) OpsPrmFillOpoPrmCtx()
      TIME:2011/06/04-22:21:54:136 TID:2618 (EXIT) OpsPrmFillOpoPrmCtx(): RetCode=0 Line=303
      TIME:2011/06/04-22:21:54:136 TID:2618 (EXIT) OpsSqlPrepare2(): RetCode=0 Line=429
      TIME:2011/06/04-22:21:54:151 TID:2618 (ENTRY) OpsErrAllocCtx(): (47ff330)=0
      TIME:2011/06/04-22:21:54:151 TID:2618 (EXIT) OpsErrAllocCtx(): RetCode=0 Line=195 (47ff330)=0
      TIME:2011/06/04-22:21:54:151 TID:2618 (ENTRY) OpsSqlExecuteReader(): (47ff330)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OpsMetAddRef(): (56a2d0)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OpsMetAddRef(): RefCount=1 Line=88 (56a2d0)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OpsSqlExecuteReader(): RetCode=0 Line=534
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OpsConAddRef(): (47ff330)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OpsConAddRef(): RefCount=3 RetCode=0 Line=2353 (47ff330)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OpsErrAllocCtx(): (47ff330)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OpsErrAllocCtx(): RetCode=0 Line=195 (47ff330)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OpsMetAddRef(): (56a2d0)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OpsMetAddRef(): RefCount=2 Line=88 (56a2d0)=0
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OpsDacAllocValCtx()
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OpsDacAllocValCtx(): RetCode=0 Line=299
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (EXIT) OracleDataReader::GetOrdinal()
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OracleDataReader::Read()
      TIME:2011/06/04-22:21:54:198 TID:2618 (ENTRY) OpsDacRead(): (47ff330)=0

      At which point, the trace file ends. I had it set to use a separate trace file for each thread. The thread itself does not actually exit at this point, according to procmon, it is just left alone.

      I tried taking Wireshark captures on the server in question, and we are receiving a response from Oracle. However, it seems to be an incomplete response, cutting off after one of the data packets - I am guessing because the Reader never reads anything, and never sends any signal back to Oracle that it is still getting the data, and to keep sending more. Just a guess, though.

      Simplified, this is the code we are using:
      rdr = query.ExecuteReader (CommandBehavior.SingleResult);

      using (rdr)
      {
      string field;
      int ord;

      {
      int ord1;
      int ord2;

      field = string.Empty;
      ord = 0;

      try
      {
      ord1 = rdr.GetOrdinal (field = "FIELD1");
      ord2 = rdr.GetOrdinal (field = "FIELD2");
      }
      catch (Exception e)
      {
      throw CustomException();
      }

      try
      {
      while (rdr.Read ())
      {
      //Stuff done with the returned data here
      }
      }
      catch (Exception e)
      {
      throw CustomException();
      }


      Has anyone seen an issue similar to this, with a program hanging at OpsDacRead? Any ideas on where to investigate next would be much appreciated.

      Thanks.