7 Replies Latest reply: Apr 6, 2013 9:13 PM by 1000381 RSS

    Secondary is corrupt: the primary record contains a key that is not present

    1000381
      The following code (in Scala)
          withCursorInTransaction(queueDatabase) {
            case (cursor, txn) => cursor.getFirst(LockMode.RMW).map {
              case (key, pKey, value) =>
                val link = linkSerializer.entryToObject(value)
                // update it's state to "in progress"
                link.storageState match {
                  case LinkStorageState.QUEUED =>
                    val newLink = link.copy(storageState = LinkStorageState.IN_PROGRESS)
                    mainDatabase.put(txn,pKey,linkSerializer.objectToEntry(newLink))
                    Right(newLink)
                  case LinkStorageState.IN_PROGRESS =>
                    Left(AlreadyInProgress)
                }
            }.getOrElse(Left(NoRecordInDatabase))
      throws the exception:
      2013-04-01 20:50:57,906 ERROR  [BDBJEQueue] Error in transaction 
      com.sleepycat.je.SecondaryIntegrityException: (JE 5.0.58) Secondary is corrupt: the primary record contains a key that is not present in the secondary
           at com.sleepycat.je.SecondaryDatabase.deleteKey(SecondaryDatabase.java:938)
           at com.sleepycat.je.SecondaryDatabase.updateSecondary(SecondaryDatabase.java:901)
           at com.sleepycat.je.SecondaryTrigger.databaseUpdated(SecondaryTrigger.java:41)
           at com.sleepycat.je.Database.notifyTriggers(Database.java:2144)
           at com.sleepycat.je.Cursor.putNotify(Cursor.java:2136)
           at com.sleepycat.je.Cursor.putNoDups(Cursor.java:2052)
           at com.sleepycat.je.Cursor.putInternal(Cursor.java:2020)
           at com.sleepycat.je.Database.putInternal(Database.java:1324)
           at com.sleepycat.je.Database.put(Database.java:1194)
           at com.webspider.storage.bdbje.BDBJEQueue$$anonfun$pop$1$$anonfun$apply$1.apply(BDBJEQueue.scala:110)
      on the line:
      mainDatabase.put(txn,pKey,linkSerializer.objectToEntry(newLink))
      I added debug statements to verify the key content and got:

      2013-04-01 21:01:15,945 DEBUG [BDBJEQueue] Key: 19=>11D9FD73687474703A2F2F3132332E636F6D00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

      2013-04-01 21:01:15,964 DEBUG [BDBJEQueue] KEY: 19=>11D9FD73687474703A2F2F3132332E636F6D00

      where 19 is DatabaseEntry.getSize

      so the primary keys are the same, what may be wrong with secondary key?

      What I'm trying to do is to
      - get first record from secondary database
      - update content of the record in primary database, after the update this record should not appear in secondary database because the key is changed

      I assume that this is not allowed anymore - so if I need to modify the record in primary database, I can not do it having transaction on secondary database, and 'remove' record from secondary database at same time?

      P/S it's weird - I was registered on the forum since 2005 or so, why do I have only one post and registered just today?!
        • 1. Re: Secondary is corrupt: the primary record contains a key that is not present
          Bogdan Coman-Oracle
          I'm looking to understand what's happening and to provide you with a fix/workaround.
          I assume that this is not allowed anymore
          My understanding is that you are trying to upgrade BDB JE. Can you tell me what release you used before?
          P/S it's weird - I was registered on the forum since 2005 or so, why do I have only one post and registered just today?!
          This may be related to some change that took place in the forum accounts a few years ago when the point system got implemented. If you want to ask about it, I suggest you to try the Community Feedback and Suggestions forum: Community Feedback (No Product Questions)

          Bogdan
          • 2. Re: Secondary is corrupt: the primary record contains a key that is not present
            1000381
            No, I am not upgrading the DB from any previous version, it's new DB created during tests.

            I remember that previously it was possible to update the record in primary database within the transaction on secondary database, so that key in secondary DB is no longer exists.

            Like:

            - primary database contains record with field "state" set to A
            - there is the secondary database which contains only records with state field set to 'A'
            - open cursor on secondary database and take first record from it
            - get the record and update it's state to 'B'
            - update primary database with this record

            now the last step results in exception.
            • 3. Re: Secondary is corrupt: the primary record contains a key that is not present
              Bogdan Coman-Oracle
              Can you please answer the following questions:
              jdevelop wrote:
              The following code (in Scala)
              withCursorInTransaction(queueDatabase) {
              case (cursor, txn) => cursor.getFirst(LockMode.RMW).map {
              case (key, pKey, value) =>
              val link = linkSerializer.entryToObject(value)
              // update it's state to "in progress"
              link.storageState match {
              case LinkStorageState.QUEUED =>
              val newLink = link.copy(storageState = LinkStorageState.IN_PROGRESS)
              mainDatabase.put(txn,pKey,linkSerializer.objectToEntry(newLink))
              Right(newLink)
              case LinkStorageState.IN_PROGRESS =>
              Left(AlreadyInProgress)
              }
              }.getOrElse(Left(NoRecordInDatabase))
              1) We would like to know whether your withCursorInTransaction passes the txn to the openCursor method (or you can post the source code for us to investigate).
              2) Does withCursorInTransaction close the cursor?
              I added debug statements to verify the key content and got:

              2013-04-01 21:01:15,945 DEBUG [BDBJEQueue] Key: 19=>11D9FD73687474703A2F2F3132332E636F6D00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

              2013-04-01 21:01:15,964 DEBUG [BDBJEQueue] KEY: 19=>11D9FD73687474703A2F2F3132332E636F6D00
              3) What keys did you print here exactly?

              >
              where 19 is DatabaseEntry.getSize

              so the primary keys are the same, what may be wrong with secondary key?
              4) What you can do is to call SecondaryIntegrityException.getSecondaryKey and print the secondary key when you catch the exception.

              >
              What I'm trying to do is to
              - get first record from secondary database
              - update content of the record in primary database, after the update this record should not appear in secondary database because the key is changed

              I assume that this is not allowed anymore - so if I need to modify the record in primary database, I can not do it having transaction on secondary database, and 'remove' record from secondary database at same time?
              5) The functionality in JE didn't change. What was 'allowed' in this area in the previous releases should work now too.
              6) Do you know what might have changed in your application recently that caused this?
              7) If you've seen a change in JE behavior, it would help us a great deal if you can find out between which two releases it happened.
              8) Are you using a key comparator?
              9) Can you post the code for opening the primary database and the secondary key creator code (if any)?
              10) Can you reproduce the problem starting from scratch, with an empty environment? Can you reproduce it with a standalone test case?
              11) Does it matter which record you try to update, or it can happen for any key in your database?
              12) Can you reproduce the error consistently (at each run) or does it happen randomly?

              Thanks,
              Bogdan
              • 4. Re: Secondary is corrupt: the primary record contains a key that is not present
                1000381
                The failing test is "populate record from queue and wait for another one"

                https://github.com/jdevelop/webspider/blob/jdevelop-fix-tests-5/webspider-storage/src/test/scala/com/webspider/storage/bdbje/BDBJEQueueTest.scala

                the test class is

                https://github.com/jdevelop/webspider/blob/jdevelop-fix-tests-5/webspider-storage/src/main/scala/com/webspider/storage/bdbje/BDBJEQueue.scala

                method name is "def pop():"

                DB is opened in

                https://github.com/jdevelop/webspider/blob/jdevelop-fix-tests-5/webspider-storage/src/main/scala/com/webspider/storage/bdbje/BDBJEInitAndClose.scala

                test is failing every time on clean environment.
                • 5. Re: Secondary is corrupt: the primary record contains a key that is not present
                  1000381
                  Any update on this issue? Perhaps I can not update primary node if it will remove current secondary node from secondary database?

                  If this is true - then how can I change status of the node within same transaction?
                  • 6. Re: Secondary is corrupt: the primary record contains a key that is not present
                    Greybird-Oracle
                    Any update on this issue? Perhaps I can not update primary node if it will remove current secondary node from secondary database?
                    If this is true - then how can I change status of the node within same transaction?
                    No, that is not the case. As Bogdan said, the functionality in JE didn't change. He is taking a look at your test. The mostly likely thing is a problem with the secondary key creator.
                    --mark                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
                    • 7. Re: Secondary is corrupt: the primary record contains a key that is not present
                      1000381
                      The issue was related to the secondary key creator.

                      Thanks for the tip! Now it works as expected.