Forum Stats

  • 3,759,183 Users
  • 2,251,510 Discussions
  • 7,870,526 Comments

Discussions

Hello, we think there is a bug in the com.sleepycat.je.util.DbBackup.startBackup method.

User_FH6VH
User_FH6VH Member Posts: 1 Green Ribbon
edited Apr 15, 2021 10:39AM in Berkeley DB Java Edition

I work at Ping Identity Corp. and our PingDirectory product uses Berkeley DB. One of our customer hit the error below when trying to backup the database (their version of PingDirectory uses Berkeley DB JE Version: 7.5.12):

 IllegalArgumentException: fromKey out of range (TreeMap.java:1880 TreeSet.java:350 FileProtector.java:841 DbBackup.java:578...)

Line 578 of DbBackup.java is the last line in the startBackup(), i.e.:

protectedFileSet.truncateHead(firstFileInBackup);

. We suspect that since the method truncates both head and tail, there is some bug when the size of the proectedFileSet is 1, which triggers an IllegalArgumentException. Thank you for reading (pasted the startBackup method below)!

/**
 * Start backup mode in order to determine the definitive backup set needed
 * at this point in time.
 *
 * <p>This method determines the last file in the backup set, which is the
 * last log file in the environment at this point in time.  Following this
 * method call, all new data will be written to other, new log files.  In
 * other words, the last file in the backup set will not be modified after
 * this method returns.</p>
 *
 * <p><em>WARNING:</em> After calling this method, deletion of log files in
 * the backup set by the JE log cleaner will be disabled until {@link
 * #endBackup()} is called.  To prevent unbounded growth of disk usage, be
 * sure to call {@link #endBackup()} to re-enable log file deletion.
 * Additionally, the Environment can't be closed until endBackup() is
 * called.
 * </p>
 *
 * @throws com.sleepycat.je.rep.LogOverwriteException if a replication
 * operation is overwriting log files. The backup can not proceed because
 * files may be invalid. The backup may be attempted at a later time.
 *
 * @throws EnvironmentFailureException if an unexpected, internal or
 * environment-wide failure occurs.
 *
 * @throws IllegalStateException if a backup is already in progress
 */
public synchronized void startBackup()
    throws DatabaseException {

    if (backupStarted) {
        throw new IllegalStateException("startBackup was already called");
    }

    /* Throw a LogOverwriteException if the Environment is rolling back. */
    if (!envImpl.addDbBackup(this)) {
        throw envImpl.createLogOverwriteException
            ("A replication operation is overwriting log files. The " +
             "backup can not proceed because files may be invalid. The " +
             "backup may be attempted at a later time.");
    }

    final FileProtector fileProtector = envImpl.getFileProtector();

    /*
     * For a network restore we use a different backup name/id and also
     * protect the 2 newest (highest numbered) reserved files. This is a
     * safeguard to ensure that the restored node can function as a master.
     * In a future release this approach may be improved by additionally
     * copying all reserved files to the restored node, but only after it
     * has recovered using the active files [#25783].
     */
    final long backupId = networkRestore ?
        envImpl.getNodeSequence().getNextNetworkRestoreId() :
        envImpl.getNodeSequence().getNextBackupId();

    final String backupName =
        (networkRestore ?
         FileProtector.NETWORK_RESTORE_NAME :
         FileProtector.BACKUP_NAME) +
        "-" + backupId;

    final int nReservedFiles = networkRestore ? 2 : 0;

    /*
     * Protect all files from deletion momentarily, while determining the
     * last file and protecting the active files.
     */
    final ProtectedFileRange allFilesProtected =
        fileProtector.protectFileRange(backupName + "-init", 0);

    try {
        if (envIsReadOnly) {
            /*
             * All files are currently immutable, so the backup list is the
             * current set of files. However, we can't add a marker to the
             * last file in list, and therefore it will not be immutable
             * after being restored to a read-write directory (unless the
             * user sets ENV_RECOVERY_FORCE_NEW_FILE after restoring).
             */
            lastFileInBackup = envImpl.getFileManager().getLastFileNum();
        } else {
            /*
             * Flip the log so that all files in the backup list are
             * immutable. But first, write an "immutable file" marker in
             * the last file in the backup, so it cannot be modified after
             * it is restored. Recovery enforces this rule.
             */
            LogEntry marker = new SingleItemEntry<>(
                LogEntryType.LOG_IMMUTABLE_FILE, new EmptyLogEntry());

            long markerLsn = envImpl.getLogManager().log(
                marker, ReplicationContext.NO_REPLICATE);

            envImpl.forceLogFileFlip();
            lastFileInBackup = DbLsn.getFileNumber(markerLsn);
        }

        /*
         * Protect all active files from deletion. This includes files
         * prior to firstFileInBackup, in order to get a snapshot of all
         * active files. New files do not need protection, since the backup
         * set does not include them. lastFileInBackup will be protected by
         * protectActiveFiles because it is not the last file in the env.
         */
        protectedFileSet = fileProtector.protectActiveFiles(
            backupName, nReservedFiles, false /*protectNewFiles*/);

    } finally {
        fileProtector.removeFileProtection(allFilesProtected);
    }

    /* At this point, endBackup must be called to undo file protection. */
    backupStarted = true;

    /* Files after lastFileInBackup do not need protection. */
    protectedFileSet.truncateTail(lastFileInBackup);

    /* Snapshot the complete backup file set. */
    snapshotFiles = protectedFileSet.getProtectedFiles();

    /*
     * Now that we have the snapshot, files before firstFileInBackup do not
     * need protection.
     */
    protectedFileSet.truncateHead(firstFileInBackup);
}


Answers