Berkeley DB Family

Hello Katta,

Are you using transactions, and if so, are you committing the transactions?

Regards,

Charles Lamb

Hi Charles,

I'm using transactions & committing the transactions. Please go through the following code.

EnvironmentConfig envConfig = new EnvironmentConfig();
envConfig.setTransactional(true);
envConfig.setAllowCreate(true);

DatabaseConfig dbConfig = new DatabaseConfig();
dbConfig.setAllowCreate(true);
dbConfig.setTransactional(true);

Environment env = new Environment(dbHome, envConfig);
Transaction txn = null;
Database index, blocks;

try {
txn = env.beginTransaction(null, null);
index = env.openDatabase(txn, "__index__", dbConfig);
blocks = env.openDatabase(txn, "__blocks__", dbConfig);
} catch (DatabaseException e) {
if (txn != null)
{
txn.abort();
txn = null;
}
throw e;
} finally {
if (txn != null)
txn.commit();
txn = null;
}
try{
txn = env.beginTransaction(null, null);
indexDir = new JEDirectory(txn, index, blocks);
System.out.println("INDEXING INFO: Start Indexing updated content.");

IndexWriter indexWriter = new IndexWriter(indexDir,new StandardAnalyzer(), false);
indexWriter.setUseCompoundFile(false);
// configure the writer
indexWriter.setMergeFactor(1000);
indexWriter.setMaxMergeDocs(9999999);
indexWriter.setMaxBufferedDocs(1000);

// call the updateLuceneIndex method
updateLuceneIndex(indexWriter );

indexWriter.optimize();
indexWriter.close();

System.out.println("INDEXING INFO: Optimizing Index finished......");
} catch (IOException e)
{
txn.abort();
txn = null;
e.printStackTrace();
throw e;
} finally
{
if (txn != null)
txn.commit();

index.close();
blocks.close();
env.close();
}


Thanks,

Katta.

Hi Katta,

Can you please do a DbDump of the database before and after to see if anything has been added to it when you run this? You can do this with

java com.sleepycat.je.util.DbDump -h <envdir> -s <dbname>

Thanks.

Charles Lamb

Hi Charles ,

Even if i run DbDump before and after lucene incremental update there is no change in the db size.I did the following way.

Step:

1. java com.sleepycat.je.util.DbDump -h D:\berkely\kindex -s __blocks__

2. java com.sleepycat.je.util.DbDump -h D:\berkely\kindex -s __index__

3. java com.tg.lucene.UpdateLuceneIndex D:\berkely\dbEnvHome update

4. java com.sleepycat.je.util.DbDump -h D:\berkely\kindex -s __blocks__

5.java com.sleepycat.je.util.DbDump -h D:\berkely\kindex -s __index__

Please can you explain what DbDump does. How it reduce the size. I indexed Country data(list of countries in my table), it took 360KB. After that in update, i deleted 2 countries data from that db, and optimized it. But the size grown to 770KB. While deleting data why it's size is growing instead of shrinking.


Thanks,

Katta.

Hi Katta, JE has a log based storage system, so deletes will write to the log. You may want to read the following FAQ's for more information on why you're seeing the log grow during deletes: http://forums.oracle.com/forums/thread.jspa?messageID=1411144&tstart=0#1411144.

Hi,

The suggested link on JE log & FAQ taking me to the same page.Still i'm unable to get the solution for JE log grow during deletes.

Thanx,

Katta.

Try this link:

http://www.oracle.com/technology/products/berkeley-db/faq/je_faq.html#14

Ron