6 Replies Latest reply: Feb 22, 2011 3:00 AM by 637288 RSS

    DB_RUNRECOVERY: Fatal error, run database recovery:

    756043
      Hello,

      I've an application deployed with GlassFish v3 in Solaris.
      This application used database Berkeley DB XML.
      Sometimes, when a query to the database ends with error, i have to deploy the application again and I get this error:

      The log message is null. com.sleepycat.db.RunRecoveryException
      Complete Message
           
      DB_RUNRECOVERY: Fatal error, run database recovery: PANIC: fatal region error detected; run recovery: DB_RUNRECOVERY: Fatal error, run database recovery at com.sleepycat.db.internal.db_javaJNI.DbEnv_open(Native Method) at com.sleepycat.db.internal.DbEnv.open(DbEnv.java:317) at com.sleepycat.db.EnvironmentConfig.openEnvironment(EnvironmentConfig.java:3886) at com.sleepycat.db.Environment.<init>(Environment.java:93) at com.era7.lib.bdbxmlapi.BdbxmlManager.initEnvironment(BdbxmlManager.java:348) at com.era7.lib.bdbxmlapi.BdbxmlManager.init(BdbxmlManager.java:264) at com.era7.lib.bdbxmlapi.BdbxmlManager.<init>(BdbxmlManager.java:134) at com.era7.doPlanning.appConf.listeners.DoPlanningApplicationListener.createManager(DoPlanningApplicationListener.java:171) at com.era7.doPlanning.appConf.listeners.DoPlanningApplicationListener.configureBdbxmlInitializationParams(DoPlanningApplicationListener.java:104) at com.era7.lib.slbdbxml.listeners.AbstractApplicationListener.contextInitialized(AbstractApplicationListener.java:51) at org.apache.catalina.core.StandardContext.contextListenerStart(StandardContext.java:4591) at com.sun.enterprise.web.WebModule.contextListenerStart(WebModule.java:535) at org.apache.catalina.core.StandardContext.start(StandardContext.java:5193) at com.sun.enterprise.web.WebModule.start(WebModule.java:499) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:928) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:912) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:694) at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1933) at com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1605) at com.sun.enterprise.web.WebApplication.start(WebApplication.java:90) at org.glassfish.internal.data.EngineRef.start(EngineRef.java:126) at org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:241) at org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:236) at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:339) at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:340) at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:163) at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174) at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:87) at java.security.AccessController.doPrivileged(Native Method) at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:84) at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77) at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58) at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107) at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60) at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236) at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:128) at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:457) at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:401) at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:125) at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640) at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700) at org.apache.felix.framework.Felix.startBundle(Felix.java:1622) at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:915) at org.jvnet.hk2.osgimain.Main.start(Main.java:140) at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:640) at org.apache.felix.framework.Felix.activateBundle(Felix.java:1700) at org.apache.felix.framework.Felix.startBundle(Felix.java:1622) at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1077) at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264) at java.lang.Thread.run(Thread.java:619)








      My configuration is this:

      public static int MAX_LOG_BUFFER_SIZE = 50 * 1024 * 1024;
      public static int MAX_CACHE_SIZE = 5 * 1024 * 1024;
      +/**+
      * max num of simultaneous active transactions
      */
      public static int MAX_ACTIVE_TXNS = 10000;
      +//---------------------DEFAULT ENVIRONMENT CONFIGURATION VALUES---------------------------------+
      public static boolean RUN_RECOVERY_BY_DEFAULT = true;
      public static int CACHE_SIZE_BY_DEFAULT = 5 * 1024 * 1024;
      public static int MAX_ACTIVE_TXNS_BY_DEFAULT = 5000;
      public static int LOG_BUFFER_SIZE_BY_DEFAULT = 50 * 1024 * 1024;
      public static boolean ENVIRONMENT_LOG_IN_MEMORY = false;
      public static boolean ENVIRONMENT_LOG_AUTO_REMOVE = true;
      public static int ENVIRONMENT_LOG_REGION_SIZE = 3 * 1024 * 1024;
      public static boolean ENVIRONMENT_TRANSACTIONAL = true;
      public static boolean ENVIRONMENT_THREADED = true;
      public static int ENVIRONMENT_TXN_MAX_ACTIVE = 5000;
      public static int DEFAULT_MAX_LOCKS = 15000;//2000;
      public static int DEFAULT_MAX_LOCKERS = 15000;//2000;
      public static int DEFAULT_MAX_LOCK_OBJECTS = 15000;//2000;
      +//---------------------------------------------------------------------------------+
      +//--------------------SEQUENCE CONFIGURATION VALUES-------------------+
      public static boolean ALLOW_CREATE_SEQUENCE_CONFIG = true;
      public static int SEQUENCE_CACHE_SIZE = 0;
      public static boolean DECREMENT_SEQUENCE_CONFIG = false;
      public static int SEQUENCE_MIN_VALUE = 0;
      public static int SEQUENCE_MAX_VALUE = 99999999;
      public static int SEQUENCE_INITIAL_VALUE = 0;
      public static boolean EXCLUSIVE_CREATE_SEQUENCE_CONFIG = false;
      public static boolean WRAP_SEQUENCE_CONFIG = false;
      +//----------------------------------------------------------------------ç+
      +//--------------------XMLMANAGER CONFIGURATION VALUES-------------------+
      public static boolean XML_MANAGER_LOG_ENABLED = false;
      public static int XML_MANAGER_LOG_LEVEL = XmlManager.LEVEL_ERROR;
      public static int XML_MANAGER_LOG_CATEGORY = XmlManager.CATEGORY_MANAGER;
      +//----------------------------------------------------------------------+
      +//--------------------XMLCONTAINER CONFIGURATION VALUES-------------------+
      public static boolean XML_CONTAINER_INDEX_NODES_AND_STATISTICS_ENABLED = true;
      public static boolean XML_CONTAINER_ALLOW_VALIDATION = false;
      +//----------------------------------------------------------------------+
      +//--------------------SEQUENCE DATABASE CONFIGURATION VALUES-------------------+
      public static boolean ALLOW_CREATE_SEQUENCE_DATABASE_CONFIG = true;
      public static boolean TRANSACTIONAL_SEQUENCE_DATABASE_CONFIG = true;
      +//----------------------------------------------------------------------+






      Reading the logs I don't find any other error apart from these WARNING:

      +(javax.enterprise.system.container.web.com.sun.enterprise.web) Unsupported http-service property (redirect_2) is being ignored+

      +(com.sun.grizzly.config.GrizzlyServiceListener) Interrupting idle Thread: http-thread-pool-8080+





      Maybe it is neccesary to close the environment when some query to the database fails, i'm not sure.

      Any help is welcome

      Thank you
        • 1. Re: DB_RUNRECOVERY: Fatal error, run database recovery:
          655560
          Hi,

          It seems that some transactions does not be committed/aborted when the program exit. So an recovery procedure is required when you re-open the environment.

          Please add a catch {} to catch and deal with those query exception, commit/abort transactions explicitly and close environment at the end.

          Best regards,
          Rucong Zhao
          Oracle Berkeley DB XML
          • 2. Re: DB_RUNRECOVERY: Fatal error, run database recovery:
            637288
            Hi,

            you code should be responsible for trying to do recovery if needed. That is whenever you receive a RunDbRecovery exception, you must abort all transactions, close all opened environment handles and open an environment again with RUN_DB_RECOVERY flag. More can be found at http://download.oracle.com/docs/cd/E17276_01/html/gsg_xml_txn/java/recovery.html

            If you have further questions, please ask

            Vyacheslav
            • 3. Re: DB_RUNRECOVERY: Fatal error, run database recovery:
              756043
              Hi,

              This is the code of one of my simplest servlets I run in the application:


              import com.sleepycat.dbxml.XmlResults;
              import com.sleepycat.dbxml.XmlTransaction;

              /**
              *
              * @author
              */
              public class GetMainTreeService extends DPService{



              @Override
              protected Response processRequest(Request request, BasicSession session,
              HttpServletRequest httpRequest, BdbxmlManager manager,
              QueryProvider queryProvider, SequenceProvider sequenceProvider,
              User user) throws Exception {

              Response response = new Response();


                        try{

                             //Transaction get tree
                             XmlTransaction transaction = manager.createTransaction();
                             GetMainTreeQuery getMainTreeQuery = queryProvider.getMainTreeQuery();
                             getMainTreeQuery.setIdUser(user.getIdUser());
                             XmlResults getMainTreeResults = manager.executeQuery(transaction, getMainTreeQuery, false);


                             if(getMainTreeResults.hasNext()){

                                  Areas areas = new Areas(getMainTreeResults.next().asString());
                                  response.addChild(areas);

                                  //Successful response
                                  response.setStatus(Response.SUCCESSFUL_RESPONSE);

                             }else{
                                  response.setError(Response.ERROR_RESPONSE);
                             }
                             getMainTreeResults.delete();
                             transaction.commit();

                        }catch(Exception e){

                             //Sends error
                             e.printStackTrace();
                             response.setError(Response.ERROR_RESPONSE);

                        }


              return response;

              }



              }

              I see that if any exception is caught I don't run getMainTreeResults.delete(); or transaction.commit();

              Do I need to include it like this?



                        }catch(Exception e){

                             //Sends error
                             getMainTreeResults.delete();
                             transaction.commit();
                             e.printStackTrace();
                             response.setError(Response.ERROR_RESPONSE);

                        }
                        
                        
              I'm not sure because this servlet is very simple, but sometimes I have many queries with the same transaction and many xmlresults.
              So if the exception is caused by other reason, I don't Know if transaction is committed yet or xmlResult is closed.



              Thank you


              Sorry, I tried to put italics for code, but I coudn't
              • 4. Re: DB_RUNRECOVERY: Fatal error, run database recovery:
                637288
                Hi,

                as I said you should have a special catch clause for the RunRecoveryException exception. In this case you should try to abort a transaction, but the latter may fail as well, so you should have another try clause for this.
                Please read the documentation first, the recovery procedure is explained in a nice way

                Vyacheslav
                • 5. Re: DB_RUNRECOVERY: Fatal error, run database recovery:
                  756043
                  Ok, thank you.

                  Is there any way to close all opened handle in the environment, or have I got to close one by one?
                  • 6. Re: DB_RUNRECOVERY: Fatal error, run database recovery:
                    637288
                    Hi,

                    by the env handle I meant the Environment object, or objects if you have a multiprocess application. Before closing the environment (env.close()) you, of course, have to close all the containers you have opened.

                    Vyacheslav