4 Replies Latest reply: Mar 3, 2011 3:35 AM by 835648 RSS

    Uncaught exception from C++ API, errcode = INTERNAL_ERROR preparing queries

    835648
      Hi,

      I have XQuery code with some fairly hefty XQueries that I have been working on. I have had to make a number of changes to the queries so that they run but individually they appear to work fine.

      The problem is that they take a couple of seconds to compile, so I need to prepare and store the XmlQueryExpressions rather than compile them on the fly.

      This is fine at first but after I have run prepare a dozen times, I get the dreaded Uncaught exception from C++ API on all successive attempts to run prepare.

      I have turned on all the logging I can and only get "INFO: Optimizer - Started parse" before the error:

      INFO: Optimizer - Started parse
      com.sleepycat.dbxml.XmlException: Uncaught exception from C++ API, errcode = INTERNAL_ERROR
           at com.sleepycat.dbxml.dbxml_javaJNI.XmlManager_prepare__SWIG_0(Native Method)
           at com.sleepycat.dbxml.XmlManager.prepare(XmlManager.java:534)
      ...

      I am guessing that I am running out of some kind of resource. Are there any clues about what I need to increase to make this work?

      Thanks.
        • 1. Re: Uncaught exception from C++ API, errcode = INTERNAL_ERROR preparing queries
          655560
          Hi,

          * Did you apply Transaction on the prepare?
          * If yes, did you apply one Transaction on multiple prepare operations? Or create a new Transactions for each prepare? Please also ensure that you didn't use aborted/committed transactions.

          If possible, please show the code logci. And a program to reproduce it is better.

          If you want more clues, this FAQ item might help you to debug the Java application:
          http://www.oracle.com/technetwork/database/berkeleydb/xml-faq-088319.html#WhendebuggingmyBDBXMLJavaapplicationinEclipsehowcanIstepintothenativeClibraries

          Best regards,
          Rucong
          Oracle Berkeley DB XML
          • 2. Re: Uncaught exception from C++ API, errcode = INTERNAL_ERROR preparing queries
            835648
            Hi Rucong,

            I am currently not using transactions.

            The code for retrieving an XQuery to run got a bit convoluted. Note that for simplicity I have defined all XQuery functions that I call with the same parameters. An example is this function that fetches the details about a user in the system:

            declare function api:user-details($session as xs:string, $user as xs:string, $silo as xs:string, $values as item()*) {
            api:_debug("user-details", $session, $user, $silo, $values),
            let $skey := api:_skey($session, $user, $silo)
            let $username := $values[1] cast as xs:string?
            return
            if ( string-length($username) eq 0 ) then
            users:getCurrentUserDetails($skey)
            else
            users:getUserDetails($skey, $username)
            };

            The java code that compiles the queries is below. savedQuery is a HashMap<String, XmlQueryExpression> that stores the precompiled queries. Without precompiling them it can take several seconds for a query to run. Because I can only store a few queries before I get exceptions, I am counting the usage of the query to only store the most commonly used ones, throwing less used ones away. The code will call getQuery to get the query then creates a new XmlQueryContext to execute it with, but simply calling the code below with a dozen complex XQueries is enough to cause the error.

                 private XmlQueryExpression getQuery(String libraryName, String libraryUri, String name) {
                      String key = libraryUri + "/" + name;
                      XmlQueryExpression queryExp = savedQuery.get(key);
                      System.out.println("XQUERY: "+key);
                      
                      boolean failed = false;
                      
                      synchronized (savedQuery) {
                           while ( queryExp == null ) {
                                if ( failed ) {
                                     int min = -1;
                                     String delkey = null;
                                     for (String k: savedQuery.keySet()) {
                                          int count = savedQueryCount.get(k);
                                          if ( min == -1 || min > count ) {
                                               min = count;
                                               delkey = k;
                                          }
                                     }
                                     if ( delkey != null ) {
                                          queryExp = savedQuery.get(delkey);
                                          savedQuery.remove(delkey);
                                          System.out.println("XQUERY["+(savedQuery.size()+1)+"]: "+delkey+" REMOVED");
                                          queryExp.delete();
                                     }
                                     else {
                                          return null;
                                     }
                                }
                                
                                try {
                                     String query = "import module namespace napi='"
                                          + libraryUri + "' at '" + libraryName + ".xql';napi:"+name+"($skey,$u,$s,$v)";
                                
                                     XmlQueryContext context = notusManager.createQueryContext();
                                     context.setBaseURI(Configuration.getInstance().getDBXmlHomeRoot());
                                     System.out.println("XQUERY["+(savedQuery.size()+1)+"]: "+query);
                                     context.setVariableValue("skey", new XmlValue(""));
                                     context.setVariableValue("u", new XmlValue(""));
                                     context.setVariableValue("s", new XmlValue(""));
                                     context.setVariableValue("v", new XmlValue(""));
                                     queryExp = notusManager.prepare(query, context);
                                     
                                     savedQuery.put(key, queryExp);
                                } catch (XmlException e) {
                                     System.err.println("getQuery error: " + e.toString());
                                     failed = true;
                                }
                           }
                           int count = 1;
                           if (savedQueryCount.containsKey(key))
                                count = savedQueryCount.get(key)+1;
                           savedQueryCount.put(key, count);
                           System.out.println("................" + name + " USED: "+count+ " times.");
                      }
                      return queryExp;
                 }

            Unfortunately I have an additional problem to debug the application as it runs inside a web service application rather than a stand-alone executable, but I may be able to copy the key code into a test app to try this.

            Best regards
            Pete
            • 3. Re: Uncaught exception from C++ API, errcode = INTERNAL_ERROR preparing queries
              655560
              Hi, Pete

              Prepared queries are extremely large and use a lot of process heap (C++) memory. An application realistically cannot store that many of them. So if you executed "dozen" of prepare, the exception is probably the C++ code running out of heap space for a prepared query.

              Please try to limit the number of prepared queries stores and make sure that useless prepare object is freed in time (In Java, there is no unnecessary long-term additional quotations).

              Best regards,
              Rucong Zhao
              Oracle Berkeley DB XML
              • 4. Re: Uncaught exception from C++ API, errcode = INTERNAL_ERROR preparing queries
                835648
                Hi Rucong,

                I was afraid that this might be the answer. Thanks for the information. I will start re-coding in Java.

                Best Regards,
                Pete.