11 Replies Latest reply on Oct 25, 2010 9:08 AM by 801615

    Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?

    766393
      Hi,

      I have a question regarding query IDs used in ORACLE_SEM_FS_NS prefix definition and in Joseki / Oracle querymgt servlet.

      It seems that :
      * ORACLE_ORARDF_QUERY_MGT_TAB table apparently can hold longs
      * the querymgt servlet accepts longs and returns "No valid query ID specified." for anything over 64 bits.
      * running queries with long qid (either from a Joseki endpoint or from Jena) results in weird behaviour :
      ** qid < 2^32 : OK
      ** qid < 2^64 : returns either an exception (see below) or an empty result set immediately (with no exception)
      ** qid > 2^64 : query runs OK but qid cannot be used to stop query since it is over 64 bits


      Here is the exception shown if you use a qid between 32 and 64 bits :
      Exception in thread "AWT-EventQueue-0"
      java.lang.NumberFormatException: For input string: "8947349906"
      at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
      at java.lang.Integer.parseInt(Integer.java:459)
      at java.lang.Integer.parseInt(Integer.java:497)
      at oracle.spatial.rdf.client.jena.OracleQueryContextParameters.getQID(OracleQueryContextParameters.java:83)


      My question: due to the discrepancies when using large values, I do not exactly understand : are the query ids meant to be 32 or 64 bits ? Moreover: I haven't checked whether the id was signed or unsigned. Could you please specify exactly the range of possible values that qid can be assigned ?

      Thanks

      Regards,
      Julien
        • 1. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
          715399
          Hi Julien,

          There is a client side limitation of 32 bits on the query IDs.

          Cheers,
          Vladimir
          1 person found this helpful
          • 2. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
            766393
            Thanks for your reply.

            However, I have just done a bit more testing and it appears that the only values that really work are between (including) 1 and 2^31-1. For example, using 2^31 as a qid raises an exception when running the query.

            And the client side also accept negative values (no size limitation) and positive values over 64 bits, which it should not, apparently.
            • 3. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
              766393
              Would you be so kind as to explain a bit more / point to some documentation about how querymgt works ?

              Because I have to say that it does not seem very reliable.
              Quite often my queries are not stopped at all, even if using a working qid (between 1 and 2^31-1).

              A few details on how it works might help me find out what could be causing these issues. (in case it is due to something on my side).

              Thanks.

              Cheers,
              Julien
              • 4. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
                715399
                Hi Julien,

                Here's the general flow of query management with the Jena Adapter:

                1. User submits a query using ORACLE_SEM_FS_NS prefix, for example:

                PREFIX ORACLE_SEM_FS_NS: <http://example.com/semtech#qid=1234>
                SELECT ?subject WHERE {?subject ?property ?object }

                2. If the query is taking too much time for some reason, user can submit a kill request through the querymgt servlet:

                http://<hostname>:7001/joseki/querymgt?abortqid=1234

                3. The above request is persisted in the DB.

                4. Every couple of seconds, the Jena Adapter polls the DB to check if there's any query that needs to be killed. After query 1234 shows up and is returned, the statement used to execute the given query is canceled form the client side.

                Is this the behavior you have observed? If not please post what happens in your case. Are there any exceptions thrown?

                We'll fix the documentation, thanks for pointing out the long/negative number issue.

                Cheers,
                Vladimir
                1 person found this helpful
                • 5. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
                  766393
                  Hi,

                  Thanks for these details.

                  Well, to be honest, I cannot reproduce the issue I faced yesterday. The weblogic server providing Joseki was overloaded for some other reason (out of memory) and was only responding to querymgt queries (the SPARQL queries themselves were stuck threads).

                  Regarding the 4th point of your explanation : When you say that "the Jena Adapter" polls the DB regularly to check if any query needs to be killed, do you mean that the following would work ?
                  Example : A user runs a SPARQL query via a load balancer that uses two instances of Joseki w/ querymgt. The same user then wants to kill the same long-running query via the same load balancer and hits the other instance of Joseki w/ querymgt.

                  If I understood what you said, the query management features are only available for queries ran through Joseki since they are stopped on the client side. In that case, is there a way to stop long-running queries that are sent directly through JDBC (using Jena adapter) ?

                  Regarding the first question (available values for qid), do you plan to fix the fact that only less than half of all 32-bit values are available ? I am asking this because it is quite common to use 32-bit (or multiples of 32 anyway) IDs for instances / sessions of client applications and it seems to me that it would make it easier to use query IDs.

                  Thanks again for your help,

                  Regards
                  Julien
                  • 6. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
                    715399
                    Hi Julien,

                    My comments are inline.

                    Cheers,
                    Vlad


                    ----
                    Well, to be honest, I cannot reproduce the issue I faced yesterday. The weblogic server providing Joseki was overloaded for some other reason (out of memory) and was only responding to querymgt queries (the SPARQL queries themselves were stuck threads).

                    Regarding the 4th point of your explanation : When you say that "the Jena Adapter" polls the DB regularly to check if any query needs to be killed, do you mean that the following would work ?
                    Example : A user runs a SPARQL query via a load balancer that uses two instances of Joseki w/ querymgt. The same user then wants to kill the same long-running query via the same load balancer and hits the other instance of Joseki w/ querymgt.
                    ----

                    V: The two instances of Joseki are using the same datasource right? In that case, the kill request will be sent to both Joseki instances.

                    --------
                    If I understood what you said, the query management features are only available for queries ran through Joseki since they are stopped on the client side. In that case, is there a way to stop long-running queries that are sent directly through JDBC (using Jena adapter) ?
                    --------
                    V: If you submit a SPARQL query programmatically through the Jena Adapter against an Oracle-backed GraphOracleSem, and you specify a QID in the prefix, then the query is registered and you will still be able to kill it using the querymgt servlet. Let me know if this answered your question.

                    ------
                    Regarding the first question (available values for qid), do you plan to fix the fact that only less than half of all 32-bit values are available ? I am asking this because it is quite common to use 32-bit (or multiples of 32 anyway) IDs for instances / sessions of client applications and it seems to me that it would make it easier to use query IDs.
                    ------
                    V: We will look into allowing 64-bit query IDs in the next Jena Adapter release.
                    • 7. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
                      766393
                      Thanks a lot for your comments.

                      What I still do not understand is the case in which I "submit a SPARQL query programmatically through the Jena Adapter against an Oracle-backed GraphOracleSem".

                      Anyway, what would kill the query in this case ? You said that the query was stopped on the client side, so I suspected that there would be a thread running something like OracleQueryProgressMonitor.run() in the background, but there isn't any. Do I need to run OracleQueryProgressMonitor.getInstance().startMonitor() or something to be able to kill these queries ?

                      Thanks
                      Regards
                      Julien
                      • 8. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
                        715399
                        Hi Julien,

                        My reply is below.

                        Vlad

                        -----
                        What I still do not understand is the case in which I "submit a SPARQL query programmatically through the Jena Adapter against an Oracle-backed GraphOracleSem".

                        Anyway, what would kill the query in this case ? You said that the query was stopped on the client side, so I suspected that there would be a thread running something like OracleQueryProgressMonitor.run() in the background, but there isn't any. Do I need to run OracleQueryProgressMonitor.getInstance().startMonitor() or something to be able to kill these queries ?
                        -----

                        The OracleQueryProgressMonitor (OQPM) is started in conjunction with the Joseki SPARQL Endpoint. I believe the first time you run a query against the endpoint, it initializes the datasets and kicks off the OQPM.

                        I'm still kind of unclear about your setup. If you don't plan to use the Joseki SPARQL endpoint and you still want to perform query management, then it's doable, but you'll have to replicate some of the functionality yourself. First, you will have to kick off the OQPM like you said. You will also need to make sure your SPARQL queries have a QID in the PREFIX (as usual). Finally, whenever you want to kill a specific query, you will have to persist the kill request yourself in the DB. This discussion is getting too much into the internals of the Jena Adapter, so at this point I suggest we take it offline - contact me at vladimir dot kolovski at oracle dot com if you have any additional questions.
                        • 9. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
                          801615
                          Just wanted to document here, based on support from Oracle(Vlad), how to setup query monitoring with Jena Adapter in case you are not using Joseki and having your own SPARQL endpoint or similar

                          Oracle oracle = new Oracle(szJdbcURL, szUser, szPasswd);
                          OracleQueryProgressMonitor oqpm = OracleQueryProgressMonitor.getInstance();
                          oqpm.setConnection(oracle.getConnection());
                          oqpm.startMonitor();
                          ...
                          graph = new GraphOracleSem(oracle, szModelName, attachment);
                          __________________________________________

                          Now when you send abort request http://host/QM?abortqid=1234 to OracleQueryMgtServlet, then QueryExecution->execSelect throws an exception

                          Regards
                          Jürgen

                          Edited by: user8793616 on 2010-okt-22 06:49
                          • 10. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
                            715399
                            Hi Jürgen,

                            Thanks for posting this. One small correction,
                            the setConnection method in OracleQueryProgressMonitor is private. So, you cannot provide the connection to the monitor explicitly.

                            However, when starting the monitor instance:
                            OracleQueryProgressMonitor oqpm = OracleQueryProgressMonitor.getInstance();
                            oqpm.startMonitor();

                            the monitor looks for connections in a J2EE datasource called OracleSemDS. As long as you have the OracleSemDS datasource declared in your app server, this will work fine.

                            Regards,
                            Vlad
                            • 11. Re: Query IDs in ORACLE_SEM_FS_NS : 32 or 64 bits?
                              801615
                              Exactly. I forgot to remove setConnection when posting the snippet. It works without this line