1 2 Previous Next 17 Replies Latest reply on Jun 21, 2015 8:17 AM by Mandeep_lohan

    Loading Ontology (.owl file) to oracle 11.2g through jena taking time

    Mandeep_lohan

      I have created an ontology  using OWL api and i am loading this owl file to oracle 11.2g  as triple store. I am using JENA to load the file. But for 1.5gb file it is taking 1 hour. Is there any other to load the file faster to oracle. In oracle my ontology is as triples store.

      Any suggestion would be appreciated . Thanks in Advance.

      Mandeep

        • 1. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
          alwu-Oracle

          Hi Mandeep,

           

          Which API did you use to load the file? Given the size, bulk loader (prepareBulk, completeBulk APIs) is a good choice.

           

          Thanks,

           

          Zhe Wu

          • 2. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
            Mandeep_lohan

            Thanks for the concern Zhe.

            I have a 2Gb owl file. and the size will increase in future. I am using jena to load it. here is small code template i am using to load :

             

              Model model= ModelFactory.createDefaultModel();

              InputStream is = FileManager.get().open(inputOwl);

              model.read(is, "", "RDF/XML");

              is.close();

             

             

              oracle= new Oracle(szJdbcURL, szUser, szPasswd);

              ModelOracleSem modelDest = ModelOracleSem.createOracleSemModel(oracle, szModelName);

              GraphOracleSem g = modelDest.getGraph();

              g.dropApplicationTableIndex();

             

             

              int method = 1;

              String tbs = "SYSAUX";

              if (method == 1) {

               System.out.println("start batch load");

               g.getBulkUpdateHandler().addInBatch(

                   GraphUtil.findAll(model.getGraph()), tbs);

               System.out.println("end size " + modelDest.size());

              }

              else {

               System.out.println("start bulk load");

               g.getBulkUpdateHandler().addInBulk(

                   GraphUtil.findAll(model.getGraph()), tbs);

               System.out.println("end size " + modelDest.size());

              }

              g.rebuildApplicationTableIndex();

             

            Thanks,

            Mandeep

            • 3. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
              alwu-Oracle

              Hi Mandeep,

               

              If you are using this code snippet, then please set method=2 (so that bulk load is used).

               

              Note that, addInBulk can accept a third parameter (String flags). If you are using a quad core

              machine with a decent I/O storage (or something more powerful), you can set flags to

               

              " PARSE PARALLEL_CREATE_INDEX PARALLEL=4  mbv_method=shadow "

               

              As your graph size grows bigger, you may want to use prepareBulk and completeBulk.

               

              Thanks,


              Zhe Wu

              • 4. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                Mandeep_lohan

                Thanks Zhe,

                 

                I tried with first approach, giving the 3rd argument. but it throwing tis exception. But without this not exception. But bulk load is taking more time than batch loading.

                 

                then i tried using this :

                g.getBulkUpdateHandler().completeBulk(inputOwl, tbs);  but it also thrown exception:

                 

                java.sql.SQLSyntaxErrorException: ORA-00942: table or view does not exist. and i think it is because i am giving owlFile path and not the model. But this method completeBulk(String,Strng) takes strings so i cant give model instance.

                 

                is there as better way to implement these methods or any other way to load a file using jena or any other api that i can use to load my owl to triple store and fire sparql on that.

                 

                Again, thanx for all your help

                Mandeep

                • 5. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                  alwu-Oracle

                  Re:

                     I tried with first approach, giving the 3rd argument. but it throwing tis exception.

                   

                  What exactly was the exception?

                   

                  Re:

                  g.getBulkUpdateHandler().completeBulk(inputOwl, tbs);  but it also thrown exception:

                   

                  No. This is not the way to use this API.

                  Please take a look at Section 7.12 Bulk Loading Using RDF Semantic Graph Support for Apache Jena

                  https://docs.oracle.com/database/121/RDFRM/sem_jena.htm#RDFRM267

                   

                  You will need to use prepareBulk first.

                   

                  InputStream is = new FileInputStream(szIndFileName);

                  graph.getBulkUpdateHandler().prepareBulk(

                    is, // input stream

                    "http://example.com", // base URI

                    "RDF/XML", // data file type: can be RDF/XML, N-TRIPLE, etc.

                    "SEMTS", // tablespace

                    null, // flags

                    null, // listener

                    null // staging table name.

                    );

                    is.close();

                   

                  After that,

                  graph.getBulkUpdateHandler().completeBulk(

                    null, // flags for invoking SEM_APIS.bulk_load_from_staging_table

                    null // staging table name

                  );

                   

                  Hope it helps,


                  Zhe Wu

                  • 6. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                    alwu-Oracle

                    Note that the above snippet assumes you have a SEMTS tablespace. You can change it to one of the tablespace name you have. Or you can create it with (assume you are not using ASM) the following:

                     

                    create bigfile tablespace semts

                    datafile '?/dbs/semts01.dat' size 512M reuse autoextend on next 512M maxsize unlimited

                      EXTENT MANAGEMENT LOCAL

                      segment space management auto;

                     

                    Hope it helps,

                     

                    Zhe Wu

                    • 7. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                      Mandeep_lohan

                      Thank Zhe.

                      let me try this. i will get back

                      • 8. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                        Mandeep_lohan

                        This is the exception i am getting when i am trying this :

                         

                        graph.getBulkUpdateHandler().addInBulk(

                               GraphUtil.findAll(model.getGraph()), tbs," PARSE PARALLEL_CREATE_INDEX PARALLEL=4  mbv_method=shadow ");

                         

                        exception:

                         

                        java.sql.SQLException: ORA-13199: Invalid option (#2 on list): PARSE

                        ORA-06512: at "MDSYS.MD", line 1723

                        ORA-06512: at "MDSYS.MDERR", line 17

                        ORA-06512: at "MDSYS.SDO_RDF_INTERNAL", line 414

                        ORA-06512: at "MDSYS.SDO_RDF_INTERNAL", line 3601

                        ORA-06512: at "MDSYS.SDO_RDF", line 276

                        ORA-06512: at "MDSYS.RDF_APIS", line 693

                        ORA-06512: at line 1

                         

                         

                          at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)

                          at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)

                          at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)

                          at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)

                          at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)

                          at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)

                          at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:204)

                          at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:1041)

                          at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)

                          at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)

                          at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3685)

                          at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:4714)

                          at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1376)

                          at oracle.spatial.rdf.client.jena.Oracle.executeCall(Oracle.java:818)

                          at oracle.spatial.rdf.client.jena.OracleBulkUpdateHandler.completeBulk(OracleBulkUpdateHandler.java:1644)

                          at oracle.spatial.rdf.client.jena.OracleBulkUpdateHandler.addInBulk(OracleBulkUpdateHandler.java:677)

                          at oracle.spatial.rdf.client.jena.OracleBulkUpdateHandler.addInBulk(OracleBulkUpdateHandler.java:512)

                          at com.bosch.ebr.jena.LoadOwl.main(LoadOwl.java:95)

                         

                         

                        Thanks

                        Mandeep

                        • 9. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                          Mandeep_lohan

                          Now i tried for the prepare and bulk loader. it also thrown exception. here is my code sample

                           

                          oracle= new Oracle(szJdbcURL, szUser, szPasswd);

                            ModelOracleSem modelDest = ModelOracleSem.createOracleSemModel(oracle, szModelName);

                            GraphOracleSem graph = modelDest.getGraph();

                            DatasetGraphOracleSem dataset = DatasetGraphOracleSem.createFrom(graph);     //********************************* here it is throwing exception

                            graph.dropApplicationTableIndex();

                           

                            dataset.prepareBulk(is, "http://myontlogyURL",

                            "RDF/XML", "SEMTS", null, null, null, false);

                            is.close();

                           

                          graph.getBulkUpdateHandler().completeBulk(

                             null, // flags for invoking SEM_APIS.bulk_load_from_staging_table

                             "SEMTS" // staging table name

                            );

                           

                          Exception in thread "main" java.lang.NoClassDefFoundError: org/openjena/atlas/lib/Sink

                            at com.bosch.ebr.jena.LoadOwl.main(LoadOwl.java:75)

                          Caused by: java.lang.ClassNotFoundException: org.openjena.atlas.lib.Sink

                            at java.net.URLClassLoader$1.run(Unknown Source)

                            at java.net.URLClassLoader$1.run(Unknown Source)

                            at java.security.AccessController.doPrivileged(Native Method)

                            at java.net.URLClassLoader.findClass(Unknown Source)

                            at java.lang.ClassLoader.loadClass(Unknown Source)

                            at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)

                            at java.lang.ClassLoader.loadClass(Unknown Source)

                            ... 1 more

                           

                           

                          I know i need to create tablespace 'SEMTS'. I  was just giving a try. but before it reaches at semts. it thrown.

                          i have one doubt. in prepareBulk()  we load to staging table and then to sematic store using bulk(). so now we are going to have 2 tablles. one in staging and one my triple store. why we need to have 2 steps process. will it be fast enough?

                           

                          Thanks,

                          Mandeep

                          • 10. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                            alwu-Oracle

                            Hi Mandeep,

                             

                            What is the exact DB version (11.2.0.3 or 11.2.0.4?) What happens if you take out "PARSE" from the string?

                             

                            Thanks,


                            Zhe

                            • 11. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                              Mandeep_lohan

                              Hi Zhe,

                               

                              this the version we are using :

                               

                              Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit

                               

                              yeah i tried that too after seeing exception. but that also got exception. but now for the method name we are giving in string.

                               

                              java.sql.SQLException: ORA-13199: Invalid option (#4 on list): MBV_METHOD

                              ORA-06512: at "MDSYS.MD", line 1723

                              ORA-06512: at "MDSYS.MDERR", line 17

                              ORA-06512: at "MDSYS.SDO_RDF_INTERNAL", line 414

                              ORA-06512: at "MDSYS.SDO_RDF_INTERNAL", line 3601

                              ORA-06512: at "MDSYS.SDO_RDF", line 276

                              ORA-06512: at "MDSYS.RDF_APIS", line 693

                              ORA-06512: at line 1

                               

                              Thanks for all your help and constant support.

                              Mandeep

                              • 12. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                                alwu-Oracle

                                Hi Mandeep,

                                 

                                Can you please upgrade to 11.2.0.4 or maybe download and start using 12.1.0.1?

                                 

                                BTW, the following error indicates that there is a version mismatch of the Jena Adapter and the Apache Jena.

                                What jar files do you have on your classpath?

                                 

                                Exception in thread "main" java.lang.NoClassDefFoundError: org/openjena/atlas/lib/Sink

                                 

                                Thanks,

                                 

                                Zhe

                                • 13. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                                  Mandeep_lohan

                                  Hi Zhe,

                                   

                                  I am not sure about upgradation of db. Its in production. I need to ask.

                                   

                                  I am using jena-2.11.2 . and the jar files they have given with this.

                                   

                                  Also, the string we are giving with bulkLoad, how much it will impact the load time.

                                   

                                  Thanks,

                                  Mandeep

                                  • 14. Re: Loading Ontology (.owl file) to oracle 11.2g through jena taking time
                                    alwu-Oracle

                                    Hi Mandeep,

                                     

                                    The Apache Jena version you are using is not compatible with the Jena Adapter (and also the Oracle DB version) you are using.

                                    That's the reason why you saw the missing class.

                                     

                                    If you cannot upgrade to 11.2.0.4 or 12.1.0.1, then you need to apply patch 10098816 as described in Section 7.1 of the following document.

                                     

                                    Jena Adapter for Oracle Database

                                     

                                    Note that, you will need to use Jena 2.6.3, ARQ 2.8.5, and Joseki 3.4.2.

                                     

                                    Thanks,

                                     

                                    Zhe Wu

                                    1 2 Previous Next