6 Replies Latest reply: Jan 3, 2013 5:47 PM by Davin Fifield-Oracle RSS

    Batch inferred instance to database

    Paul Fowler
      How do we write the results of an inferred instance to a database table when using the batch processor?

      Is there an example? We are not sure how the database table for the inferred entity is supposed to be designed... Primary key, etc...
        • 1. Re: Batch inferred instance to database
          Paul Fowler
          More information... I have gotten the following error:

          Worker 1 - Unexpected exception occurred: An inferred relationship cannot be set by a user
          java.lang.IllegalStateException: An inferred relationship cannot be set by a user
               at com.oracle.determinations.engine.local.LocalRelationship.setInternal(LocalRelationship.java:818)
               at com.oracle.determinations.engine.local.LocalRelationship.setInstance(LocalRelationship.java:769)
               at com.oracle.determinations.batch.dataset.provider.BatchLoader.processRelationshipReferences(BatchLoader.java:302)
               at com.oracle.determinations.batch.dataset.provider.BatchLoader.process(BatchLoader.java:82)
               at com.oracle.determinations.batch.dataset.DataSetProvider.prepareCaseBlock(DataSetProvider.java:156)
               at com.oracle.determinations.batch.dataset.DataSetProvider.readNextCase(DataSetProvider.java:71)
               at com.oracle.determinations.batch.BatchProcessor.execute(BatchProcessor.java:279)
               at com.oracle.determinations.batch.CmdLine.main(CmdLine.java:106)
          • 2. Re: Batch inferred instance to database
            Matt Sevin-Oracle
            From the 10.4.2 OPA Developer's Guide ... Sorry :-(

            Include inferred entities in Batch Processor output

            The Batch Processor supports the ability to include inferred entities as output when reading from and writing to CSV data files. Inferred entities are not supported for other input or output types, such as database input or test script output. Output includes the containment relationships of the inferred entities only; it does not include any other inferred relationship details.

            More details: http://docs.oracle.com/html/E38272_01/Content/Batch%20Processor/Batch_Processor_Output.htm#Include_inferred_entities_in_Batch_Processor_output
            • 3. Re: Batch inferred instance to database
              Davin Fifield-Oracle
              Matt's correct - inferred entity instances cannot currently be written to database output. We plan to address this in the upcoming 10.4.3 release.
              • 4. Re: Batch inferred instance to database
                Paul Fowler
                It just means we get a little creative.

                I can seed the inferred database table with key data first. Basically, the rules for creation of an inferred entity are fairly simply and I can apply those rules ahead of time in seeding the database table with rows. No biggie.
                • 5. Re: Batch inferred instance to database
                  Paul Fowler
                  One more note.

                  Our LARGEST challenge by far using the batch processor is the "understanding" of performance related issues...

                  It appears a little bit too black boxish.

                  I think the real problems occur in the relationships between views. We have 1,000,000 records in each view. I can tell the batch processor to only process 100 rows, but I think it limits the "global" view to 100 rows and then performs a very slow join against multiple 1000000 row views for child data. I can't see the sql and there is very little tunning assistance provided.

                  As such 100 rows can take an hour to process unless I put temporary where clauses on all my child views. Of course, this also leads to a desire to add filters to the configuration. I would like to tell OPA to only process global (or child records) where a base attribute is a particular value.

                  I guess I am asking for any hints or advise in this forum.
                  • 6. Re: Batch inferred instance to database
                    Davin Fifield-Oracle
                    Joins will be fast provided they are on indexed fields. My DBA skills are pretty shaky, but you should be able to create an index on the join columns of the view: that would certainly speed things up significantly. Also, most databases come with query optimizing tools that should be able to give you some hints.

                    We've added to our backlog the feature request for verbose debug logging of the SQL queries being used by the batch processor: it may even make it into 10.4.3.