Parallel loading of semantic data

Hi Fred,

Few questions for you:

1) Roughly how much data (in total number of triples) do you expect to generate using the series of sem_apis.export_rdfview_model calls?

2) Do you have any CLOBs or geospatial data?

3) Are there quads or just triples?

4) Do you expect lots of duplicate triples (more than a million or so) if we combine all the exported data under one database view object and provide that view as input row source to sem_apis.bulk_load_from_staging_table?

I would suggest doing steps 3 and 4 as follows:

Step 3:

---------

alter session force parallel dml parallel <degree>;

alter session force parallel ddl parallel <degree>;

alter session force parallel query parallel <degree>;

exec sem_apis.export_rdfview_model(...);

Step 4 (do it outside of loop -- invoking a single bulk-load for the combined data would allow avoiding incremental index maintenance overheads):

---------

create a view, say stage_view, as UNION ALL of all the staging tables where the RDF triples were exported to.

alter session force parallel dml parallel <degree>;

alter session force parallel ddl parallel <degree>;

alter session force parallel query parallel <degree>;

-- load from stage_view

-- if you expect lots of duplicate triples in the stage_view row source, say more than a million duplicates,

-- then add the following in the flags string below: DEL_BATCH_DUPS=USE_INSERT

exec sem_apis.bulk_load_from_staging_table(..., flags=>' PARSE MBV_METHOD=SHADOW PARALLEL=<degree> ');

If you have any problems, feel free to contact me directly: souripriya dot das at oracle dot com.

Thanks,

- Souri.