-
1. Re: Parallel loading of semantic data
Sdas-Oracle Jun 6, 2017 4:14 PM (in response to 3054700)Hi Fred,
Few questions for you:
1) Roughly how much data (in total number of triples) do you expect to generate using the series of sem_apis.export_rdfview_model calls?
2) Do you have any CLOBs or geospatial data?
3) Are there quads or just triples?
4) Do you expect lots of duplicate triples (more than a million or so) if we combine all the exported data under one database view object and provide that view as input row source to sem_apis.bulk_load_from_staging_table?
I would suggest doing steps 3 and 4 as follows:
Step 3:
---------
alter session force parallel dml parallel <degree>;
alter session force parallel ddl parallel <degree>;
alter session force parallel query parallel <degree>;
exec sem_apis.export_rdfview_model(...);
Step 4 (do it outside of loop -- invoking a single bulk-load for the combined data would allow avoiding incremental index maintenance overheads):
---------
create a view, say stage_view, as UNION ALL of all the staging tables where the RDF triples were exported to.
alter session force parallel dml parallel <degree>;
alter session force parallel ddl parallel <degree>;
alter session force parallel query parallel <degree>;
-- load from stage_view
-- if you expect lots of duplicate triples in the stage_view row source, say more than a million duplicates,
-- then add the following in the flags string below: DEL_BATCH_DUPS=USE_INSERT
exec sem_apis.bulk_load_from_staging_table(..., flags=>' PARSE MBV_METHOD=SHADOW PARALLEL=<degree> ');
If you have any problems, feel free to contact me directly: souripriya dot das at oracle dot com.
Thanks,
- Souri.
-
2. Re: Parallel loading of semantic data
3054700 Jun 6, 2017 6:14 PM (in response to Sdas-Oracle)Hi Souri, thank you for your help.
1) We generate a total of about 140 millions triples
2) There is no CLOBs or geospatial data in this dataset (but might have in the future)
3) Just triples
4) I'm not sure about the number of duplicates but I'd say far less than 1 million.
I'm making some tests here. I got an ORA-13199: Insufficient privilege for using MBV_METHOD=SHADOW option. What privilege is needed?