Oracle Analytics Cloud and Server

Products Banner

BICC and external storage

Received Response
364
Views
4
Comments

Summary

Need recommendation how to automate data flow from BICC to DW

Content

Was researching what options are available to automate data flow from BICC to corporate DW

Ended up with 3 distinct options:

  1. Use seeded capability to extract PVO to UCM. Then build custom bash script to download extracts and upload to corporate FTP, then use standard data loading to DW
  2. Configure external storage in BICC and leverage native capability to pull data to DW. for example Amazon (but read that BICC can only extract PVO to UCM or OCI Storage Container; see https://cloudcustomerconnect.oracle.com/posts/7383390b51?commentId=273695#273695
  3. Use Oracle ODI to bridge UCM and corporate DW (based on https://cloudcustomerconnect.oracle.com/posts/de2d544b6d)

 

Anything else you can suggest? If no please share how above options may rank from most favorite to least

 

Answers

  • User_J3T0A
    User_J3T0A ✭✭✭

    Hey Roman,

    I can share my experience with you.

    I have tried below 2 distinct options.

    Use seeded capability to extract PVO to UCM. Then build custom bash script to download extracts and upload to corporate FTP, then use standard data loading to DW

    • This option is really good when data size is comparatively lesser and PVO row count is not too high,
    • Additionally, you can twist above solution, use this - PVO to UCM (down load files using BIACM - SaaS component) --> use OAC data replication utility and have target Oracle ADW to store all this and you can do it on incremental basis as well, just like any other ETL tool, pretty clean approach. I have done it for customer demo
    • this comes with OCPU cost, as there are limits to number of PVOs to run in parallel based on OAC horsepower/OCPU's   
    1. Use Oracle ODI to bridge UCM and corporate DW (based on https://cloudcustomerconnect.oracle.com/posts/de2d544b6d)
      • this is Oracle standard OBIA approach for on-prem solution. Java code with extract files from UCM downloaded by BIACM (SaaS) and ODI will pickup all files and load into DW.
      • this approach is really useful when PVO count and Data volume is really high and less frequent.

    Hope this helps.

  • thank you! this is interesting.

    is "less frequent" comment relates to ODI licensing cost? or just long execution time for large volumes?

  • Hi Roman,

     

    Less frequent has nothing to do with ODI licensing but more towards getting large volume of data, execution time for multiple layers, SDS - SDE - SIL - PLP. These option to be used for analytical purpose rather than transactional.

  • Hi Roman,

    I hope this clarifies details/query you had.