Support BI CC Downloads in Parquet format

Pam Koertshuis

Organization Name: Gemeente Rotterdam / City of Rotterdam

Description (Required): We have an external datawarehouse and need to export the data from SaaS to that datawarehouse. We currently use the BI CC programs to export the data in csv (zipped) to file storage and use dbms_cloud to read the files from there. However, BI CC only supports zip (not supported by dbms_cloud) and gzip.

We can use the gzip, but we would like the files to be parquet so we do not have to specify the schema in the create external table command.

Use Case and Business Need (Required): Parquet is more flexible for downloading the files in a virtual manner.

Ryan Waller

We have a similar need to be able to define an export via OTBI and/or BI Publisher and export the format of the data via Parquet format. The ideal situation is that the data types and structure of the data would be known at the time the Parquet file is created (since the source system knows what these fields are at the time we created the extract). This would eliminate the need for additional integration flows/ETL pipelines to be created just to copy/convert a file into a parquet format and then have to re-define or infer the data types, etc in that process when the source system already knows this information. This would also reduce our logic and validation processes that need to be done when we ingest a file that might have various delimiters (i.e. Pipe, Comma, Tab, etc).