Ability to Append new data to existing data set in OAS using data flow

Angel Shipp

Organization Name

Verizon

Description

We would like to have the ability to append new data to existing data set storage in OAS. This could be added as an option when saving data during data flow creation.

Note: This idea is specific to appending new data to dataset storage not database connection. Majority of our users do not have write access to source/target databases and some use flat files from feeds or other sources that they will need to append to dataset storage in OAS DV during data flow creation.   

Use Case and Business Need

When updating data sets using data flows, we do not always want to overwrite data on existing data sets. Sometimes we have to append new data to existing data sets. This way, we are only having to pull new data from external sources and append them to existing data set storage instead of pulling all data from external sources and overwriting the data set every time the data flow runs.

More details

The option to append data to data set storage currently does not exist. The only option available is to overwrite data set storage. 

Original Idea Number: 5027728761

Pravin Janardanam-Oracle

Hi Angel,

 

Thank you for sharing this idea! Appending data to existing data set is available for database targets. The source and the target data should be database based. You must set New Data Identifier for the source data set to identify the column for incremental load. When using a data set with New Data Identifier set, you will see the option to Append Data in Save Data Set. Incremental append to a data set based on data set storage is not feasible now.

 

You can find more details in this blog. Perform Incremental Data Loads in Oracle Analytics Cloud | Oracle Analytics Blog

 

Regards,

Pravin

Angel Shipp

Hi Pravin,

Thank you for your feedback.

Right, the ability to append to data sets is available for database targets/ database connection only not for dataset storage. This idea is specific to having the ability to append/add new data to existing dataset storage because users do not always have write access to database targets or sometimes users use flat files from a feed that they need to append to dataset storage in OAS DV. Having this flexibility to append. Is this idea possible?