Oracle Communities

Oracle Analytics and AI

Child Item

Is there a way to extend the data retention period in FDI for longer than 3 months?

We would like a way to retain data for longer than 3 months in FDI. We do understand that this data is only wiped out if there is a refresh done of the environment, but there are concerns with wiping out some data that is needed for long-term reporting.

Find more posts tagged with

Fusion AI Data Platform

Accepted answers

Sumanth V -Oracle

@Kevin Riedy - Yes, the FDI pipeline pulls data based on the Initial Extract Date. If data from that time is unavailable in the source, it will pull from the oldest available date[source's data retention date]. Example if Initial Extract Date is set to 1/1/2000 and data available in source is from 1/1/2001. It gets all data from 1/1/2001 to DataWarehouse.

All comments

Sumanth V -Oracle

@Kevin Riedy -

There is no retention period defined in FDI. It will load all data starting from the Initial Extract Date set in the pipeline parameters. Even after a data refresh, the full load will begin from the Initial Extract Date and continue up to the current date.

Reference: https://docs.oracle.com/en/cloud/saas/analytics/24r3/fawag/set-pipeline-parameters.html

Kevin Riedy

Hi @Sumanth V -Oracle thanks for the response. Any data retention limitation is from the source (Oracle Cloud), correct?

So, FDI will retain all of the data that has been loaded unless a full refresh is performed, at which point, it would pull in all data back to the initial extract date OR the source's data retention date (whichever is most recent)?

Sumanth V -Oracle

Kevin Riedy

Hi @Sumanth V -Oracle thanks for clarifying. So, just to confirm, as long as we do not perform another full load or a refresh, our data will stay there indefinitely. However, if we do perform a full load, or refresh, the FDI will load in data back to the initial extract date of the source application (so, this would be 3 months worth of data, since Oracle Cloud only retains 3 months of data).

Kevin Riedy

@Sumanth V -Oracle An additional follow up question: is there any way to preserve data in FDI so that it is not wiped out during a refresh? Or is there a way to store it elsewhere?

Sumanth V -Oracle

@Kevin Riedy - During the initial configuration of FDI only the Initial Extract Date is set and Full load is run to get the data from source. Post that data remains in data warehouse indefinitely. Yes, upon refresh and Full load request on demand data is wiped and reloaded. There is no way this can be avoided.

Prabha-Oracle

@Sumanth V -Oracle Is there a way to "purge" data to keep only a defined time-span, as a "work-around" for a "retention policy" as requested?