FAW Data Augmentation - "Versioned Dataset" Option

D.Angle

Hi,

We are working with the FAW data augmentation interface and noticed an option when creating a new fact or dimension to make the data set Versioned.

I was trying to understand the benefits of the "Versioned Dataset" option being checked.

So when this box isn't checked, the incremental/primary keys are used as defined - but when this box is checked - it just appends the data? The description is kind of vague - this "enables full load of the source table data every time "

Sinduja Sekar-Oracle

Hi @User_9STVF

Thank you for posting in Oracle Analytics Communities. As described in the FDI guide, the Versioned Dataset feature enables to run the corresponding Data Augmentation pipeline as a full load every time. This means the data in the target table will be truncated and reloaded every time.

This feature is helpful in cases where there are deletes in the source. Because if a row is deleted in source, it cannot be tracked in incremental runs and hence the record will still exist in the warehouse table. This causes data inconsistencies. Enabling this checkbox will truncate and reload the warehouse table with each run.

Another possible use is when there are no last updated date (LUD) columns or if there are no updates done to LUD columns when there is a change in source record. This feature helps to maintain latest data in warehouse.

Please also note that care should be taken with highly increasing data volumes as it might have an impact in data refresh times.

D.Angle

This was a helpful response, Sinduja. Thank you.

Yogesh Yadav

@D.Angle @Sinduja Sekar-Oracle do we get last update date or Last refresh date column added to the target table to check when the last data augmentation pipeline ran.

P.S: there is no such column in source table and assumption is, it might get added as a pipeline/augmentation feature.