Impact on BICC Extract Prefrenences Configuration on FDI Data Augumentation

Question

We have updated the Prune Time settings to 120 instead of the default setting of 1440 in our BICC console. We also have some data augmentations on certain PVO's. Does the BICC Extract Preference setting have any impact on the PVO defined in FDI.

Appreciate any inputs on this.

RVohra · Answer

@fusionenthu , as per my understanding, prune time does not need to match hourly frequency

* It must protect daily + retry + ingestion latency so with mixed schedules 1440 minutes is ideal

* Anything ≤ 240 minutes is unsafe for FDI + augmented PVOs. Hope it helps!

fusionenthu · Answer

@RVohra @Bhaskar Konar Thanks for you responses. Appreciate it. I have a follow up question. I did have some BICC Jobs that run Daily and some that run Hourly. What would the the ideal prune time for such scenario?

RVohra · Answer

Hi @fusionenthu,

Welcome to the Oracle Analytics Community! Thanks for the Question.

The BICC Prune Time setting only impacts incremental extract data window lookback i.e., how far back in time the delta extract should pull changes (relative to the last extract date). It does not modify the PVO SQL definitions themselves or how FDI treats/view those PVOs.  There is no linkage between extract preferences and the FDI augmentation logic.

Here is the oracle doc about prune time behavior in BICC ..

To ensure data dependencies across objects are maintained during incremental extracts, you can set a prune time that determines how long before the last extract date to extract data from. The default, 1,440 minutes or 24 hours, ensures a look-back that works best for extracts with daily or higher reoccurrence. You should adjust prune time to a smaller window if your extracts are scheduled more frequently or if any downstream system can handle objects extracted in any order. In general, use prune time in accordance with the frequency of your extracts. If you run extracts hourly, set the prune time to 60 minutes. If you run daily, leave it as the default of 1,440 minutes. If you don't need overlapping data to manage early or late dependencies across data stores, you can set the prune time to zero.

Examples:

• With Pruning Time set to the default of 1440 minutes or 24 hours, a daily extract has 48 hours of data because it includes data from 24 hours before the last extract date.

a. An extract runs on 7/28, so the last extract date is updated to 7/28.

b. Another extract runs 24 hours later on 7/29. It would filter for data from 7/27 forward, representing the last extract date of 7/28 minus prune time of 24 hours. The extract would have data for 7/27 - 7/29, or 48 hours.

• If Pruning Time is set to zero, a daily extract has 24 hours of data.

a. An extract runs on 7/28, so the last extract date is updated to 7/28.

b. Another extract runs 24 hours later on 7/29. It would filter for data from 7/28 forward, representing the last extract date of 7/28 minus prune time of zero hours. The extract would have data for 7/28 - 7/29, or24 hours.

Hope it helps!