nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Oracle Communities

Oracle Analytics and AI

Child Item

Prebuilt Fusion Data Pipeline Refresh Times Optimization

akhilsaikaza

Hi Team,

The Oracle Managed Fusion data pipeline is taking a very long time around 4hrs 30mins for the incremental refresh every night, is there a way we can optimize this schedule by considering any key factors. This is causing daily data load delays into our business system.

Please let us know any feasible approaches and solutions for this scenario.

Find more posts tagged with

refresh

FDI

Frequent Data Refresh (FDR)

Fusion Data Intelligence

Fusion AI Data Platform

Comments

Rob Toogood

This is an issue I'm also facing into at the moment. The current daily refresh is taking around 8 hours in our UAT environment.

Are there any knowledge articles available surrounding best practice for data load scheduling? For example, for ERP Analytics, would it be wise to use frequent data refresh on select GL and CL _CF tables to obtain financial transaction listings regularly with a weekly refresh of all tables scheduled around weekends? Would this improve performance and timeliness of data?

akhilsaikaza

Yes that is definitely a good approach to consider,But having said that i see you are refreshing UAT on weekends which is good,and 8 hours for that is not bad because the data load is too big.But if you are looking to avoid that as well then yeah you can observe and schedule the high load modules for frequently,Which will reduce the load on the Pipeline.

Mallikarjuna Kuppauru-Oracle

Hi @akhilsaikaza

We have an various factors which will effect the data pipeline processing timelines. Refer below -

FAW: Data Load Refresh Performance Guidelines [Video Content] --> PNEWS1104

https://docs.oracle.com/en/cloud/saas/analytics/24r4/fawag/data-refresh-performance.html —> About Data Refresh Performance

Regards,

Arjun

Rob Toogood

https://community.oracle.com/products/oracleanalytics/discussion/comment/61280#Comment_61280

I have reviewed that article a few times.

You may want to schedule the data load to run during off-peak hours, for example, run initial full warehouse loads during weekends and incremental loads during weeknights to ensure that users aren't impacted.

Is this referring to having a weekly refresh in the main pipeline parameters and utilising frequent refresh tables during the week?