Categories
Prebuilt Fusion Data Pipeline Refresh Times Optimization
Hi Team,
The Oracle Managed Fusion data pipeline is taking a very long time around 4hrs 30mins for the incremental refresh every night, is there a way we can optimize this schedule by considering any key factors. This is causing daily data load delays into our business system.
Please let us know any feasible approaches and solutions for this scenario.
Comments
-
This is an issue I'm also facing into at the moment. The current daily refresh is taking around 8 hours in our UAT environment.
Are there any knowledge articles available surrounding best practice for data load scheduling? For example, for ERP Analytics, would it be wise to use frequent data refresh on select GL and CL _CF tables to obtain financial transaction listings regularly with a weekly refresh of all tables scheduled around weekends? Would this improve performance and timeliness of data?
0 -
Yes that is definitely a good approach to consider,But having said that i see you are refreshing UAT on weekends which is good,and 8 hours for that is not bad because the data load is too big.But if you are looking to avoid that as well then yeah you can observe and schedule the high load modules for frequently,Which will reduce the load on the Pipeline.
0 -
We have an various factors which will effect the data pipeline processing timelines. Refer below -
FAW: Data Load Refresh Performance Guidelines [Video Content] --> PNEWS1104
—> About Data Refresh Performance
Regards,
Arjun
0 -
I have reviewed that article a few times.
You may want to schedule the data load to run during off-peak hours, for example, run initial full warehouse loads during weekends and incremental loads during weeknights to ensure that users aren't impacted.
Is this referring to having a weekly refresh in the main pipeline parameters and utilising frequent refresh tables during the week?
0
