Incremental Load Bi Apps

user8819264

Hello All,

We have done the BI Apps configuration for FIN, SCM and Proc & Spend. And finished the full load and the data for these modules are populating correctly on Analytics.

The issue i have is, the following loads after the full load is also taking the same time as full load which is about an hour for each module. Just to update, i have not reset my datawarehouse scenario.

Can anyone point me how can i force for an incremental load to reduce the load time.

Thank you for your update!

OBI Apps (BIACM) - 11.1.1.10.2

OBIEE 11.1.1.9.0

Oracle Data Integrator (ODI)- 11.1.1.9.0

Weblogic- 10.3.6

Charles M

Hi -

Based on your versions, there is no need to "force" an incremental load after running the initial full load. The ETL logic will handle this. Why do you think it is doing another full load?

Regards,

Charles

user8819264

The only reason that force me to think is the time it is taking for the incremental load. Full load after resetting the data warehouse scenario is taking about 55 min to load Financial Analytics. The next load soon after the full load, may be after 5 min with almost no data updates on the source system also taking about same 55 min to finish.

Soon after the full load, I checked the table W_ETL_LOAD_DATES and all the TARGET_TABLE_NAME related to Financials exists with latest ETL_LOAD_DATE.

Charles M

Hi,

Its hard to say how long your incremental could/should be taking. Maybe it would be useful to compare the loads in ODI (full vs. inc.). Go to the Operator tab and bring up the Load Plan Executions. Then, drill-down into the different phases/steps.You should be able to see where the time is being spent. Certain tasks will take some amount of time, no matter what. Like, re-building indexes, gather statistics, etc.

Also, how much time are you spending in the PLP? Are there any steps you could remove, possibly?

Note: I am assuming that these subsequent ETL's are incremental.

Regards,

Charles

Judith Ullman

We have the situation for certain source tables where the table does not have a datetime stamp.   Therefore, the ETL treats any load as a full load, as it cannot tell if a record was updated or not.  So one very simple check would be to see if your source does have a datetime stamp that the ETL can use.