This discussion is archived
4 Replies Latest reply: Oct 15, 2013 12:55 AM by 886365 RSS

How to reduce the file loading time in FDM?

886365 Newbie
Currently Being Moderated

How to optimize the loading process in FDQM ?


When I'm loading the data files to my Essbase target, the files for which don't have mappings tables are taking very less time. Whereas the files for which mapping tables (Explicit mapping) are there, they are taking high time to load to target. Earlier they used to take 3mnts time for each file, but now they are taking around 10mnts


Can anyone pls help me how can I optimize this loading process for the files which use mapping tables in fdm.


Thanks in advance,

-Krish

  • 1. Re: How to reduce the file loading time in FDM?
    tomblakeley Newbie
    Currently Being Moderated

    Hi Krish,

     

    Are you noticing performance degradation? Or is a location with mappings just "slower"?

     

    The mapping process is going to increase the time it takes to load data - but you can think about the following things:

     

    Order of mappings (put the most common mappings at the top of the order)

     

    Order of dimensions - is mapping one dimension dependent on the mapping in another?

     

    Has the volume of data increased?

  • 2. Re: How to reduce the file loading time in FDM?
    user10757003 Journeyer
    Currently Being Moderated

    Are you talking about just the load step taking longer, or the full import / validate / export / load process?

     

    1. If the latter, you can check the FDM log to see a breakdown of times for each step and for each dimension mapping table. This might help identify the problem point.

     

    2. If the former, have you optimised the dimension order in FDM? (I think the information on this is in the adaptor readme doc). Also, are you using different locations, and if so  are you using different load rules or has the adaptor export / load script been customised and if so is the customisation different for each location? 

  • 3. Re: How to reduce the file loading time in FDM?
    886365 Newbie
    Currently Being Moderated

    Hi Tom,

    - I'm noticing performance degradation

    -Volume of the data is same like earlier - ~250KB. (Also, checked everything... dimensions order and common mappings..everything is fine as earlier)

     

    Thanks!

  • 4. Re: How to reduce the file loading time in FDM?
    886365 Newbie
    Currently Being Moderated

    Hi user10757003,

     

    - I'm talking about all the 3 fish automated process. (Import, validate, export).

    - In log, I could see these details for one file (JAN file)

      Run Begun Time: 02:01:53

      Run Ended: 02:01:54

      Elapsed time was: 00:00:0.90

      CPU time was: 00:00:0.31

    2:13:36 - AMFDM load script started

    2:13:36 - Load script executed.

     

    I'm using a single location and one load script here.

     

    But, overall process is taking long time. Also, Is this process depends on Remote server Infrastructure (RAM/CPU performance) ? Because, the same files used to load in 3mnts earlier, but not 10mnts.

     

    Thanks !

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points