You're almost there! Please answer a few more questions for access to the Applications content. Complete registration
Interested in joining? Complete your registration by providing Areas of Interest here. Register

What is the maximum size for a CSV file to upload from FDMEE to EPBCS in Data Load Rule?

Yaniel Delgado
Yaniel Delgado Green Ribbon
edited Dec 7, 2018 8:31PM in Planning 1 comment
Tagged:

Answers

  • Peter Nitschke-113632
    Peter Nitschke-113632 Silver Medal
    edited Dec 13, 2018 8:41PM

    Hey Yaniel, 

    To the best of my knowledge there isn't a 'maximum' size for a CSV file as such - you're more likely to hit a performance problem in the actual loading process once the data volume gets too big. 

    For an example, we tested the following loading directly to Data Management in PBCS:

    Row Count: 1m Rows, 25 columns

    Size: 110mb

    Mapping: Mostly #SQL mapping 

    Time Taken: 4 mins to upload, 4 mins to map, 1 min to load (we had a 10 min goal)

    We did have an issue with EPMAutomate timing out and had to change to the REST API to upload the file (note this is likely fixed with a recent EPMAutomate update). The workbench performance was reasonable poor - slow to open and slow to filter (circa 20-30 secs to refresh the page) because of the volume of data, but still reasonably manageable. 

    If you're going with much greater volume than that I would recommend looking at using FDMEE on-premises and connecting directly to the source system - using SQL queries on the source side to split the loads up as needed. 

    Cheers

    Pete