Forum Stats

  • 3,851,971 Users
  • 2,264,055 Discussions
  • 7,904,922 Comments

Discussions

Data Loading Error

AbanoubZaki
AbanoubZaki Member Posts: 144 Red Ribbon
edited Sep 4, 2019 10:58AM in Financial Data Management

Dears,

I have the below error while exporting data to target application, any advise:

2019-09-04 13:42:10,374 INFO [AIF]: FDMEE Process: 309, Log Level: 4

2019-09-04 13:42:10,375 INFO [AIF]: Location : WORKFORCE_LOC (Partitionkey:15)

2019-09-04 13:42:10,375 INFO [AIF]: Period Name : Jul-19 (Period Key:7/31/19 12:00 AM)

2019-09-04 13:42:10,375 INFO [AIF]: Category Name: Actual (Category key:5)

2019-09-04 13:42:10,375 INFO [AIF]: Rule Name : Workforce_Rule (Rule ID:8)

2019-09-04 13:42:11,607 INFO [AIF]: Log File Encoding: UTF-8

2019-09-04 13:42:12,721 INFO [AIF]: -------START IMPORT STEP-------

2019-09-04 13:42:12,923 INFO [AIF]: File Name: AAW1.csv

2019-09-04 13:42:13,058 INFO [AIF]: EPMFDM-140274:Message - LOADID:PARTKEY:CATKEY:RULEID:CURRENCYKEY:FILEPATH:IMPGROUPFILETYPE::309;15:5:8:[NONE]:/u03/inbox/:MULTIDATA

2019-09-04 13:42:13,062 INFO [AIF]: EPMFDM-140274:Message - File CharEncoding as per settings: UTF-8

2019-09-04 13:42:13,062 INFO [AIF]: EPMFDM-140274:Message - ImportTextData - Start

2019-09-04 13:42:13,142 INFO [AIF]: EPMFDM-140274:Message - Initialize error maps

2019-09-04 13:42:13,142 INFO [AIF]: EPMFDM-140274:Message - File Name AAW1.csv

periodKey2019-07-31

2019-09-04 13:42:13,400 INFO [AIF]: EPMFDM-140274:Message - PROCESS ID: 309

PARTITIONKEY: 15

IMPORT GROUP: Workforse_Data

FILE TYPE: FIXED

SOURCE FILE: AAW1.csv

PROCESSING CODES:

-----------------------------------------------------------------------------------

BLANK............. Line is blank or empty.

NN................ Non-Numeric, Amount field contains non numeric characters.

TC................ Type Conversion, Amount field could not be converted to a number.

ZP................ Zero Suppress, Amount field contains a 0 value and zero suppress is ON.

SKIP FIELD.............. SKIP field value was found

NULL ACCOUNT VALUE.............. Account Field is null

SKIP FROM SCRIPT.............. Skipped through Script

HEADER ROW.............. Header Row

ERROR_INVALID_PERIOD.............. Invalid period specified in the file.

[HEADER_ROW] Business Unit,Employee,Date Join,Job Desc,Period,7020,7030,7070,7076,7035,7077,8030,8035,8180,8200

2019-09-04 13:42:13,655 INFO [AIF]: EPMFDM-140274:Message - Rows Loaded: 6944

Rows Rejected: 0

2019-09-04 13:42:13,656 INFO [AIF]: EPMFDM-140274:Message - ARCHIVE MODE: Copy

2019-09-04 13:42:13,656 INFO [AIF]: EPMFDM-140274:Message - Start archiving file:

2019-09-04 13:42:13,657 INFO [AIF]: EPMFDM-140274:Message - Archive file name: 30920190731.csv

2019-09-04 13:42:13,659 INFO [AIF]: EPMFDM-140274:Message - Deleting the source file: AAW1.csv

2019-09-04 13:42:13,659 INFO [AIF]: EPMFDM-140274:Message - File not deleted: /u03/inbox/AAW1.csv

2019-09-04 13:42:13,669 INFO [AIF]: EPMFDM-140274:Message - ImportTextData - End

2019-09-04 13:42:13,669 INFO [AIF]: EPMFDM-140274:Message - Total time taken for the import in ms = 607

2019-09-04 13:42:13,751 INFO [AIF]:

Import Data from Source for Period 'Jul-19'

2019-09-04 13:42:13,796 INFO [AIF]: Executing delete statement:

2019-09-04 13:42:13,796 INFO [AIF]: Successfully executed delete statement

2019-09-04 13:42:13,816 INFO [AIF]:

Map Data for Period 'Jul-19'

2019-09-04 13:42:13,823 INFO [AIF]:

Processing Mappings for Column 'ACCOUNT'

2019-09-04 13:42:13,895 INFO [AIF]: Data Rows Updated by Location Mapping EXPLICIT: 3614

2019-09-04 13:42:13,936 INFO [AIF]: Data Rows Updated by Location Mapping 'Account_all' (LIKE): 3330

2019-09-04 13:42:13,939 INFO [AIF]:

Processing Mappings for Column 'ENTITY'

2019-09-04 13:42:14,128 INFO [AIF]: Data Rows Updated by Location Mapping 'BU_RULE' (LIKE): 6944

2019-09-04 13:42:14,130 INFO [AIF]:

Processing Mappings for Column 'UD1'

2019-09-04 13:42:14,397 INFO [AIF]: Data Rows Updated by Location Mapping 'Version_rule' (LIKE): 6944

2019-09-04 13:42:14,399 INFO [AIF]:

Processing Mappings for Column 'UD2'

2019-09-04 13:42:14,548 INFO [AIF]: Data Rows Updated by Location Mapping 'Detail' (LIKE): 6944

2019-09-04 13:42:14,551 INFO [AIF]:

Processing Mappings for Column 'UD3'

2019-09-04 13:42:14,916 INFO [AIF]: Data Rows Updated by Location Mapping 'EMP_RULE' (LIKE): 6944

2019-09-04 13:42:14,919 INFO [AIF]:

Processing Mappings for Column 'UD4'

2019-09-04 13:42:15,057 INFO [AIF]: Data Rows Updated by Location Mapping 'sku_rule' (LIKE): 6944

2019-09-04 13:42:15,060 INFO [AIF]:

Processing Mappings for Column 'UD5'

2019-09-04 13:42:15,211 INFO [AIF]: Data Rows Updated by Location Mapping 'View_rule' (LIKE): 6944

2019-09-04 13:42:15,380 INFO [AIF]: Data Rows Updated by Default Data Mapping: 6944

2019-09-04 13:42:15,389 INFO [AIF]:

Stage Data for Period 'Jul-19'

2019-09-04 13:42:15,391 INFO [AIF]: Number of Rows deleted from TDATAMAPSEG: 9

2019-09-04 13:42:15,392 INFO [AIF]: Number of Rows inserted into TDATAMAPSEG: 9

2019-09-04 13:42:15,393 INFO [AIF]: Number of Rows deleted from TDATAMAP_T: 9

2019-09-04 13:42:15,396 INFO [AIF]: Executing delete statement:

2019-09-04 13:42:15,661 INFO [AIF]: Successfully executed delete statement

2019-09-04 13:42:15,661 INFO [AIF]: Number of Rows deleted from TDATASEG: 6944

2019-09-04 13:42:15,662 INFO [AIF]: Number of Rows deleted from TDATAMEMOITEMS: 0

2019-09-04 13:42:15,663 INFO [AIF]: Number of Rows deleted from TDATAARCHIVE: 0

2019-09-04 13:42:15,663 INFO [AIF]: Executing insert statement:

2019-09-04 13:42:16,099 INFO [AIF]: Successfully executed insert statement

2019-09-04 13:42:16,099 INFO [AIF]: Number of Rows inserted into TDATASEG: 6944

2019-09-04 13:42:16,099 INFO [AIF]: Executing delete statement:

2019-09-04 13:42:16,349 INFO [AIF]: Successfully executed delete statement

2019-09-04 13:42:16,349 INFO [AIF]: Number of Rows deleted from TDATASEG_T: 6944

2019-09-04 13:42:16,389 INFO [AIF]: -------END IMPORT STEP-------

2019-09-04 13:42:16,419 INFO [AIF]: -------START VALIDATE STEP-------

2019-09-04 13:42:16,447 INFO [AIF]:

Validate Data Mappings for Period 'Jul-19'

2019-09-04 13:42:16,461 INFO [AIF]: Total Data Rows available for Export to Target: 6944

2019-09-04 13:42:16,462 INFO [AIF]:

Validate Data Members for Period 'Jul-19'

2019-09-04 13:42:16,465 INFO [AIF]: Total Data Rows available for Export to Target: 6944

2019-09-04 13:42:16,487 INFO [AIF]: -------END VALIDATE STEP-------

2019-09-04 13:42:16,541 INFO [AIF]: -------START EXPORT STEP-------

2019-09-04 13:42:16,580 INFO [AIF]: EssbaseService.exportData - START

2019-09-04 13:42:16,659 INFO [AIF]: cloudServiceType: PBCSE, Resolved user: [email protected], planningUserName: [email protected]

2019-09-04 13:42:16,705 INFO [AIF]: Creating data file: /u03/inbox/outbox/FPnR_AAW_309.dat

2019-09-04 13:42:16,902 INFO [AIF]: Data file creation completed: 6944

2019-09-04 13:42:16,902 INFO [AIF]: EssbaseService.exportData - END (true)

2019-09-04 13:42:16,990 INFO [AIF]: EssbaseService.loadData - START

2019-09-04 13:42:17,062 INFO [AIF]: cloudServiceType: PBCSE, Resolved user: [email protected], planningUserName: [email protected]

[Wed Sep 04 13:42:18 UTC 2019]Batch Mode (/B) turned off (/-B) for strict data loads or save of metadata specified as false (/-IMD).

[Wed Sep 04 13:42:18 UTC 2019]Saving metadata on data loads (/IMD)

Property file arguments: /DF:DD-MM-YYYY /DL:comma /I:***FPnR_AAW_309.dat /TR

No arguments were derived from the command line.Submitted (merged) command line:

/DF:DD-MM-YYYY /DL:comma /I:***FPnR_AAW_309.dat /TR

[Wed Sep 04 13:42:18 UTC 2019]Successfully located and opened input file.

[Wed Sep 04 13:42:18 UTC 2019]Input file being read with UTF-8 encoding.

[Wed Sep 04 13:42:18 UTC 2019]Date format pattern "DD-MM-YYYY" specified for date data type loading.

[Wed Sep 04 13:42:18 UTC 2019]Header record fields: Driver Member, Point-of-View, Data Load Cube Name, Value

[Wed Sep 04 13:42:18 UTC 2019]Located and using "OFADataLoadDimension" dimension for loading data in "FPnR_AAW" application.

[Wed Sep 04 13:42:18 UTC 2019]Parent-child ordering option (/H switch) will not be performed: this option is not available for the "OFADataLoadDimension" dimension.

[Wed Sep 04 13:42:18 UTC 2019]Input file member load ordering option (/O switch) will not be performed: this option is only available for (non /IE) import operations.

[Wed Sep 04 13:42:18 UTC 2019]HspOutlineLoad::dateFormatSpecified is true, SessionHalDateFormat stored on session: dd/MM/yy, sessionId: 1929669934

[Wed Sep 04 13:42:18 UTC 2019] [393] Ignoring data record 393, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

[Wed Sep 04 13:42:18 UTC 2019] [834] Ignoring data record 834, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

[Wed Sep 04 13:42:18 UTC 2019] [1072] Ignoring data record 1072, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

[Wed Sep 04 13:42:18 UTC 2019] [1684] Ignoring data record 1684, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

[Wed Sep 04 13:42:18 UTC 2019] [2067] Ignoring data record 2067, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

[Wed Sep 04 13:42:18 UTC 2019] [2809] Ignoring data record 2809, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

[Wed Sep 04 13:42:18 UTC 2019] [2962] Ignoring data record 2962, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

[Wed Sep 04 13:42:18 UTC 2019] [5446] Ignoring data record 5446, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

Outline data store load process finished. 6944 data records were read, 6945 data records were processed, 6937 were accepted for loading (verify actual load with Essbase log files), 8 were rejected.

[Wed Sep 04 13:42:19 UTC 2019]Outline data store load process finished. 6944 data records were read, 6945 data records were processed, 6937 were accepted for loading (verify actual load with Essbase log files), 8 were rejected.

2019-09-04 13:42:19,415 ERROR [AIF]: The data load process has failed.

2019-09-04 13:42:19,417 INFO [AIF]: EssbaseService.loadData - END (false)

2019-09-04 13:42:19,422 FATAL [AIF]: Error in CommData.loadData

Traceback (most recent call last):

  File "<string>", line 4025, in loadData

RuntimeError: false

2019-09-04 13:42:19,484 FATAL [AIF]: Error in COMM Load Data

2019-09-04 13:42:19,487 INFO [AIF]: FDMEE Process End, Process ID: 309

AbanoubZaki

Best Answer

  • SH_INT
    SH_INT Member Posts: 3,192 Bronze Crown
    edited Sep 4, 2019 10:35AM Answer ✓

    You probably have a comma i.e. your delimiter in one of the fields you are exporting for the records indicated. Check the dataset in FDMEE and identify those records. Then either change the export delimiter or fix the source data to remove commas

    AbanoubZakiAbanoubZaki

Answers

  • SH_INT
    SH_INT Member Posts: 3,192 Bronze Crown
    edited Sep 4, 2019 10:11AM

    Your answer is in the log:

    [Wed Sep 04 13:42:18 UTC 2019] [393] Ignoring data record 393, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 13:42:18 UTC 2019] [834] Ignoring data record 834, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 13:42:18 UTC 2019] [1072] Ignoring data record 1072, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 13:42:18 UTC 2019] [1684] Ignoring data record 1684, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 13:42:18 UTC 2019] [2067] Ignoring data record 2067, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 13:42:18 UTC 2019] [2809] Ignoring data record 2809, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 13:42:18 UTC 2019] [2962] Ignoring data record 2962, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 13:42:18 UTC 2019] [5446] Ignoring data record 5446, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    Outline data store load process finished. 6944 data records were read, 6945 data records were processed, 6937 were accepted for loading (verify actual load with Essbase log files), 8 were rejected.

    [Wed Sep 04 13:42:19 UTC 2019]Outline data store load process finished. 6944 data records were read, 6945 data records were processed, 6937 were accepted for loading (verify actual load with Essbase log files), 8 were rejected.

    It even tells you the record numbers of the data causing the issues. Check the records in your .dat file again all the information is in the log:

    Creating data file: /u03/inbox/outbox/FPnR_AAW_309.dat

  • AbanoubZaki
    AbanoubZaki Member Posts: 144 Red Ribbon
    edited Sep 4, 2019 10:25AM

    Thanks SH_INT for your reply,

    but the above record numbers is changing by every load process.

    i couldn't figure out what is exactly the issue, and where can I find it, please check the below

    [Wed Sep 04 14:21:20 UTC 2019] [671] Ignoring data record 671, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 14:21:20 UTC 2019] [1704] Ignoring data record 1704, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 14:21:20 UTC 2019] [2316] Ignoring data record 2316, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 14:21:20 UTC 2019] [3158] Ignoring data record 3158, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 14:21:20 UTC 2019] [3247] Ignoring data record 3247, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 14:21:20 UTC 2019] [4189] Ignoring data record 4189, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 14:21:20 UTC 2019] [5079] Ignoring data record 5079, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

    [Wed Sep 04 14:21:20 UTC 2019] [6530] Ignoring data record 6530, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.

  • SH_INT
    SH_INT Member Posts: 3,192 Bronze Crown
    edited Sep 4, 2019 10:35AM Answer ✓

    You probably have a comma i.e. your delimiter in one of the fields you are exporting for the records indicated. Check the dataset in FDMEE and identify those records. Then either change the export delimiter or fix the source data to remove commas

    AbanoubZakiAbanoubZaki