Discussions
Categories
- 17.9K All Categories
- 3.4K Industry Applications
- 3.3K Intelligent Advisor
- 65 Insurance
- 536.7K On-Premises Infrastructure
- 138.4K Analytics Software
- 38.6K Application Development Software
- 5.9K Cloud Platform
- 109.5K Database Software
- 17.6K Enterprise Manager
- 8.8K Hardware
- 71.2K Infrastructure Software
- 105.3K Integration
- 41.6K Security Software
Data Loading Error
Dears,
I have the below error while exporting data to target application, any advise:
2019-09-04 13:42:10,374 INFO [AIF]: FDMEE Process: 309, Log Level: 4
2019-09-04 13:42:10,375 INFO [AIF]: Location : WORKFORCE_LOC (Partitionkey:15)
2019-09-04 13:42:10,375 INFO [AIF]: Period Name : Jul-19 (Period Key:7/31/19 12:00 AM)
2019-09-04 13:42:10,375 INFO [AIF]: Category Name: Actual (Category key:5)
2019-09-04 13:42:10,375 INFO [AIF]: Rule Name : Workforce_Rule (Rule ID:8)
2019-09-04 13:42:11,607 INFO [AIF]: Log File Encoding: UTF-8
2019-09-04 13:42:12,721 INFO [AIF]: -------START IMPORT STEP-------
2019-09-04 13:42:12,923 INFO [AIF]: File Name: AAW1.csv
2019-09-04 13:42:13,058 INFO [AIF]: EPMFDM-140274:Message - LOADID:PARTKEY:CATKEY:RULEID:CURRENCYKEY:FILEPATH:IMPGROUPFILETYPE::309;15:5:8:[NONE]:/u03/inbox/:MULTIDATA
2019-09-04 13:42:13,062 INFO [AIF]: EPMFDM-140274:Message - File CharEncoding as per settings: UTF-8
2019-09-04 13:42:13,062 INFO [AIF]: EPMFDM-140274:Message - ImportTextData - Start
2019-09-04 13:42:13,142 INFO [AIF]: EPMFDM-140274:Message - Initialize error maps
2019-09-04 13:42:13,142 INFO [AIF]: EPMFDM-140274:Message - File Name AAW1.csv
periodKey2019-07-31
2019-09-04 13:42:13,400 INFO [AIF]: EPMFDM-140274:Message - PROCESS ID: 309
PARTITIONKEY: 15
IMPORT GROUP: Workforse_Data
FILE TYPE: FIXED
SOURCE FILE: AAW1.csv
PROCESSING CODES:
-----------------------------------------------------------------------------------
BLANK............. Line is blank or empty.
NN................ Non-Numeric, Amount field contains non numeric characters.
TC................ Type Conversion, Amount field could not be converted to a number.
ZP................ Zero Suppress, Amount field contains a 0 value and zero suppress is ON.
SKIP FIELD.............. SKIP field value was found
NULL ACCOUNT VALUE.............. Account Field is null
SKIP FROM SCRIPT.............. Skipped through Script
HEADER ROW.............. Header Row
ERROR_INVALID_PERIOD.............. Invalid period specified in the file.
[HEADER_ROW] Business Unit,Employee,Date Join,Job Desc,Period,7020,7030,7070,7076,7035,7077,8030,8035,8180,8200
2019-09-04 13:42:13,655 INFO [AIF]: EPMFDM-140274:Message - Rows Loaded: 6944
Rows Rejected: 0
2019-09-04 13:42:13,656 INFO [AIF]: EPMFDM-140274:Message - ARCHIVE MODE: Copy
2019-09-04 13:42:13,656 INFO [AIF]: EPMFDM-140274:Message - Start archiving file:
2019-09-04 13:42:13,657 INFO [AIF]: EPMFDM-140274:Message - Archive file name: 30920190731.csv
2019-09-04 13:42:13,659 INFO [AIF]: EPMFDM-140274:Message - Deleting the source file: AAW1.csv
2019-09-04 13:42:13,659 INFO [AIF]: EPMFDM-140274:Message - File not deleted: /u03/inbox/AAW1.csv
2019-09-04 13:42:13,669 INFO [AIF]: EPMFDM-140274:Message - ImportTextData - End
2019-09-04 13:42:13,669 INFO [AIF]: EPMFDM-140274:Message - Total time taken for the import in ms = 607
2019-09-04 13:42:13,751 INFO [AIF]:
Import Data from Source for Period 'Jul-19'
2019-09-04 13:42:13,796 INFO [AIF]: Executing delete statement:
2019-09-04 13:42:13,796 INFO [AIF]: Successfully executed delete statement
2019-09-04 13:42:13,816 INFO [AIF]:
Map Data for Period 'Jul-19'
2019-09-04 13:42:13,823 INFO [AIF]:
Processing Mappings for Column 'ACCOUNT'
2019-09-04 13:42:13,895 INFO [AIF]: Data Rows Updated by Location Mapping EXPLICIT: 3614
2019-09-04 13:42:13,936 INFO [AIF]: Data Rows Updated by Location Mapping 'Account_all' (LIKE): 3330
2019-09-04 13:42:13,939 INFO [AIF]:
Processing Mappings for Column 'ENTITY'
2019-09-04 13:42:14,128 INFO [AIF]: Data Rows Updated by Location Mapping 'BU_RULE' (LIKE): 6944
2019-09-04 13:42:14,130 INFO [AIF]:
Processing Mappings for Column 'UD1'
2019-09-04 13:42:14,397 INFO [AIF]: Data Rows Updated by Location Mapping 'Version_rule' (LIKE): 6944
2019-09-04 13:42:14,399 INFO [AIF]:
Processing Mappings for Column 'UD2'
2019-09-04 13:42:14,548 INFO [AIF]: Data Rows Updated by Location Mapping 'Detail' (LIKE): 6944
2019-09-04 13:42:14,551 INFO [AIF]:
Processing Mappings for Column 'UD3'
2019-09-04 13:42:14,916 INFO [AIF]: Data Rows Updated by Location Mapping 'EMP_RULE' (LIKE): 6944
2019-09-04 13:42:14,919 INFO [AIF]:
Processing Mappings for Column 'UD4'
2019-09-04 13:42:15,057 INFO [AIF]: Data Rows Updated by Location Mapping 'sku_rule' (LIKE): 6944
2019-09-04 13:42:15,060 INFO [AIF]:
Processing Mappings for Column 'UD5'
2019-09-04 13:42:15,211 INFO [AIF]: Data Rows Updated by Location Mapping 'View_rule' (LIKE): 6944
2019-09-04 13:42:15,380 INFO [AIF]: Data Rows Updated by Default Data Mapping: 6944
2019-09-04 13:42:15,389 INFO [AIF]:
Stage Data for Period 'Jul-19'
2019-09-04 13:42:15,391 INFO [AIF]: Number of Rows deleted from TDATAMAPSEG: 9
2019-09-04 13:42:15,392 INFO [AIF]: Number of Rows inserted into TDATAMAPSEG: 9
2019-09-04 13:42:15,393 INFO [AIF]: Number of Rows deleted from TDATAMAP_T: 9
2019-09-04 13:42:15,396 INFO [AIF]: Executing delete statement:
2019-09-04 13:42:15,661 INFO [AIF]: Successfully executed delete statement
2019-09-04 13:42:15,661 INFO [AIF]: Number of Rows deleted from TDATASEG: 6944
2019-09-04 13:42:15,662 INFO [AIF]: Number of Rows deleted from TDATAMEMOITEMS: 0
2019-09-04 13:42:15,663 INFO [AIF]: Number of Rows deleted from TDATAARCHIVE: 0
2019-09-04 13:42:15,663 INFO [AIF]: Executing insert statement:
2019-09-04 13:42:16,099 INFO [AIF]: Successfully executed insert statement
2019-09-04 13:42:16,099 INFO [AIF]: Number of Rows inserted into TDATASEG: 6944
2019-09-04 13:42:16,099 INFO [AIF]: Executing delete statement:
2019-09-04 13:42:16,349 INFO [AIF]: Successfully executed delete statement
2019-09-04 13:42:16,349 INFO [AIF]: Number of Rows deleted from TDATASEG_T: 6944
2019-09-04 13:42:16,389 INFO [AIF]: -------END IMPORT STEP-------
2019-09-04 13:42:16,419 INFO [AIF]: -------START VALIDATE STEP-------
2019-09-04 13:42:16,447 INFO [AIF]:
Validate Data Mappings for Period 'Jul-19'
2019-09-04 13:42:16,461 INFO [AIF]: Total Data Rows available for Export to Target: 6944
2019-09-04 13:42:16,462 INFO [AIF]:
Validate Data Members for Period 'Jul-19'
2019-09-04 13:42:16,465 INFO [AIF]: Total Data Rows available for Export to Target: 6944
2019-09-04 13:42:16,487 INFO [AIF]: -------END VALIDATE STEP-------
2019-09-04 13:42:16,541 INFO [AIF]: -------START EXPORT STEP-------
2019-09-04 13:42:16,580 INFO [AIF]: EssbaseService.exportData - START
2019-09-04 13:42:16,659 INFO [AIF]: cloudServiceType: PBCSE, Resolved user: [email protected], planningUserName: [email protected]
2019-09-04 13:42:16,705 INFO [AIF]: Creating data file: /u03/inbox/outbox/FPnR_AAW_309.dat
2019-09-04 13:42:16,902 INFO [AIF]: Data file creation completed: 6944
2019-09-04 13:42:16,902 INFO [AIF]: EssbaseService.exportData - END (true)
2019-09-04 13:42:16,990 INFO [AIF]: EssbaseService.loadData - START
2019-09-04 13:42:17,062 INFO [AIF]: cloudServiceType: PBCSE, Resolved user: [email protected], planningUserName: [email protected]
[Wed Sep 04 13:42:18 UTC 2019]Batch Mode (/B) turned off (/-B) for strict data loads or save of metadata specified as false (/-IMD).
[Wed Sep 04 13:42:18 UTC 2019]Saving metadata on data loads (/IMD)
Property file arguments: /DF:DD-MM-YYYY /DL:comma /I:***FPnR_AAW_309.dat /TR
No arguments were derived from the command line.Submitted (merged) command line:
/DF:DD-MM-YYYY /DL:comma /I:***FPnR_AAW_309.dat /TR
[Wed Sep 04 13:42:18 UTC 2019]Successfully located and opened input file.
[Wed Sep 04 13:42:18 UTC 2019]Input file being read with UTF-8 encoding.
[Wed Sep 04 13:42:18 UTC 2019]Date format pattern "DD-MM-YYYY" specified for date data type loading.
[Wed Sep 04 13:42:18 UTC 2019]Header record fields: Driver Member, Point-of-View, Data Load Cube Name, Value
[Wed Sep 04 13:42:18 UTC 2019]Located and using "OFADataLoadDimension" dimension for loading data in "FPnR_AAW" application.
[Wed Sep 04 13:42:18 UTC 2019]Parent-child ordering option (/H switch) will not be performed: this option is not available for the "OFADataLoadDimension" dimension.
[Wed Sep 04 13:42:18 UTC 2019]Input file member load ordering option (/O switch) will not be performed: this option is only available for (non /IE) import operations.
[Wed Sep 04 13:42:18 UTC 2019]HspOutlineLoad::dateFormatSpecified is true, SessionHalDateFormat stored on session: dd/MM/yy, sessionId: 1929669934
[Wed Sep 04 13:42:18 UTC 2019] [393] Ignoring data record 393, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [834] Ignoring data record 834, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [1072] Ignoring data record 1072, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [1684] Ignoring data record 1684, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [2067] Ignoring data record 2067, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [2809] Ignoring data record 2809, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [2962] Ignoring data record 2962, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [5446] Ignoring data record 5446, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
Outline data store load process finished. 6944 data records were read, 6945 data records were processed, 6937 were accepted for loading (verify actual load with Essbase log files), 8 were rejected.
[Wed Sep 04 13:42:19 UTC 2019]Outline data store load process finished. 6944 data records were read, 6945 data records were processed, 6937 were accepted for loading (verify actual load with Essbase log files), 8 were rejected.
2019-09-04 13:42:19,415 ERROR [AIF]: The data load process has failed.
2019-09-04 13:42:19,417 INFO [AIF]: EssbaseService.loadData - END (false)
2019-09-04 13:42:19,422 FATAL [AIF]: Error in CommData.loadData
Traceback (most recent call last):
File "<string>", line 4025, in loadData
RuntimeError: false
2019-09-04 13:42:19,484 FATAL [AIF]: Error in COMM Load Data
2019-09-04 13:42:19,487 INFO [AIF]: FDMEE Process End, Process ID: 309
Best Answer
-
You probably have a comma i.e. your delimiter in one of the fields you are exporting for the records indicated. Check the dataset in FDMEE and identify those records. Then either change the export delimiter or fix the source data to remove commas
Answers
-
Your answer is in the log:
[Wed Sep 04 13:42:18 UTC 2019] [393] Ignoring data record 393, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [834] Ignoring data record 834, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [1072] Ignoring data record 1072, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [1684] Ignoring data record 1684, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [2067] Ignoring data record 2067, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [2809] Ignoring data record 2809, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [2962] Ignoring data record 2962, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 13:42:18 UTC 2019] [5446] Ignoring data record 5446, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
Outline data store load process finished. 6944 data records were read, 6945 data records were processed, 6937 were accepted for loading (verify actual load with Essbase log files), 8 were rejected.
[Wed Sep 04 13:42:19 UTC 2019]Outline data store load process finished. 6944 data records were read, 6945 data records were processed, 6937 were accepted for loading (verify actual load with Essbase log files), 8 were rejected.
It even tells you the record numbers of the data causing the issues. Check the records in your .dat file again all the information is in the log:
Creating data file: /u03/inbox/outbox/FPnR_AAW_309.dat
-
Thanks SH_INT for your reply,
but the above record numbers is changing by every load process.
i couldn't figure out what is exactly the issue, and where can I find it, please check the below
[Wed Sep 04 14:21:20 UTC 2019] [671] Ignoring data record 671, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 14:21:20 UTC 2019] [1704] Ignoring data record 1704, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 14:21:20 UTC 2019] [2316] Ignoring data record 2316, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 14:21:20 UTC 2019] [3158] Ignoring data record 3158, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 14:21:20 UTC 2019] [3247] Ignoring data record 3247, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 14:21:20 UTC 2019] [4189] Ignoring data record 4189, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 14:21:20 UTC 2019] [5079] Ignoring data record 5079, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
[Wed Sep 04 14:21:20 UTC 2019] [6530] Ignoring data record 6530, number of header record fields (4) does not match the number of data record fields (5). Check delimiter setting and record fields.
-
You probably have a comma i.e. your delimiter in one of the fields you are exporting for the records indicated. Check the dataset in FDMEE and identify those records. Then either change the export delimiter or fix the source data to remove commas