Forum Stats

  • 3,782,429 Users
  • 2,254,644 Discussions
  • 7,880,078 Comments

Discussions

LKM (file to SQL) and multiple csv files

User510080-Oracle
User510080-Oracle Member Posts: 3
edited Sep 3, 2008 10:23AM in Data Integrator
Can I use LKM (file to SQL) to load multiple csv files to staging table in one execution and how? in my business scenario, I have account records that are saved in multiple csv files because one csv file may not be able to hold million records.

thx,
vd
Tagged:

Answers

  • 566902
    566902 Member Posts: 809
    Each interface will only load a single source file. You can run multiple interfaces to load multiple files. The physical resource can be identified by a variable, which you can populate, potentially in a loop to get the data into a temporary staging table.
    Craig
  • Hi-

    It is not possible to load FILEA then FILEB then FILEC into Target developing a single interface as ODI approach is not row by row.

    Furthermore, this not recommended: when working with file Sunopsis loads everything in memory so if you try to run that huge amount of data in a single execution you may meet some performance issue (then no data at all will be loaded).
    So 1 source=1 interface seems to me a good solution.

    For the alternate way, you can use your source as delimitted/fixed files instead of .csv if possible to get this format. In this way you can load millions of records in a single execution by using a single file.

    Also LKM_File_SQL is NOT RECOMMENDED when using LARGE VOLUMES. You can use bulk copy method by using LKM_File_Oracle(SQLLDR Bulk utilities).


    Thanks,
    Saraanan Rajavel
This discussion has been closed.