What happens to the imported csv file?

Stefano_Mazzocca

I want to setup a recurring reading of a csv file from SFTP server.

What happens to the file once read? Does it remain there or will it be moved/deleted?

RVohra

@Stefano_Mazzocca ,

• BIP does NOT automatically delete or move the file after reading.
• The file remains on the SFTP server in its original location.
• BIP simply reads the file contents at the scheduled time, processes it, and generates the report.

This means if the schedule runs multiple times, it will read the same file again, unless the filename changes or you implement some rotation mechanism.

We have a few ways to manage the file after BIP reads it:

a) Leave it as-is (Default)

• Useful if multiple systems need access to the same file
• BIP will read the same file each time

b) Move or Archive the File

• BIP itself cannot automatically move files on SFTP.
• We need an external process:
  • A shell/PowerShell script on the SFTP server
  • An OIC integration workflow
• After the file is read successfully, the script can move it to an archive folder or rename it to prevent re-processing.

c) Delete the File

• Same as above, BIP doesn’t delete by itself.
• If deletion is needed, implement an external process triggered after BIP completes successfully.

Bhaskar Konar

https://community.oracle.com/products/oracleanalytics/discussion/comment/75453#Comment_75453

Hi @Stefano_Mazzocca,

Apologies for late reply as I was not online.

There are couple of things that you can do.

• Store the CSV files with system timestamp so it will create a new file every time and it will not get overwritten.
• Introduce a new process which will delete those files from the SFTP location. This one I mentioned in my previous post.

In my previous project we did the above and cleared those file with some additional process (like windows programming where it will check the files and delete last 7 days) in an agreed timeline i.e. every Sunday clearing last weeks files.

Hope this help.

Thank you.

Bhaskar Konar

https://community.oracle.com/products/oracleanalytics/discussion/28340/what-happens-to-the-imported-csv-file

Hi @Stefano_Mazzocca,

Please take a look into the following post which might be useful.

How to ingest CSV extract from object storage to oracle ADW database using OOTB features in oracle F — Oracle

Ideally the csv file will not be automatically deleted unless there's a process defined which will delete the file after the reading.

Hope this help.

Thank you.

Stefano_Mazzocca

Hi @Bhaskar Konar , your link and sublinks seem not to answer my question.

If csv is not deleted/moved automatically after reading it am I supposed to daily overwrite it to get new data?