Forum Stats

  • 3,780,847 Users
  • 2,254,447 Discussions
  • 7,879,484 Comments

Discussions

How to use Oracle Direct Connector for HDFS(DCH)?

user1745344
user1745344 Member Posts: 6
edited May 27, 2012 10:12PM in Big Data Connectors/Hadoop
hi all,

I am trying to use Oracle database to access HDFS through DCH. Now I installed Oracle database 11.2, hadoop-0.20.2 and DCH on the same linux machine.
I also configured hdfs_stream properly and created Oracle directory HDFS_BIN_PATH according to README.txt file within DCH.
Now I encountered a problem, how I create a external table which access data residing on HDFS?

Can anyone help me ?
Thanks in advance.

Chopsun

Edited by: user1745344 on 2012-5-21 下午11:30

Answers

  • user1745344
    user1745344 Member Posts: 6
    edited May 22, 2012 4:04AM
    I figure out the way which access HDFS data from Oracle External Table through DCH as below, but still face some problems

    Step 1
    Created Oracle directories as following:
    CREATE OR REPLACE DIRECTORY hdfs_bin_path AS '/home/oracle/orahdfs/bin';
    CREATE OR REPLACE DIRECTORY data_dir AS '/user/hadoop';
    where /user/hadoop is HDFS directory in which data file test.txt located.

    Step 2
    Created Oracle external table as following:
    create table xqs_ext_test(col1 varchar2(100))
    organization external (
    TYPE oracle_loader
    DEFAULT DIRECTORY data_dir
    ACCESS PARAMETERS (
    records delimited by newline
    preprocessor hdfs_bin_path:'hdfs_stream'
    badfile hdfs_bin_path:'empxt%a_%p.bad'
    logfile hdfs_bin_path:'empxt%a_%p.log'
    fields terminated by ','
    )
    LOCATION ('test.txt'))
    reject limit unlimited
    parallel 2;

    when run SQL statement : select * from xqs_ext_test; the following error was prompted:

    SELECT * FROM xqs_ext_test
    *
    ERROR at line 1:
    ORA-29913: error in executing ODCIEXTTABLEOPEN callout
    ORA-29400: data cartridge error
    KUP-04040: file test.txt in DATA_DIR not found


    who can help me out with this problem?


    Best Regards

    Chopsun

    Edited by: user1745344 on 2012-5-22 上午1:02
  • Ok, silly question, but does the file exist there? Also, just to check, do you have the db patches/code installed?

    JP
  • this problem got solved.

    thanks

    Chopsun.
This discussion has been closed.