This discussion is archived
3 Replies Latest reply: May 27, 2012 7:12 PM by user1745344 RSS

How to use Oracle Direct Connector for HDFS(DCH)?

user1745344 Newbie
Currently Being Moderated
hi all,

I am trying to use Oracle database to access HDFS through DCH. Now I installed Oracle database 11.2, hadoop-0.20.2 and DCH on the same linux machine.
I also configured hdfs_stream properly and created Oracle directory HDFS_BIN_PATH according to README.txt file within DCH.
Now I encountered a problem, how I create a external table which access data residing on HDFS?

Can anyone help me ?
Thanks in advance.

Chopsun

Edited by: user1745344 on 2012-5-21 下午11:30
  • 1. Re: How to use Oracle Direct Connector for HDFS(DCH)?
    user1745344 Newbie
    Currently Being Moderated
    I figure out the way which access HDFS data from Oracle External Table through DCH as below, but still face some problems

    Step 1
    Created Oracle directories as following:
    CREATE OR REPLACE DIRECTORY hdfs_bin_path AS '/home/oracle/orahdfs/bin';
    CREATE OR REPLACE DIRECTORY data_dir AS '/user/hadoop';
    where /user/hadoop is HDFS directory in which data file test.txt located.

    Step 2
    Created Oracle external table as following:
    create table xqs_ext_test(col1 varchar2(100))
    organization external (
    TYPE oracle_loader
    DEFAULT DIRECTORY data_dir
    ACCESS PARAMETERS (
    records delimited by newline
    preprocessor hdfs_bin_path:'hdfs_stream'
    badfile hdfs_bin_path:'empxt%a_%p.bad'
    logfile hdfs_bin_path:'empxt%a_%p.log'
    fields terminated by ','
    )
    LOCATION ('test.txt'))
    reject limit unlimited
    parallel 2;

    when run SQL statement : select * from xqs_ext_test; the following error was prompted:

    SELECT * FROM xqs_ext_test
    *
    ERROR at line 1:
    ORA-29913: error in executing ODCIEXTTABLEOPEN callout
    ORA-29400: data cartridge error
    KUP-04040: file test.txt in DATA_DIR not found


    who can help me out with this problem?


    Best Regards

    Chopsun

    Edited by: user1745344 on 2012-5-22 上午1:02
  • 2. Re: How to use Oracle Direct Connector for HDFS(DCH)?
    277379 Newbie
    Currently Being Moderated
    Ok, silly question, but does the file exist there? Also, just to check, do you have the db patches/code installed?

    JP
  • 3. Re: How to use Oracle Direct Connector for HDFS(DCH)?
    user1745344 Newbie
    Currently Being Moderated
    this problem got solved.

    thanks

    Chopsun.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points