Forum Stats

  • 3,784,355 Users
  • 2,254,928 Discussions
  • 7,880,790 Comments

Discussions

Oracle SQL Connector for HDFS - Where is the filter applied?

mweedman
mweedman Member Posts: 6 Blue Ribbon
edited Oct 28, 2013 8:42AM in Big Data Connectors/Hadoop
I have just managed to get the Oracle SQL Connector for HDFS (YEAH!)
working on a Linux 2VM cluster and was curious where the filter in the where clause is being applied to limit the data.
i.e. is it being pushed clear down into hadoop or is hadoop pulling all the data and then then it's being filtered or does all the data get buffered into the database or what?

Thanks

Mark

Best Answer

  • User_S7LLG
    User_S7LLG Member Posts: 1 Employee
    Accepted Answer
    The filter will be applied in Oracle. OSCH does not push down the filter to Hadoop to execute and MapReduce code. So the predicate would be handled the same way as any other external table or I presume any (non-exadata) table would be.

Answers

  • mweedman
    mweedman Member Posts: 6 Blue Ribbon
    After thinking a little more on this, the more appropriate question might be , "does all the data get loaded into the buffer cache before the filter is applied?"
  • User_S7LLG
    User_S7LLG Member Posts: 1 Employee
    Accepted Answer
    The filter will be applied in Oracle. OSCH does not push down the filter to Hadoop to execute and MapReduce code. So the predicate would be handled the same way as any other external table or I presume any (non-exadata) table would be.
  • mweedman
    mweedman Member Posts: 6 Blue Ribbon

    Along the same lines of this question, does anybody know a way to connect from Oracle to Hadoop/HDFS in such a way as to push the query filter down into the Hadoop query?

    I really don't want to pull ALL my hdfs data into the Oracle buffer cache for processing!

    Thanks

    Mark

This discussion has been closed.