This discussion is archived
5 Replies Latest reply: May 6, 2013 8:30 PM by 1005853 RSS

Oracle NoSQL Hadoop CountMinorKeys

1005853 Newbie
Currently Being Moderated
I have installed the latest Oracle NoSQL EE-Enterprise Edition 11gR2 (11.2.2.0.39) on my VM running CentOS 6 and CDH4.2 for MRv1 (single node). I have bumped into a different error than previously posted when running CountMinorKeys.java where CountMinorKeys$Map is not recognized as a class.

My run: hadoop jar hadoop2.jar hadoop.CountMinorKeys -libjars kvclient.jar kvstore localhost.localdomain:5000 /MyCountOutput
Error:
Exception in thread "main" java.lang.NoClassDefFoundError: hadoop/CountMinorKeys$Map
     at hadoop.CountMinorKeys.run(CountMinorKeys.java:103)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
     at hadoop.CountMinorKeys.main(CountMinorKeys.java:119)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:601)
     at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: hadoop.CountMinorKeys$Map
     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
     at java.security.AccessController.doPrivileged(Native Method)
     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
     at java.lang.ClassLoader.loadClass(ClassLoader.java:356)

I have created the jar files three different ways all to get the same above error message:
1. created jar file via command line
2. created jar file using Oracle's Jdeveloper
3. created jar file using Eclipse on windows and transferred the jar to my VM.

Three classes are created called CountMinorKeys.class, CountMinorKeys$Map.class and CountMinorKeys$Reduce.class. (the CountMinorKeys$Map.class is not being recognized.) Any recommendations? Thank you. (P.S. I am a beginner JAVA programmer and recompiling the Oracle source code would not be an option for me - as noted in previous thread as a workaround.)

I am using jdk= jdk1.7.0_17. The following 3 jar files are needed: (hadoop-common-2.0.0-cdh4.2.0.jar, hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar, kvclient.jar) --> note: the hadoop jar files are not noted in the Oracle documentation for running the tutorial. I have the 3 jars located in my /home/cloudera/myjars folder:

echo $HADOOP_CLASSPATH displays:
/home/cloudera/myjars/kvclient.jar:/home/cloudera/myjars/hadoop-common-2.0.0-cdh4.2.0.jar:/home/cloudera/myjars/hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar:/home/cloudera/myjars

Here's a sample run from /home/cloudera/myjars:
1. javac -cp kvclient.jar:hadoop-common-2.0.0-cdh4.2.0.jar:hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar CountMinorKeys.java
This produces the following 3 classes:
-rwxrwxrwx. 1 cloudera cloudera 2008 May 5 14:51 CountMinorKeys.class
-rwxrwxrwx. 1 cloudera cloudera 1827 May 5 14:51 CountMinorKeys$Map.class
-rwxrwxrwx. 1 cloudera cloudera 425 May 5 14:51 CountMinorKeys$Reduce.class

2. jar cvf Count.jar Manifest.txt *.class
output:
added manifest
adding: Manifest.txt(in = 34) (out= 36)(deflated -5%)
adding: CountMinorKeys.class(in = 2008) (out= 1047)(deflated 47%)
adding: CountMinorKeys$Map.class(in = 1827) (out= 755)(deflated 58%)
adding: CountMinorKeys$Reduce.class(in = 425) (out= 266)(deflated 37%)

3. hadoop jar Count.jar hadoop.CountMinorKeys –libjars kvclient.jar kvstore localhost.localdomain:5000 /myCountOut
Output: (error as noted above)
Exception in thread "main" java.lang.NoClassDefFoundError: hadoop/CountMinorKeys$Map
     at hadoop.CountMinorKeys.run(CountMinorKeys.java:103)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
  • 1. Re: Oracle NoSQL Hadoop CountMinorKeys
    btm Newbie
    Currently Being Moderated
    User 1002850 wrote:
    My run: hadoop jar hadoop2.jar hadoop.CountMinorKeys -libjars kvclient.jar kvstore localhost.localdomain:5000 /MyCountOutput
    Error:
    Exception in thread "main" java.lang.NoClassDefFoundError: hadoop/CountMinorKeys$Map
    at hadoop.CountMinorKeys.run(CountMinorKeys.java:103)
    I am using jdk= jdk1.7.0_17. The following 3 jar files are needed: (hadoop-common-2.0.0-cdh4.2.0.jar, hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar, kvclient.jar)
    For what it's worth, there's actually a third hadoop jar file that's needed, hadoop-annotations-2.0.0-cdh4.2.0.jar;
    although excluding this jar file should only result in warnings, not runtime errors. So I don't think that's the
    cause of your problem.
    echo $HADOOP_CLASSPATH displays:
    /home/cloudera/myjars/kvclient.jar:/home/cloudera/myjars/hadoop-common-2.0.0-cdh4.2.0.jar:/home/cloudera/myjars/hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar:/home/cloudera/myjars
    I think you need the HADOOP_CLASSPATH environment variable
    to include the Count.jar file you generate below; although I'm not
    sure the contents of that jar file is correct (see below).
    Here's a sample run from /home/cloudera/myjars:
    1. javac -cp kvclient.jar:hadoop-common-2.0.0-cdh4.2.0.jar:hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar CountMinorKeys.java

    This produces the following 3 classes:
    -rwxrwxrwx. 1 cloudera cloudera 2008 May 5 14:51 CountMinorKeys.class
    -rwxrwxrwx. 1 cloudera cloudera 1827 May 5 14:51 CountMinorKeys$Map.class
    -rwxrwxrwx. 1 cloudera cloudera 425 May 5 14:51 CountMinorKeys$Reduce.class
    Did you copy CountMinorKeys.java from the examples directory to
    the myjars directory? I ask because I would have expected the contents
    of Count.jar to include the hadoop path; that is, something like:

    -rwxrwxrwx. 1 cloudera cloudera 2008 May 5 14:51 hadoop/CountMinorKeys.class
    -rwxrwxrwx. 1 cloudera cloudera 1827 May 5 14:51 hadloop/CountMinorKeys$Map.class
    -rwxrwxrwx. 1 cloudera cloudera 425 May 5 14:51 hadoop/CountMinorKeys$Reduce.class

    Brian
  • 2. Re: Oracle NoSQL Hadoop CountMinorKeys
    1005853 Newbie
    Currently Being Moderated
    Thanks Brian. It's so great to hear from you as I have been battling this for two days.

    My CountMinorKeys.java and all my jars are in my /home/cloudera/myjars directory. I checked to ensure I am using Oracle's sample CountMinorKeys.java file. I brought over the new jar file: hadoop-annotations-2.0.0-cdh4.2.0.jar. (As you mentioned, this hadoop-annotations file does remove the warning.) Thanks for the tip!

    I fixed my output argument on my hadoop run to align with core-site.xml.

    I am getting a bit of a different error message now!
    Here's the output from the Linux command line:
    exception in thread "main" java.lang.ClassNotFoundException: hadoop.CountMinorKeys
    (bottom of thread has additional error messages)

    Here's my run from the Linux Command line:
    1. [cloudera@localhost myjars]$ javac -cp kvclient.jar:hadoop-common-2.0.0-cdh4.2.0.jar:hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar:hadoop-annotations-2.0.0-cdh4.2.0.jar CountMinorKeys.java
    Output:
    [cloudera@localhost myjars]$ ls -l *.class
    -rwxrwxrwx. 1 cloudera cloudera 1893 May 6 12:21 CountMinorKeys.class (yes the hadoop.CountMinorKeys.class is not present.)
    -rwxrwxrwx. 1 cloudera cloudera 1827 May 6 12:21 CountMinorKeys$Map.class " "
    -rwxrwxrwx. 1 cloudera cloudera 425 May 6 12:21 CountMinorKeys$Reduce.class " "


    2. jar cvf CountKeysMondayPM.jar Manifest.txt *.class
    Output:
    [cloudera@localhost myjars]$ jar cvf CountKeysMondayPM.jar Manifest.txt *.class
    added manifest
    adding: Manifest.txt(in = 34) (out= 36)(deflated -5%)
    adding: CountMinorKeys.class(in = 1893) (out= 980)(deflated 48%)
    adding: CountMinorKeys$Map.class(in = 1827) (out= 755)(deflated 58%)
    adding: CountMinorKeys$Reduce.class(in = 425) (out= 265)(deflated 37%)

    [cloudera@localhost myjars]$ cat Manifest.txt
    Main-Class: hadoop.CountMinorKeys --> Is this correct?

    3. my .bash_profile:
    # Add HADOOP home
    HADOOP_HOME=/usr/lib/hadoop
    # Add HADOOP_STREAMING
    export HADOOP_STREAMING=/usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.2.0.jar
    # Add HADOOP CMD
    export HADOOP_CMD=/usr/bin/hadoop
    # Add KVHOME
    export KVHOME=/home/cloudera/Oracle_nosql/kv-2.0.39
    # Add HADOOP_CLASSPATH
    export HADOOP_CLASSPATH=/home/cloudera/myjars/*.jar
    # export CLASSPATH
    export CLASSPATH=/home/cloudera/myjars/kvclient.jar:/home/cloudera/myjars/*

    [cloudera@localhost myjars]$ hadoop jar CountKeysMondayPM.jar hadoop.CountMinorKeys –libjars kvclient.jar kvstore localhost.localdomain:5000 hdfs://localhost:8020/tmp/hadoop/output/
    Output:
    Exception in thread "main" java.lang.ClassNotFoundException: hadoop.CountMinorKeys
         at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
         at java.lang.Class.forName0(Native Method)
         at java.lang.Class.forName(Class.java:266)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:201)

    I think there is a small tweak that is missing now. I would greatly appreciate any assistance as I really need to get this demo working before Thursday!
    Regards
  • 3. Re: Oracle NoSQL Hadoop CountMinorKeys
    btm Newbie
    Currently Being Moderated
    1002850 wrote:
    My CountMinorKeys.java and all my jars are in my /home/cloudera/myjars directory.
    Okay. But just so you know, you could have left CountMinorKeys.java in the
    <kv-install>/examples/hadoop directory, and compiled and created your jar file
    from there; although moving it to /home/cloudera/myjars should work too.
    Here's my run from the Linux Command line:
    1. [cloudera@localhost myjars]$ javac -cp kvclient.jar:hadoop-common-2.0.0-cdh4.2.0.jar:hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar:hadoop-annotations-2.0.0-cdh4.2.0.jar CountMinorKeys.java
    Output:
    [cloudera@localhost myjars]$ ls -l *.class
    -rwxrwxrwx. 1 cloudera cloudera 1893 May 6 12:21 CountMinorKeys.class (yes the hadoop.CountMinorKeys.class is not present.)
    -rwxrwxrwx. 1 cloudera cloudera 1827 May 6 12:21 CountMinorKeys$Map.class " "
    -rwxrwxrwx. 1 cloudera cloudera 425 May 6 12:21 CountMinorKeys$Reduce.class " "
    Okay, I think this is where the problem is. Your resulting class files should reside
    under a directory named 'hadoop'; which is the package name of the CountMinorKeys
    class. So try running the command with the '-d .' option; that is, try the following:

    $ cd /home/cloudera/myjars
    $ rm CountMinorKeys*.class [optional: get rid of the old class files, just to avoid confusion]

    $ javac -cp kvclient.jar:hadoop-common-2.0.0-cdh4.2.0.jar:hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar:hadoop-annotations-2.0.0-cdh4.2.0.jar -d . CountMinorKeys.java

    The '-d .' option should create a directory named 'hadoop' under /home/cloudera/myjars.
    If you look in that directory, you should see the class files generated from the above
    javac command. That is,

    $ cd /home/cloudera/myjars
    $ ls -al hadoop

    -rwxrwxrwx. 1 cloudera cloudera 1893 May 6 12:21 CountMinorKeys.class
    -rwxrwxrwx. 1 cloudera cloudera 1827 May 6 12:21 CountMinorKeys$Map.class
    -rwxrwxrwx. 1 cloudera cloudera 425 May 6 12:21 CountMinorKeys$Reduce.class

    Now, you can generate your jar file from the hadoop directory by doing the following:

    $ cd /home/cloudera/myjars
    $ jar cvf CountKeysMondayPM.jar hadoop

    Note that you don't need to add the manifiest (unless you really want to); as the
    'java jar' command will be able to figure out the main class to execute. More
    importantly, don't specify '*.class'. You just need to specify the hadoop directory,
    which corresponds to the package name. This will result in a jar file with content
    that is format properly. Thus, to verify the contents of the jar file, type the following:

    $ jar tvf CountKeysMondayPM.jar [which should look something like the following]

    0 Mon May 06 17:55:06 EDT 2013 META-INF/
    71 Mon May 06 17:55:06 EDT 2013 META-INF/MANIFEST.MF
    0 Mon May 06 17:55:06 EDT 2013 hadoop/
    1893 Mon May 06 17:55:06 EDT 2013 hadoop/CountMinorKeys.class
    425 Mon May 06 17:55:06 EDT 2013 hadoop/CountMinorKeys$Reduce.class
    1827 Mon May 06 17:55:06 EDT 2013 hadoop/CountMinorKeys$Map.class

    It is in the contents of the jar file that you should see the hadoop directory
    path. I suspect that if you were to look at the contents of your original
    jar file, that you probably have only class names, not the hadoop package
    component.

    I think your command line for executing the example looks right. For
    what it's worth, I believe it wasn't working because you were asking
    that the class hadoop.CountMinorKeys be executed, but the jar file
    contained only 'CountMinorKeys.class', not 'hadoop/CountMinorKeys.class';
    that is, the jar file contents did not reflect the package component of
    the class.

    Try the above, and let us know if that helps, okay?

    Brian

    PS If you want to repeat the compilation step multiple times, don't forget to
    'clean' the hadoop directory that is created each time. That is, remove the
    directory and its contents before recompiling CountMinorKeys.java; otherwise,
    the javac command will fail because the directory it tries to create already exists.
  • 4. Re: Oracle NoSQL Hadoop CountMinorKeys
    1005853 Newbie
    Currently Being Moderated
    Thank you very much Brian! That was a perfect write-up and explanation with great examples. Perfect for a newbie Java programmer as I am.

    I did all the steps and now getting the following error:
    Exception in thread "main" java.lang.NoClassDefFoundError: oracle/kv/hadoop/KVInputFormat
         at hadoop.CountMinorKeys.run(CountMinorKeys.java:103)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
         at hadoop.CountMinorKeys.main(CountMinorKeys.java:117)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:601)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
    Caused by: java.lang.ClassNotFoundException: oracle.kv.hadoop.KVInputFormat
         at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
         ... 9 more

    Here's my steps:
    A snip from environment from .bash_profile:
    export HADOOP_CMD=/usr/bin/hadoop
    # Add KVHOME
    export KVHOME=/home/cloudera/Oracle_nosql/kv-2.0.39
    # Add HADOOP_CLASSPATH
    export HADOOP_CLASSPATH=/home/cloudera/myjars/*.jar
    # export CLASSPATH
    export CLASSPATH=/home/cloudera/myjars/kvclient.jar:/home/cloudera/myjars/*

    I am in directory where CountMinorKeys.java resides from the Oracle installation: (I also tried running below steps in directory /home/cloudera/myjars (CountMinorKeys.java) with same results - as you anticipated.)

    [cloudera@localhost hadoop]$ pwd
    /home/cloudera/Oracle_nosql/kv-2.0.39/examples/hadoop
    [cloudera@localhost hadoop]$ ls
    CountMinorKeys.java

    My 4 jars needed for compiling are located in: ~/myjars
    /home/cloudera/myjars/kvclient.jar
    /home/cloudera/myjars/hadoop-common-2.0.0-cdh4.2.0.jar
    /home/cloudera/myjars/hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar
    /home/cloudera/myjars/hadoop-annotations-2.0.0-cdh4.2.0.jar

    1. [cloudera@localhost hadoop]$ pwd
    /home/cloudera/Oracle_nosql/kv-2.0.39/examples/hadoop
    [cloudera@localhost hadoop]$ javac -cp /home/cloudera/myjars/kvclient.jar:/home/cloudera/myjars/hadoop-common-2.0.0-cdh4.2.0.jar:/home/cloudera/myjars/hadoop-mapreduce-client-core-2.0.0-cdh4.2.0.jar:/home/cloudera/myjars/hadoop-annotations-2.0.0-cdh4.2.0.jar -d . CountMinorKeys.java
    [cloudera@localhost hadoop]$ ls -al hadoop
    total 20
    drwxrwxr-x. 2 cloudera cloudera 4096 May 6 18:58 .
    drwxrwxr-x. 3 cloudera cloudera 4096 May 6 18:58 ..
    -rw-rw-r--. 1 cloudera cloudera 1893 May 6 18:58 CountMinorKeys.class
    -rw-rw-r--. 1 cloudera cloudera 1827 May 6 18:58 CountMinorKeys$Map.class
    -rw-rw-r--. 1 cloudera cloudera 425 May 6 18:58 CountMinorKeys$Reduce.class
    (same as yours - great!)

    2. [cloudera@localhost hadoop]$ pwd
    /home/cloudera/Oracle_nosql/kv-2.0.39/examples/hadoop
    [cloudera@localhost hadoop]$ jar cvf CountMinorKeysTest.jar hadoop
    added manifest
    adding: hadoop/(in = 0) (out= 0)(stored 0%)
    adding: hadoop/CountMinorKeys$Reduce.class(in = 425) (out= 265)(deflated 37%)
    adding: hadoop/CountMinorKeys.class(in = 1893) (out= 980)(deflated 48%)
    adding: hadoop/CountMinorKeys$Map.class(in = 1827) (out= 755)(deflated 58%)
    (ah the haoop is now added ! Just like you recommended!)

    mkdir extract and copy CountMinorKeysTest.jar into extract
    [cloudera@localhost extract]$ pwd
    /home/cloudera/Oracle_nosql/kv-2.0.39/examples/hadoop/extract

    [cloudera@localhost extract]$ ls
    CountMinorKeysTest.jar

    [cloudera@localhost extract]$ jar tvf CountMinorKeysTest.jar
    0 Mon May 06 19:00:24 PDT 2013 META-INF/
    68 Mon May 06 19:00:24 PDT 2013 META-INF/MANIFEST.MF
    0 Mon May 06 18:58:34 PDT 2013 hadoop/
    425 Mon May 06 18:58:34 PDT 2013 hadoop/CountMinorKeys$Reduce.class
    1893 Mon May 06 18:58:34 PDT 2013 hadoop/CountMinorKeys.class
    1827 Mon May 06 18:58:34 PDT 2013 hadoop/CountMinorKeys$Map.class
    (just like you said would happen!)

    3. [cloudera@localhost hadoop]$ pwd
    /home/cloudera/Oracle_nosql/kv-2.0.39/examples/hadoop
    [cloudera@localhost hadoop]$ hadoop jar CountMinorKeysTest.jar hadoop.CountMinorKeys –libjars /home/cloudera/myjars/kvclient.jar kvstore localhost.localdomain:5000 hdfs://localhost:8020/tmp/Output/

    Output:
    Exception in thread "main" java.lang.NoClassDefFoundError: oracle/kv/hadoop/KVInputFormat
         at hadoop.CountMinorKeys.run(CountMinorKeys.java:103)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
         at hadoop.CountMinorKeys.main(CountMinorKeys.java:117)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:601)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
    Caused by: java.lang.ClassNotFoundException: oracle.kv.hadoop.KVInputFormat
         at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
         ... 9 more

    Note: I also ran the command without the -libjars argument with same result.

    [cloudera@localhost hadoop]$ hadoop jar CountMinorKeysTest.jar hadoop.CountMinorKeys kvstore localhost.localdomain:5000 hdfs://localhost:8020/tmp/Output/
    Exception in thread "main" java.lang.NoClassDefFoundError: oracle/kv/hadoop/KVInputFormat
    .........

    Not sure how to resolve this. Seems I am getting closer. Could you kindly advise? Thank you so much.
  • 5. Re: Oracle NoSQL Hadoop CountMinorKeys
    1005853 Newbie
    Currently Being Moderated
    I am happy to report I got it to work!!! Thanks so much Brian for helping this newbie Java programmer out! Awesome help!

    Here's the details:

    Same as steps 1 and 2 above. Here's the changes for step 3.

    [cloudera@localhost hadoop]$ echo $HADOOP_CLASSPATH (this is in my /home/cloudera/.bash_profile)
    /home/cloudera/Oracle_nosql/kv-2.0.39/lib/kvclient.jar

    [cloudera@localhost hadoop]$ pwd
    /home/cloudera/Oracle_nosql/kv-2.0.39/examples/hadoop
    [cloudera@localhost hadoop]$ hadoop jar CountMinorKeysTest.jar hadoop.CountMinorKeys -libjars ../../lib/kvclient.jar kvstore localhost:5000 hdfs://localhost:8020/tmp/Output/

    Map Reduce run looks like the following:
    13/05/06 20:21:47 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
    13/05/06 20:21:49 INFO mapred.JobClient: Running job: job_201305061817_0012
    13/05/06 20:21:50 INFO mapred.JobClient: map 0% reduce 0%
    13/05/06 20:22:14 INFO mapred.JobClient: map 10% reduce 0%
    13/05/06 20:22:15 INFO mapred.JobClient: map 20% reduce 0%
    ...

    Output for successful run:
    [cloudera@localhost  hadoop]$ hadoop fs -cat /tmp/Output/part-r-00000 | head -20
    /123     1
    /1234     1
    /12345     1
    /123456     1
    /124     1
    /125     1
    /126     1
    /127     1
    ....


    TERRIFIC! Thanks again Brian!!!

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points