Working with MapReduce

Administration Lab 3: Working with MapReduce

Restore the Last State of VM

      Open Virtual Box application

      Start the last VM state

      Bounce Hadoopcluster

for x in /etc/init.d/hadoop-* ; do sudo $x stop; done

for x in /etc/init.d/hadoop-* ; do sudo $x start; done

Calcualte PI

      Execute the following command:

hadoop-0.20 jar /usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u0-*examples.jar pi 10 10000

      Result: job finished in 103.293 seconds

      Estimated value of PI is 3.1412000000000 

Execute Word Count

      Create some input: vi test.txt

      Move input to HDFS:

hadoop-0.20 dfs -put ./test.txt /test.txt

      Execute word count job

hadoop-0.20 dfs -ls/

hadoop-0.20 jar /usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u1-*examples.jar wordcount /test.txt /output

      Check what happen with output directory

hadoop-0.20 dfs -ls /output

      View output

hadoop-0.20 dfs -cat /output/part