Tuesday, December 27, 2011

Map Side Join

Programming Lab 4: Map Side Join
  1. This lab  demonstrates Map-Side join technique used to quickly and efficiently join data in multiple directories
  2. In a lot of cases this is the fastest way to process data.
 Problem to Solve
  1. We will work with two sets of data OrgChart and Salary and will will need to join them together using Map Reduce join technique
Create Project and Link with Libraries
  1. Copy provided libraries and java code from USB drives
  2. Create project in NetBeans or Eclipse
  3. Link with libraries
  4. Create new class and copy provided code
  5. Modify input and output directory to SalaryMap and OrgChartMap folders
  6. For detailed instructions on project creating refer to Lab1.
Task1. Create Map Files Using Previous Lab
  1. Using lab 3 code point input directory to lab4/input/Salary directory and point output directory to lab4/input/SalaryMap directory
  2. Run the code and it will generate map file (sequence file with index)
  3. Repeat the same procedure with OrgChart
Task2. Point MapSide Directories to Newly Generated Map Files
  1. Change directory to point to newly generated map files
  2. Run code and see the outcome
  1. Please, note that unlike regular  MR job, map-side joint quarantines order of the data.
Walk Through
  1. Map-Side join takes array of the path to join
  1. Set specific Map Side join input format
Walk Through Mapper
Mapper provides access to values via TupleWritable value
    value.has(0) checks if you have left value
    value.get(0) gives you the left value
    value.has(1) checks if you have right value
    value.get(1) gives you the right value
Task3. Adding Vacation Time
  1. Generate Map File for vacation time
  2. Extend Map-Reduce job to add another folder in join
  3. Modify Mapper code to add vacation time into output value
  4. Hint: to add two text object:

1 comment:

  1. Hi
    Really very informative.Thanks for sharing.Recently I bought The hadoop videos from www.hadooponlinetutor.com.The videos are really very good