Big Data - Java program

Steps to create a java program for map reduce

1. Open eclipse
2. Create java project
3. Add package
4. Add .java files(drive, mapper, reducer, combiner, partitioner, etc)
5. Open project properties
6. Add External jars/libs
1. HadoopHome/common/
2. HadoopHome/common/lib
3. HadoopHome/hdfs
4. HadoopHome/mapreduce
7. Complile
8. Export jar

How to run ?

Create input directory

$hdfs dfs -mkdir /input

Add files in input directory from local file system

$hdfs dfs -put  input.txt /input

or 

Create there

$hdfs dfs -touch  /input/input.txt 

To run use following command

$yarn your_mapreduce_project.jar <packagename>.<classname> /input /output

Check output

$hdfs dfs -ls /output

$hdfs dfs -cat /output/part-r-0000