Running an external jar from Aws Hadoop

Name of Innovation

Running an external jar from Aws Hadoop

February 16, 2017 AWS Hadoop Tutorial 0

Hadoop AWS

  • First select the services  and click on EMR from Analytics.

Screenshot (45)

 

  • Then click on the add cluster.

Screenshot (46)

 

  • Fill the Details of Cluster.
  1. Cluster name as Ananthapur-jntu
  2. Here we are checking the Logging
  3. Browse the s3 folder with the amar2017/feb
  4. Launch mode should be Step Extension
  5. After that select step type as custom jar and click on configure.
  6. The below image is showing the details.

Screenshot (49)

  • After clicking on the configure button we will see the popup like shown below
  1. Name as Custom JAR
  2. Jar location should be s3://amar2017/inputJar/wcount.jar
  3. Fill the Arguments with org.myorg.wordcount, s3://amar2017/deleteme.txt, s3://amar2017/output3
  4. Select the Action on failure as Terminate cluster
  5. Then click on add button.
  6. How to fill the details as shown below.

 

Screenshot (48)

  • Software configuration
  1. Select vendor as Amazon
  2. select Release as emr-5.3.1
  • Hardware Configuration
  1. select instance type as m1.medium
  2. And number of instanses as 3
  • Security and access
  1. Permissions checking as default
  2. After that click on create cluster button
  • Details as shown in the below image

Screenshot (50)

  • You will see the Cluster Ananthapur is Starting as shown below.

Screenshot (51)

  • The below image is showing that in cluster list Ananthapur is starting.

Screenshot (52)

  • After complishing the process we can see like below image.

Screenshot (53)

  • To see the result of AWS Hadoop go to the services.
  • Select S3 under storage.

Screenshot (142)

 

  • After clicking on S3
  • select amar2017

Screenshot (149)

  • Anad then select output3 folder
  • You will see the list of files as shown below

Screenshot (146)

  • Open the part-r-00000 file
  • You will see the page as shown below

Screenshot (147)

  • Click on the Download button
  • And Open the downloaded file you will see the result as shown below.

Screenshot (57)