Developing MapReduce Applications on Qubole

Qubole has uploaded its Hadoop jars into a maven repository, which can be used to develop MapReduce applications to ensure compatibility with Qubole. Sample build files for Hadoop 1 and Hadoop 2 are provided in Qubole Jar Test along with example projects.

Note

When you extract files from an Hadoop archive, by default, files get extracted to a folder with a name that matches exactly with the Hadoop archive’s name. For example, files from the user/zoo/test.tar archive are extracted into the user/zoo/test.tar folder.

Locating Logs on S3

On Amazon S3:

The Hadoop-1 logs are located at: DEFLOC/logs/hadoop/<timestamp>/.

Where:

• DEFLOC refers to the default location of an account.
• <timestamp> denotes the starting time of the cluster.