This is the documentation for CDH 5.1.x. Documentation for other versions is available at Cloudera Documentation.

Configuring Sqoop 2

This section explains how to configure the Sqoop 2 server.

Configuring which Hadoop Version to Use

The Sqoop 2 client does not interact directly with Hadoop MapReduce, and so it does not require any MapReduce configuration.

The Sqoop 2 server can work with either MRv1 or YARN. It cannot work with both simultaneously.

You set the MapReduce version the Sqoop 2 server works with by means of the alternatives command (or update-alternatives, depending on your operating system):
  • To use YARN:
    alternatives --set sqoop2-tomcat-conf /etc/sqoop2/tomcat-conf.dist
  • To use MRv1:
    alternatives --set sqoop2-tomcat-conf /etc/sqoop2/tomcat-conf.mr1
  Important: If you are upgrading from a release earlier than CDH 5 Beta 2

In earlier releases, the mechanism for setting the MapReduce version was the CATALINA_BASEvariable in the /etc/defaults/sqoop2-server file. This does not work as of CDH 5 Beta 2, and in fact could cause problems. Check your /etc/defaults/sqoop2-server file and make sure CATALINA_BASE is not set.

Installing the JDBC Drivers

Sqoop 2 does not ship with third party JDBC drivers. You must download them separately and save them to the /var/lib/sqoop2/ directory. The following sections show how to install the most common JDBC Drivers.

Installing the MySQL JDBC Driver

  1. Download the MySQL JDBC driver onto the Sqoop 2 server from http://www.mysql.com/downloads/connector/j/5.1.html. You will need to sign up for an account if you don't already have one, and log in, before you can download it.
  2. Copy the JAR file to the /var/lib/sqoop2/ directory on the Sqoop 2 server. For example: 
    $ sudo cp mysql-connector-java-version/mysql-connector-java-version-bin.jar /var/lib/sqoop2/

    At the time of publication, version was 5.1.31, but the version may have changed by the time you read this.

      Important:

    Make sure you have at least version 5.1.31. Some systems ship with an earlier version that may not work correctly with Sqoop.

  3. Restart the Sqoop 2 server to load the driver.

Installing the Oracle JDBC Driver

  1. Download the JDBC Driver from the Oracle website, for example http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-112010-090769.html. You must accept the license agreement before you can download the driver. Download the ojdbc6.jar file.
  2. Copy the JAR file to /var/lib/sqoop2/ directory: 
    $ sudo cp ojdbc6.jar /var/lib/sqoop2/
  3. Restart the Sqoop 2 server to load the driver.

Installing the Microsoft SQL Server JDBC Driver

  1. Download the Microsoft SQL Server JDBC driver from http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=11774
  2. Copy the JAR file to the /var/lib/sqoop2/ directory. For example: 
    $ curl -L 'http://download.microsoft.com/download/0/2/A/02AAE597-3865-456C-AE7F-613F99F850A8/sqljdbc_4.0.2206.100_enu.tar.gz' | tar xz
$ sudo cp sqljdbc_4.0/enu/sqljdbc4.jar /var/lib/sqoop2/
  3. Restart the Sqoop 2 server to load the driver.

Installing the PostgreSQL JDBC Driver

  1. Download the PostgreSQL JDBC driver from http://jdbc.postgresql.org/download.html
  2. Copy the JAR file to the /var/lib/sqoop2/ directory. For example: 
    $ curl -L 'http://jdbc.postgresql.org/download/postgresql-9.2-1002.jdbc4.jar' -o postgresql-9.2-1002.jdbc4.jar
$ sudo cp postgresql-9.2-1002.jdbc4.jar /var/lib/sqoop2/
  3. Restart the Sqoop 2 server to load the driver.
Page generated September 3, 2015.