Sqoop1 Install And Test MySQL/PostgreSQL-白红宇

环境: Hadoop2.7.2(NameNode HA,ResourceManage HA)+HBASE+HIVE

环境: Hadoop2.7.2(NameNode HA,ResourceManage HA)
1.下载

[root@sht-sgmhadoopnn-01 hadoop]# wget

[root@sht-sgmhadoopnn-01 hadoop]# tar xzvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

[root@sht-sgmhadoopnn-01 hadoop]# ln -s /hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop

2.配置sqoop-env.sh

[root@sht-sgmhadoopnn-01 conf]# pwd

/hadoop/sqoop/conf

[root@sht-sgmhadoopnn-01 conf]# cp sqoop-env-template.sh sqoop-env.sh

[root@sht-sgmhadoopnn-01 conf]#

[root@sht-sgmhadoopnn-01 conf]# vi sqoop-env.sh

export HADOOP_COMMON_HOME=/hadoop/hadoop

export HADOOP_MAPRED_HOME=/hadoop/hadoop/share/hadoop/mapreduce

export HBASE_HOME=/hadoop/hbase

export HIVE_HOME=/hadoop/hive

3.配置 mysql，postgresql jdbc驱动包

[root@sht-sgmhadoopnn-01 lib]# pwd

/hadoop/sqoop/lib

[root@sht-sgmhadoopnn-01 lib]# ll

total 17088

..............

-rw-r--r-- 1 root root 703265 Jul 29 21:41 mysql-connector-java-5.1.6-bin.jar

..............

-rw-r--r-- 1 root root 684978 Jul 30 10:13 postgresql-9.4.1209.jre6.jar

..............

[root@sht-sgmhadoopnn-01 lib]#

4.测试连接mysql，列举databases

[root@sht-sgmhadoopnn-01 bin]# sqoop list-databases --connect jdbc:mysql://sht-sgmhadoopnn-01:3306/ --username root --password agm43gadsg

Warning: /hadoop/sqoop/../hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

Warning: /hadoop/sqoop/../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

Warning: /hadoop/zookeeper does not exist! Accumulo imports will fail.

Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

16/07/30 10:01:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6

16/07/30 10:01:12 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

16/07/30 10:01:12 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

information_schema

dba

hive

hive_remote_meta

mysql

performance_schema

repl

test1

5.测试连接postgres，列举databases

[root@sht-sgmhadoopnn-01 bin]# sqoop-list-databases --connect jdbc:postgresql://172.16.101.54:5432/denali --username denaliadmin --password "rdyhdenali"

Warning: /hadoop/sqoop/../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

Warning: /hadoop/zookeeper does not exist! Accumulo imports will fail.

Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

16/07/30 10:28:31 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6

16/07/30 10:28:31 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

16/07/30 10:28:31 INFO manager.SqlManager: Using default fetchSize of 1000

template1

template0

postgres

denali

[root@sht-sgmhadoopnn-01 bin]#