使用Sqoop将数据从Redshift导入到Hive

Question

我收到错误：Could not load db driver class。

连接和错误如下。在该目录下是lib目录中的jar文件列表。我究竟做错了什么？

sqoop import 
--connect jdbc:redshift://< > 
--username < > --password < > 
--driver com.amazon.redshift.jdbc.Driver 
--table import-all-tables 

17/04/21 11:14:46 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.0-258 
17/04/21 11:14:46 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 
17/04/21 11:14:46 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 
17/04/21 11:14:46 INFO manager.SqlManager: Using default fetchSize of 1000 
17/04/21 11:14:46 INFO tool.CodeGenTool: Beginning code generation 
17/04/21 11:14:46 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.amazon.redshift.jdbc.Driver 
java.lang.RuntimeException: Could not load db driver class: com.amazon.redshift.jdbc.Driver 
     at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:856) 
     at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) 
     at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:744) 
     at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767) 
     at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270) 
     at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241) 
     at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227) 
     at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295) 
     at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1845) 
     at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645) 
     at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:148) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:244) 



[t lib]$ ls 
ant-contrib-1.0b3.jar   hsqldb-1.8.0.10.jar   kite-hadoop-compatibility-1.0.0.jar parquet-generator-1.4.1.jar 
ant-eclipse-1.0-jvm1.2.jar  jackson-annotations-2.3.0.jar mysql-connector-java.jar    parquet-hadoop-1.4.1.jar 
avro-1.7.5.jar     jackson-core-2.3.1.jar   opencsv-2.3.jar      parquet-jackson-1.4.1.jar 
avro-mapred-1.7.5-hadoop2.jar jackson-core-asl-1.9.13.jar paranamer-2.3.jar     RedshiftJDBC42-1.2.1.1001 (2).jar 
commons-codec-1.4.jar   jackson-databind-2.3.1.jar  parquet-avro-1.4.1.jar    slf4j-api-1.6.1.jar 
commons-compress-1.4.1.jar  jackson-mapper-asl-1.9.13.jar parquet-column-1.4.1.jar    snappy-java-1.0.5.jar 
commons-io-1.4.jar    kite-data-core-1.0.0.jar  parquet-common-1.4.1.jar    xz-1.0.jar 
commons-jexl-2.1.1.jar   kite-data-hive-1.0.0.jar  parquet-encoding-1.4.1.jar 
commons-logging-1.1.1.jar  kite-data-mapreduce-1.0.0.jar parquet-format-2.0.0.jar 

Answer 1

你的JDBC是没有sqoop/lib中存在，因此下载您的有效JDBC驱动程序，并复制到sqoop/lib目录