1. 首页
  2. 问答
  3. 从Hue运行Sqoop作业时出错

从Hue运行Sqoop作业时出错

2014-12-19 149 views
1

我正在尝试从Hue创建Sqoop作业;在“作业设计器”下，我创建了一个新作业输入teh命令来创建sqoop作业，如下所示。从Hue运行Sqoop作业时出错

sqoop job --create TestHiveImport -- import --connect jdbc:oracle:thin:@<ipaddress>:port/<database_name> --username <username> --password <password> --table TEST_SQOOP --hive-import --hive-table trade.test_sqoop --hive-drop-import-delims --incremental lastmodified --check-column LAST_MODIFIED --last-value "1970-01-01 00:01:01.0" -m 1

我提交的作业后，我正在低于错误。请帮助，我完全卡住了。

2014-12-19 09:19:40,928 INFO ActionStartXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]:start:] Start action [[email protected]:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-12-19 09:19:40,928 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action status=DONE 
2014-12-19 09:19:40,928 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action updated in DB! 
2014-12-19 09:19:40,950 INFO ActionEndXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]:start:] end executor for wf action 0000002-141008154216525-oozie-oozi-W with wf job 0000002-141008154216525-oozie-oozi-W 
2014-12-19 09:19:41,018 INFO ActionStartXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-12-19 09:19:41,179 WARN SqoopActionExecutor:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] credentials is null for the action 
2014-12-19 09:19:41,495 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Trying to get job [job_1415818455534_0502], attempt [1] 
2014-12-19 09:19:41,507 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] checking action, external ID [job_1415818455534_0502] status [RUNNING] 
2014-12-19 09:19:41,508 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action status=RUNNING 
2014-12-19 09:19:41,509 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB! 
2014-12-19 09:19:52,651 INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] callback for action [[email protected]] 
2014-12-19 09:19:52,714 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Trying to get job [job_1415818455534_0502], attempt [1] 
2014-12-19 09:19:52,714 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Trying to get job [job_1415818455534_0502], attempt [1] 
2014-12-19 09:19:52,778 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] action completed, external ID [null] 
2014-12-19 09:19:52,784 WARN SqoopActionExecutor:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] 
2014-12-19 09:19:52,808 INFO ActionEndXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] end executor for wf action 0000002-141008154216525-oozie-oozi-W with wf job 0000002-141008154216525-oozie-oozi-W 
2014-12-19 09:19:52,832 INFO ActionEndXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] ERROR is considered as FAILED for SLA 
2014-12-19 09:19:52,881 INFO ActionStartXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-12-19 09:19:52,882 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action status=DONE 
2014-12-19 09:19:52,882 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB! 
2014-12-19 09:19:52,904 INFO ActionEndXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] end executor for wf action 0000002-141008154216525-oozie-oozi-W with wf job 0000002-141008154216525-oozie-oozi-W 
2014-12-19 09:19:52,973 WARN CoordActionUpdateXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100

来源

2014-12-19 user2918922

+0

它看起来像一个权限问题给我。如果你确定你的sqoop命令在bash中有效。那么你可能想要检查你的用户帐户'scbengr2'是否在写入文件夹上有正确的权限。另外，确保您的工作流程可以访问连接器和hive-conf.xml文件。 – 2014-12-26 20:57:26

回答

0

色相目前推出的3 ways Sqoop：

  1. 职业，设计师
  2. Sqoop应用
  3. Oozie的应用

作业设计在内部使用了Oozie，因此输出你往上看。

我看到错误的命令的几件事情你上面提供：

  1. 该命令创建一个工作，而不是运行一个。
  2. 该命令包括sqoop，这将无法正常工作。
  3. 通过oozie创建作业需要external metastore，并且该作业以某种方式具有此配置。

解决方案：

  1. 除了使用metastore的，尽量只运行sqoop命令：import --connect jdbc:oracle:thin:@<ipaddress>:port/<database_name> --username <username> --password <password> --table TEST_SQOOP --hive-import --hive-table trade.test_sqoop --hive-drop-import-delims --incremental lastmodified --check-column LAST_MODIFIED --last-value "1970-01-01 00:01:01.0" -m 1。
  2. 而不是在命令之前添加sqoop，请尝试删除它：job --create TestHiveImport -- import --connect jdbc:oracle:thin:@<ipaddress>:port/<database_name> --username <username> --password <password> --table TEST_SQOOP --hive-import --hive-table trade.test_sqoop --hive-drop-import-delims --incremental lastmodified --check-column LAST_MODIFIED --last-value "1970-01-01 00:01:01.0" -m 1。
  3. 您可以将Sqoop Metastore信息添加到sqoop-site.xml。请咨询mailing lists寻求帮助。

来源

2014-12-28 18:35:21 abeaamase

相关问题

 相关问题