使用oozie的Hive加载数据操作

我在oozie workflow.xml中定义了hive操作，从hdfs路径执行数据加载。但是，幸运的是它不能正常工作。同样的脚本用于在配置单元中创建文件。请你们可以参考我的workflow.xml，job.properties，脚本文件，并纠正我是否有任何错误。任何帮助赞赏。提前致谢。
script.hql包含“将数据inpath'/../hdfs dir'载入到表测试中;”使用oozie的Hive加载数据操作

**workflow.xml** 


<workflow-app xmlns="uri:oozie:workflow:0.4" name="hive-wf"> 
    <start to="hive-action"/> 

    <action name="hive-action"> 
       <hive xmlns="uri:oozie:hive-action:0.2"> 
       <job-tracker>${jobtracker}</job-tracker> 
       <name-node>${namenode}</name-node> 
       <job-xml>hive-site.xml</job-xml> 
        <configuration> 

         <property> 
          <name>mapred.job.queue.name</name> 
          <value>${queueName}</value> 
         </property> 
         <property> 
           <name>oozie.hive.defaults</name> 
           <value>${namenode}/</value> 
         </property> 
         <property> 
          <name>mapred.reduce.tasks</name> 
           <value>2</value> 
         </property> 
        </configuration> 
       <script>script.hql</script> 
       </hive> 
       <ok to="end"/> 
       <error to="fail"/> 
    </action> 
    <kill name="fail"> 
     <message>Hive failed with some error.please look into that[${wf:errorMessage(wf:lastErrorNode())}]</message> 
    </kill> 

    <end name="end"/> 
</workflow-app> 



job.properties 
-------------- 

namenode=hdfs://namenodeipaddress:8020  
jobtracker=jobtrackeripaddress:8032  
queueName=default  
oozie.use.system.libpath=true  
oozie.libpath=${namenode}/user/oozie/share/lib  
oozie.wf.application.path=${namenode}/user/username/OozieScripts

请从oozie下面找到错误日志。

2014-11-24 11:07:43,984 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[HOSTNAME] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-141121151044934-oozie-oozi-W] ACTION[[email protected]] callback for action [[email protected]] 
2014-11-24 11:07:44,339 INFO org.apache.oozie.command.wf.ActionEndXCommand: SERVER[HOSTNAME] USER[USERNAME] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-141121151044934-oozie-oozi-W] ACTION[[email protected]] ERROR is considered as FAILED for SLA 
2014-11-24 11:07:44,391 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[HOSTNAME] USER[USERNAME] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-141121151044934-oozie-oozi-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-11-24 11:07:44,391 WARN org.apache.oozie.command.wf.ActionStartXCommand: SERVER[HOSTNAME] USER[USERNAME] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-141121151044934-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action status=DONE 
2014-11-24 11:07:44,391 WARN org.apache.oozie.command.wf.ActionStartXCommand: SERVER[HOSTNAME] USER[USERNAME] GROUP[-] TOKEN[] APP[hive-wf] JOB[0000005-141121151044934-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB!

来源

2014-11-21 Krishna Reddy

你会得到什么错误？你能提供日志吗？ – 2014-11-22 06:40:28

您能否提供错误日志？ – MarHserus 2014-11-24 04:17:52

请找到上面记录的错误代码。提前致谢。 – 2014-11-24 05:52:37

你必须定义一个蜂房default.xml中的文件来执行在Oozie的蜂巢脚本和文件已在workflow.xml被记载为

<property> 
         <name>mapred.job.queue.name</name> 
         <value>${queueName}</value> 
        </property> 
        <property> 
          <name>oozie.hive.defaults</name> 
          ***<value>/usr/foo/hive-0.6-default.xml</value>*** 
        </property> 
        <property> 
         <name>mapred.reduce.tasks</name> 
          <value>2</value> 
        </property>

欲了解更多信息请参阅，Hive workflow model

来源

2014-11-27 06:56:52

使用oozie的Hive加载数据操作

回答

相关问题