Oozie Hive行动挂起，心脏跳动永久

我试图通过我在Hue中创建的Oozie工作流运行Hive行动，但行动“心跳”永远不会执行Hive SQL。Oozie Hive行动挂起，心脏跳动永久

我已经阅读了其他关于心脏跳动的帖子，但是这个似乎是在SQL语句被解析后发生在不同的地方。我已经检查了集群中每个节点上的内存，并且验证了任务计数参数是合理的。

这里是蜂巢-config.xml文件：

<configuration> 

<property> 
    <name>javax.jdo.option.ConnectionURL</name> 
    <value>jdbc:hive://10.1.10.250:10000/testdb</value> 
    <description>JDBC connect string</description> 
</property> 

<property> 
    <name>javax.jdo.option.ConnectionDriverName</name> 
    <value>org.apache.hadoop.hive.jdbc.HiveDriver</value> 
    <description>JDBC driver</description> 
</property> 

</configuration>

我知道蜂巢连接工作，因为如果提供任何不好的SQL语句，一个错误的URL或坏的操作失败司机的名字。

这里是行动标准输出日志：

[...truncated] 

    ================================================================= 

    >>> Invoking Hive command line now >>> 

    4283 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=Driver.run from=org.apache.hadoop.hive.ql.Driver> 
    4284 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver> 
    4284 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver> 
    4339 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver> 
    4354 [main] INFO hive.ql.parse.ParseDriver - Parsing command: create table testdb.temp99 (col1 int) 
    4665 [main] INFO hive.ql.parse.ParseDriver - Parse Completed 
    4667 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - </PERFLOG method=parse start=1418968298270 end=1418968298598 duration=328 from=org.apache.hadoop.hive.ql.Driver> 
    4667 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver> 
    4733 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Starting Semantic Analysis 
    4735 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Creating table testdb.temp99 position=13 
    4760 [main] INFO org.apache.hadoop.hive.ql.Driver - Semantic Analysis Completed 
    4775 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - </PERFLOG method=semanticAnalyze start=1418968298598 end=1418968298706 duration=108 from=org.apache.hadoop.hive.ql.Driver> 
    4784 [main] INFO org.apache.hadoop.hive.ql.Driver - Returning Hive schema: Schema(fieldSchemas:null, properties:null) 
    4784 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - </PERFLOG method=compile start=1418968298215 end=1418968298715 duration=500 from=org.apache.hadoop.hive.ql.Driver> 
    4785 [main] INFO org.apache.hadoop.hive.ql.Driver - Concurrency mode is disabled, not creating a lock manager 
    4785 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver> 
    4785 [main] INFO org.apache.hadoop.hive.ql.Driver - Starting command: create table testdb.temp99 (col1 int) 
    4792 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - </PERFLOG method=TimeToSubmit start=1418968298215 end=1418968298723 duration=508 from=org.apache.hadoop.hive.ql.Driver> 
    4792 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver> 
    4792 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=task.DDL.Stage-0 from=org.apache.hadoop.hive.ql.Driver> 
    4815 [main] INFO hive.ql.exec.DDLTask - Default to LazySimpleSerDe for table testdb.temp99 
    4935 [main] INFO org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 
    4959 [main] INFO org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize called 
    5261 [main] INFO DataNucleus.Persistence - Property datanucleus.cache.level2 unknown - will be ignored 
    Heart beat 
    Heart beat 
    [...forever...]

为什么工作流心脏的跳动此时在日志中，而不是继续？

附录：

与此蜂巢行动相关的Oozie的工作流程是：

<workflow-app name="Hive-copy" xmlns="uri:oozie:workflow:0.4"> 
    <start to="Hive"/> 
    <action name="Hive"> 
     <hive xmlns="uri:oozie:hive-action:0.2"> 
      <job-tracker>${jobTracker}</job-tracker> 
      <name-node>${nameNode}</name-node> 
       <job-xml>/user/test/hive-config.xml</job-xml> 
      <script>/user/test/test.sql</script> 
      <file>hive-config.xml#hive-config.xml</file> 
     </hive> 
     <ok to="end"/> 
     <error to="kill"/> 
    </action> 
    <kill name="kill"> 
     <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> 
    </kill> 
    <end name="end"/> 
</workflow-app>

来源

2014-12-19 iwikilu

请分享您的工作流程Oozie的相关TOT他的蜂巢行动 – 2014-12-24 05:28:52

这个问题涉及到节点管理器。尝试设置在mapred-site.xml中文件中的下列财产修复该问题：

<property> 
<name>mapred.tasktracker.map.tasks.maximum</name> 
<value>50 </value> 
</property> 
<property> 
<name>mapred.tasktracker.map.tasks.maximum</name> 
<value>50 </value> 
</property>

正是类似这样的问题：https://groups.google.com/a/cloudera.org/forum/?fromgroups=#!topic/cdh-user/v0BHtQ0hlBg

来源

2014-12-31 10:07:11

我将指定的内容添加到“/etc/hadoop/conf.cloudera.yarn/mapred-site.xml”并重新启动纱线服务，但该解决方案对我无效。 – iwikilu 2015-01-03 21:53:47

集群中运行多少个nodemanager？ – 2015-01-05 05:48:33

Oozie Hive行动挂起，心脏跳动永久

回答

相关问题