使用elephantbird读取JSON时出错 - 猪

我无法使用ElephantBird和Pig读取JSON文件。我想知道我犯了什么错误。使用elephantbird读取JSON时出错 - 猪

数据：

{ "nrcpts": "1", 
    "src": "[email protected]", 
    "sendmailid": "p6D0r0u1006229", 
    "relay": "app03.example.com", 
    "classnumber": "0", 
    "msgid": "WARQZCXAEMSSVWPPOOYZXR 
[email protected]", 
    "pid": "6229", 
    "month": "Jul", 
    "time": "20:53:00", 
    "day": "12", 
    "mailserver": "mail5", 
    "size": "57395" 
}

代码：

json1 = load '/user/hdetl/funnel/uetsample.dat' using com.twitter.elephantbird.pig.load.JsonLoader(); 

dat = FOREACH json1 GENERATE $0#'mailserver' AS mailserver; 
dump dat;

错误：

Input(s): 
Failed to read data from "/user/hdetl/funnel/uetsample.dat" 

detailed error : 
Pig Stack Trace 
--------------- 
ERROR 2997: Unable to recreate exception from backed error: Error: in 

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias dat. Backend error : Unable to recreate exception from back 
ed error: Error: in 
     at org.apache.pig.PigServer.openIterator(PigServer.java:891) 
     at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:655) 
     at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303) 
     at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188) 
     at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164) 
     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) 
     at org.apache.pig.Main.run(Main.java:495) 
     at org.apache.pig.Main.main(Main.java:111) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
     at java.lang.reflect.Method.invoke(Method.java:597) 
     at org.apache.hadoop.util.RunJar.main(RunJar.java:186) 
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2997: Unable to recreate exception from backed error: Error: in 
     at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:221) 
     at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:151) 
     at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:344) 
     at org.apache.pig.PigServer.launchPlan(PigServer.java:1314) 
     at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1299) 
     at org.apache.pig.PigServer.storeEx(PigServer.java:996) 
     at org.apache.pig.PigServer.store(PigServer.java:963) 
     at org.apache.pig.PigServer.openIterator(PigServer.java:876)

来源

2012-02-24 Bharathi

出口JAVA_HOME =的/ usr/JAVA/jdk1.6.0_22 出口PIG_CLASSPATH =的/ etc/Hadoop的/ conf目录出口PATH = $ PATH：/local/hdetl/pig-0.9.2/bin REGISTER /local/hdetl/funnel/pig-jars/json-simple-1.1.jar; 注册/local/hdetl/funnel/pig-jars/google-collect-1.0.jar; REGISTER'/local/hdetl/funnel/pig-jars/elephant-bird-1.2.1-SNAPSHOT.jar'; – Bharathi 2012-02-24 22:25:59

你的问题是什么？ – 2012-02-24 22:44:57

我不能使用elephantbird和PIG来读取JSON文件。我想知道在哪里犯错误。 – Bharathi 2012-02-24 22:55:03

我没有使用过的JSON装载机但我想你应该能够降在你的foreach中$ 0。我只是认为装载程序只是将{和}之间的所有内容变成单个记录（Tuple）。

 
dat = FOREACH json1 GENERATE mailserver;

来源

2012-02-28 02:30:38 NerdyNick

相当旧的帖子，但有人可能有类似的问题。

我从问题中提供的数据创建了输入文件。
我无法加载，因为不必要的泰德文件输入行：

"msgid": "WARQZCXAEMSSVWPPOOYZXR 
[email protected]",

但固定是没有得到预期的结果。我已经从文件中删除了所有输入，所以最终我只有一行。

文件加载：

dump json1 
([time#20:53:00,msgid#WARQZCXAEMSSVWPPOOYZXRLQIKMFU[email protected],relay#app03.example.com,mailserver#mail5,month#Jul,pid#6229,classnumber#0,day#12,src#[email protected],sendmailid#p6D0r0u1006229,nrcpts#1,size#57395])

，你foreach作品：

dat = FOREACH json1 GENERATE $0#'mailserver' AS mailserver; 
dump dat 

(mail5)

来源

2014-10-03 12:32:47

使用elephantbird读取JSON时出错 - 猪

回答

相关问题