2017-08-21 26 views
0

我正在练习克卢代拉纱VMware Player(非商业用途)。 我在猪脚本是,在猪得到错误为'错误编译运算符POLocalRearrange'

a1 = load '/user/training/my_hdfs/id' using PigStorage('\t') as(id:int,name:chararray,desig:chararray); a2 = load '/user/training/my_hdfs/trips' using PigStorage('\t') as(id:int,place:chararray,no_trips:int); a3 = join a1 by id,a2 by id; a4 = group a3 by a1::id; illustrate a4;

后说明它被示出作为报文,

2017-08-21 07:52:11,926 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Encountered IOException. Exception : Error compiling operator POLocalRearrange

数据集是,

Table id 101 aaa executive 102 bbb manager 104 hhh manager 106 ccc trainee 109 hhh trainee

Table trips 101 pune 1 101 hyd 2 102 pune 2 102 hyd 3 102 bang 4

回答

0

当我试着用提供的数据运行你的程序时,我也得到一些错误,因为你文件中的分隔符不一致。一些地方的空间和一些地方的选项卡(可能是因为复制粘贴)。我使分隔符通用(使用标签),一切正常。

尝试使用dump a1或转储a2并查看是否可以在正确的列中看到数据。 对我来说,做分隔符通用之后的工作完美,并说明A4给出了以下的输出:

------------------------------------------------------------------ 
| a1  | id:int  | name:chararray  | desig:chararray  | 
------------------------------------------------------------------ 
|  | 101  | aaa    | executive   | 
|  | 101  | aaa    | executive   | 
------------------------------------------------------------------ 
---------------------------------------------------------------- 
| a2  | id:int  | place:chararray  | no_trips:int  | 
---------------------------------------------------------------- 
|  | 101  | pune    | 1    | 
|  | 101  | hyd     | 2    | 
---------------------------------------------------------------- 
------------------------------------------------------------------------------------------------------------------------------------------------ 
| a3  | a1::id:int  | a1::name:chararray  | a1::desig:chararray  | a2::id:int  | a2::place:chararray  | a2::no_trips:int  | 
------------------------------------------------------------------------------------------------------------------------------------------------ 
|  | 101   | aaa     | executive    | 101   | pune     | 1     | 
|  | 101   | aaa     | executive    | 101   | hyd      | 2     | 
|  | 101   | aaa     | executive    | 101   | pune     | 1     | 
|  | 101   | aaa     | executive    | 101   | hyd      | 2     | 
------------------------------------------------------------------------------------------------------------------------------------------------ 
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 
| a4  | group:int  | a3:bag{:tuple(a1::id:int,a1::name:chararray,a1::desig:chararray,a2::id:int,a2::place:chararray,a2::no_trips:int)}         | 
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 
|  | 101   | {(101, ..., 1), ..., (101, ..., 2)}                            | 
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 
+0

嗨user3359790,我真的很感谢您的回复,数据是在我的电脑居然一致,在复制粘贴此它成为不一致,我终于尝试了很多东西,我卸载了我的VM播放器,然后再次重新安装了所有的东西。令我惊讶的是,它正常工作并给出了输出结果,如您所示。我不明白是什么问题,但卸载并再次安装新鲜的东西解决了问题。我再次感谢你所表达的关切。 – user8167344

+0

很高兴知道它的工作 – OneUser