hadoop版本不兼容

我在Hadoop 0.20.2版本上开发了一个带reduce方联接的MapReduce。所有的java类都编译时没有任何错误。我的程序正常运行。但是这个hadoop版本在测试虚拟机上。在我的真实集群上，我有另一个版本的hadoop（hadoop 2.00-cdh4.1.2）。当我想编译java类时，它不起作用。我得到100错误如下：hadoop版本不兼容

JoinMapper.java:8: package org.apache.hadoop.filecache does not exist 
import org.apache.hadoop.filecache.DistributedCache; 
           ^
JoinMapper.java:9: package org.apache.hadoop.fs does not exist 
import org.apache.hadoop.fs.Path; 
         ^
JoinMapper.java:10: package org.apache.hadoop.io does not exist 
import org.apache.hadoop.io.Text; 
         ^
JoinMapper.java:11: package org.apache.hadoop.io does not exist 
import org.apache.hadoop.io.*; 
^ 
JoinMapper.java:12: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.FileInputFormat; 
          ^
JoinMapper.java:13: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.FileOutputFormat; 
          ^
JoinMapper.java:14: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.InputFormat; 
          ^
JoinMapper.java:15: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.JobClient; 
          ^
JoinMapper.java:16: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.JobConf; 
          ^
JoinMapper.java:17: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.MapReduceBase; 
          ^
JoinMapper.java:18: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.Mapper; 
          ^
JoinMapper.java:19: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.OutputCollector; 
          ^
JoinMapper.java:20: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.Reporter; 
          ^
JoinMapper.java:21: package org.apache.hadoop.mapred does not exist 
import org.apache.hadoop.mapred.TextOutputFormat; 
          ^
JoinMapper.java:22: package org.apache.hadoop.mapreduce.lib.input does not exist 
import org.apache.hadoop.mapreduce.lib.input.MultipleInputs; 
              ^
JoinMapper.java:23: package org.apache.hadoop.util does not exist 
import org.apache.hadoop.util.Tool; 
          ^
JoinMapper.java:24: package org.apache.hadoop.util does not exist 
import org.apache.hadoop.util.ToolRunner; 
          ^
JoinMapper.java:26: cannot find symbol 
symbol: class MapReduceBase 
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> { 
           ^
JoinMapper.java:26: cannot find symbol 
symbol: class Mapper 
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> { 
                 ^
JoinMapper.java:26: cannot find symbol 
symbol: class LongWritable 
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> { 
                   ^
JoinMapper.java:26: cannot find symbol 
symbol: class Text 
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> { 
                      ^
TextPair.java:2: package org.apache.hadoop.io does not exist 
import org.apache.hadoop.io.*; 
^ 
TextPair.java:4: cannot find symbol 
symbol: class WritableComparable 
public class TextPair implements WritableComparable<TextPair> { 
           ^
TextPair.java:4: interface expected here 
public class TextPair implements WritableComparable<TextPair> { 
               ^
JoinMapper.java:26: cannot find symbol 
symbol: class Text 
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> { 
                          ^
JoinMapper.java:26: interface expected here 
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> { 
                  ^
NcdcRecordParser.java:2: package org.apache.hadoop.io does not exist 
import org.apache.hadoop.io.*;

我明白这个版本没有我需要的软件包。但我不知道如何解决我的问题。谁能帮我？

来源

2012-12-26 Tanya

什么罐子已经在你的classpath中？ –

你是如何编译的？（什么工具，Maven？如果是这样，你用什么依赖关系来试着用2.00进行编译？） –

对于maven，试试param -Dhadoop.profile = 23。另外，请参阅https://issues.apache.org/jira/browse/HBASE-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098596#comment-13098596

来源

2012-12-26 17:46:01 hudvin

据我所知，这些Java包仍然应该存在于最新版本的Hadoop中。

我认为你的问题是，hadoop罐不在你的classpath中。当我尝试在没有它们的情况下编译我的map reduce项目时，出现类似的错误。

如果您正在使用javac，尝试这样的事情（这使用路径位置我的Ubuntu CDH4 dev的机器）：

javac -classpath /usr/lib/hadoop/*:/usr/lib/hadoop-0.20-mapreduce/*:/usr/lib/hadoop-hdfs/* JoinMapper.java TextPair.java NcdcRecordParser.java

来源

2012-12-26 17:59:23

谢谢..我昨天也是这样做了。 – Tanya

的Apache的Hadoop做0.20之间及其内部包的主要的重新命名和0.23（也被称为2.0.0）。这包括将org.apache.hadoop.mapred重命名为org.apache.hadoop.mapreduce。

Hadoop的Cloudera发行版具有单独的Maven构件，您可以依靠它们与旧版本进行构建。工件名称在CDH4 Maven page上列出;你想要版本2.0.0-mr1-cdh4.1.2的文物。

另请参阅Is it better to use the mapred or the mapreduce package to create a Hadoop Job?了解有关是否需要移植到较新的API的一些详细信息。

来源

2012-12-26 18:48:46

hadoop版本不兼容

回答

相关问题