2013-03-24 87 views
1

我正在做一个Hadoop和HBase的测试项目。目前,群集在Windows计算机上托管2个Ubuntu VM。在HBase上运行MapReduce会给Zookeeper错误

我能够执行PUT,查询和remotly DELETE操作(在我的主机),使用以下的HBase的Java API配置

config = HBaseConfiguration.create(); 
config.set("hbase.zookeeper.quorum", "192.168.56.90"); 
config.set("hbase.zookeeper.property.clientPort", "2222"); 

当我试图用同样运行在Windows上HBase的MapReduce工作如上配置,我收到以下错误

13/03/24 06:11:03 ERROR security.UserGroupInformation: PriviledgedActionException as:Joel cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Joel\mapred\staging\Joel290889388\.staging to 0700 
java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Joel\mapred\staging\Joel290889388\.staging to 0700 

从我已阅读在网络上,有似乎是在Windows上运行MapReduce作业的问题。所以我尝试使用“java-jar MR.jar”在Linux上运行MapReduce作业。

在Linux上,我无法连接到Zookeeper。对于未知的原因,动物园管理员主机和端口越来越在客户端

13/03/24 05:59:33 INFO zookeeper.ZooKeeper: Client environment:os.version=3.5.0-23-generic 
13/03/24 05:59:33 INFO zookeeper.ZooKeeper: Client environment:user.name=hduser 
13/03/24 05:59:33 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hduser 
13/03/24 05:59:33 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/hduser/testes 
13/03/24 05:59:33 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192.168.56.90:2222 sessionTimeout=180000 watcher=hconnection 
13/03/24 05:59:33 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is [email protected] 
13/03/24 05:59:33 INFO zookeeper.ClientCnxn: Opening socket connection to server node01/192.168.56.90:2222. Will not attempt to authenticate using SASL (unknown error) 
13/03/24 05:59:33 INFO zookeeper.ClientCnxn: Socket connection established to node01/192.168.56.90:2222, initiating session 
13/03/24 05:59:33 INFO zookeeper.ClientCnxn: Session establishment complete on server node01/192.168.56.90:2222, sessionid = 0x13d9afaa1a30006, negotiated timeout = 180000 
13/03/24 05:59:33 INFO client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x13d9afaa1a30006 
13/03/24 05:59:33 INFO zookeeper.ZooKeeper: Session: 0x13d9afaa1a30006 closed 
13/03/24 05:59:33 INFO zookeeper.ClientCnxn: EventThread shut down 
13/03/24 05:59:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
13/03/24 05:59:33 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 
13/03/24 05:59:33 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection 
13/03/24 05:59:33 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is [email protected] 
13/03/24 05:59:33 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 
13/03/24 05:59:33 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect 
java.net.ConnectException: Connection refused 
     at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) 
     at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692) 
     at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) 

从日志上面来看reseted,它正确地连接到NODE01:2222(NODE01解析为192.168.56.90)。但由于某种原因,它会更改为localhost:2181,然后会发出拒绝连接错误。

我该如何解决这个问题,以便在Zookeeper正在运行的同一台机器上运行Linux上的MR作业?

版本:Hbase 0.94.5/Hadoop 1.1.2

谢谢。

回答

1

您可能还需要设置hbase.master。

还检查/ etc/hosts文件,看看它是否正确。你能够使用该连接信息远程登录到动物园管理员吗?

config.set("hbase.zookeeper.quorum", "192.168.56.90"); 
config.set("hbase.zookeeper.property.clientPort", "2222"); 
config.set("hbase.master", "some.host.com:60000")