1. 首页
  2. 问答
  3. Hadoop程序运行良好，使用“java -jar”，但没有使用“hadoop jar”

Hadoop程序运行良好，使用“java -jar”，但没有使用“hadoop jar”

2014-09-10 383 views
0

我编写了一个连接到两个HBase数据库的MapReduce程序。我已经将它写在Eclipse上，并且已经使用“Runnable Jar”选项（包含所有库）将其导出。它与java -jar命令运行良好。输出是下一个：Hadoop程序运行良好，使用“java -jar”，但没有使用“hadoop jar”

$ java -jar mr.jar 
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). 
log4j:WARN Please initialize the log4j system properly. 
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

然后预期的输出。但是，如果我用hadoop jar命令运行相同的程序，我不知道发生了什么，但我猜不出什么。预期的输出不显示。

$ hadoop jar mr.jar 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class 
14/09/10 17:22:07 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name 
14/09/10 17:22:07 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class 
14/09/10 17:22:07 INFO Configuration.deprecation: dfs.df.interval is deprecated. Instead, use fs.df.interval 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.task.tracker.http.address is deprecated. Instead, use mapreduce.tasktracker.http.address 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.userlog.retain.hours is deprecated. Instead, use mapreduce.job.userlog.retain.hours 
14/09/10 17:22:07 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.local.dir.minspacestart is deprecated. Instead, use mapreduce.tasktracker.local.dir.minspacestart 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.shuffle.read.timeout is deprecated. Instead, use mapreduce.reduce.shuffle.read.timeout 
14/09/10 17:22:07 INFO Configuration.deprecation: io.sort.spill.percent is deprecated. Instead, use mapreduce.map.sort.spill.percent 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.reduce.parallel.copies is deprecated. Instead, use mapreduce.reduce.shuffle.parallelcopies 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.submit.replication is deprecated. Instead, use mapreduce.client.submit.file.replication 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.local.dir.minspacekill is deprecated. Instead, use mapreduce.tasktracker.local.dir.minspacekill 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.task.profile is deprecated. Instead, use mapreduce.task.profile 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.heartbeats.in.second is deprecated. Instead, use mapreduce.jobtracker.heartbeats.in.second 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.healthChecker.interval is deprecated. Instead, use mapreduce.tasktracker.healthchecker.interval 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.task.timeout is deprecated. Instead, use mapreduce.task.timeout 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.temp.dir is deprecated. Instead, use mapreduce.cluster.temp.dir 
14/09/10 17:22:07 INFO Configuration.deprecation: jobclient.completion.poll.interval is deprecated. Instead, use mapreduce.client.completion.pollinterval 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.tracker.persist.jobstatus.active is deprecated. Instead, use mapreduce.jobtracker.persist.jobstatus.active 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.output.compression.codec is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.codec 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.shuffle.merge.percent is deprecated. Instead, use mapreduce.reduce.shuffle.merge.percent 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.map.max.attempts is deprecated. Instead, use mapreduce.map.maxattempts 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.reduce.input.buffer.percent is deprecated. Instead, use mapreduce.reduce.input.buffer.percent 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.task.cache.levels is deprecated. Instead, use mapreduce.jobtracker.taskcache.levels 
14/09/10 17:22:07 INFO Configuration.deprecation: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.jobtracker.instrumentation is deprecated. Instead, use mapreduce.jobtracker.instrumentation 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.userlog.limit.kb is deprecated. Instead, use mapreduce.task.userlog.limit.kb 
14/09/10 17:22:07 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.speculative.execution.slowNodeThreshold is deprecated. Instead, use mapreduce.job.speculative.slownodethreshold 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.reduce.memory.mb is deprecated. Instead, use mapreduce.reduce.memory.mb 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.skip.map.max.skip.records is deprecated. Instead, use mapreduce.map.skip.maxrecords 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.tracker.jobhistory.lru.cache.size is deprecated. Instead, use mapreduce.jobtracker.jobhistory.lru.cache.size 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.tracker.persist.jobstatus.hours is deprecated. Instead, use mapreduce.jobtracker.persist.jobstatus.hours 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.tracker.handler.count is deprecated. Instead, use mapreduce.jobtracker.handler.count 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use mapreduce.reduce.markreset.buffer.percent 
14/09/10 17:22:07 INFO Configuration.deprecation: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.task.profile.maps is deprecated. Instead, use mapreduce.task.profile.maps 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.reduce.child.java.opts is deprecated. Instead, use mapreduce.reduce.java.opts 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.dns.nameserver is deprecated. Instead, use mapreduce.tasktracker.dns.nameserver 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.taskmemorymanager.monitoring-interval is deprecated. Instead, use mapreduce.tasktracker.taskmemorymanager.monitoringinterval 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.expiry.interval is deprecated. Instead, use mapreduce.jobtracker.expire.trackers.interval 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.max.tracker.failures is deprecated. Instead, use mapreduce.job.maxtaskfailures.per.tracker 
14/09/10 17:22:07 INFO Configuration.deprecation: mapreduce.jobtracker.split.metainfo.maxsize is deprecated. Instead, use mapreduce.job.split.metainfo.maxsize 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.tracker.persist.jobstatus.dir is deprecated. Instead, use mapreduce.jobtracker.persist.jobstatus.dir 
14/09/10 17:22:07 INFO Configuration.deprecation: job.end.retry.attempts is deprecated. Instead, use mapreduce.job.end-notification.retry.attempts 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 
14/09/10 17:22:07 INFO Configuration.deprecation: mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.task.tracker.task-controller is deprecated. Instead, use mapreduce.tasktracker.taskcontroller 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.jobtracker.maxtasks.per.job is deprecated. Instead, use mapreduce.jobtracker.maxtasks.perjob 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.reduce.child.log.level is deprecated. Instead, use mapreduce.reduce.log.level 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.reduce.max.attempts is deprecated. Instead, use mapreduce.reduce.maxattempts 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.map.output.compression.codec is deprecated. Instead, use mapreduce.map.output.compress.codec 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.shuffle.input.buffer.percent is deprecated. Instead, use mapreduce.reduce.shuffle.input.buffer.percent 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.task.tracker.report.address is deprecated. Instead, use mapreduce.tasktracker.report.address 
14/09/10 17:22:07 INFO Configuration.deprecation: keep.failed.task.files is deprecated. Instead, use mapreduce.task.files.preserve.failedtasks 
14/09/10 17:22:07 INFO Configuration.deprecation: tasktracker.http.threads is deprecated. Instead, use mapreduce.tasktracker.http.threads 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.speculative.execution.slowTaskThreshold is deprecated. Instead, use mapreduce.job.speculative.slowtaskthreshold 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.acls.enabled is deprecated. Instead, use mapreduce.cluster.acls.enabled 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.map.child.java.opts is deprecated. Instead, use mapreduce.map.java.opts 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.max.tracker.blacklists is deprecated. Instead, use mapreduce.jobtracker.tasktracker.maxblacklists 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.indexcache.mb is deprecated. Instead, use mapreduce.tasktracker.indexcache.mb 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.skip.attempts.to.start.skipping is deprecated. Instead, use mapreduce.task.skip.start.attempts 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.reduce.tasks.maximum is deprecated. Instead, use mapreduce.tasktracker.reduce.tasks.maximum 
14/09/10 17:22:07 INFO Configuration.deprecation: jobclient.output.filter is deprecated. Instead, use mapreduce.client.output.filter 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.jobtracker.restart.recover is deprecated. Instead, use mapreduce.jobtracker.restart.recover 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.speculative.execution.speculativeCap is deprecated. Instead, use mapreduce.job.speculative.speculativecap 
14/09/10 17:22:07 INFO Configuration.deprecation: jobclient.progress.monitor.poll.interval is deprecated. Instead, use mapreduce.client.progressmonitor.pollinterval 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.map.child.log.level is deprecated. Instead, use mapreduce.map.log.level 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.output.compression.type is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.type 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.tracker.retiredjobs.cache.size is deprecated. Instead, use mapreduce.jobtracker.retiredjobs.cache.size 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.dns.interface is deprecated. Instead, use mapreduce.tasktracker.dns.interface 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.task.profile.reduces is deprecated. Instead, use mapreduce.task.profile.reduces 
14/09/10 17:22:07 INFO Configuration.deprecation: job.end.retry.interval is deprecated. Instead, use mapreduce.job.end-notification.retry.interval 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.jobtracker.job.history.block.size is deprecated. Instead, use mapreduce.jobtracker.jobhistory.block.size 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.child.tmp is deprecated. Instead, use mapreduce.task.tmp.dir 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use mapreduce.job.committer.setup.cleanup.needed 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.map.tasks.maximum is deprecated. Instead, use mapreduce.tasktracker.map.tasks.maximum 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.queue.name is deprecated. Instead, use mapreduce.job.queuename 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.jobtracker.taskScheduler is deprecated. Instead, use mapreduce.jobtracker.taskscheduler 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.skip.reduce.max.skip.groups is deprecated. Instead, use mapreduce.reduce.skip.maxgroups 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.tracker.http.address is deprecated. Instead, use mapreduce.jobtracker.http.address 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.healthChecker.script.timeout is deprecated. Instead, use mapreduce.tasktracker.healthchecker.script.timeout 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.instrumentation is deprecated. Instead, use mapreduce.tasktracker.instrumentation 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.reuse.jvm.num.tasks is deprecated. Instead, use mapreduce.job.jvm.numtasks 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.inmem.merge.threshold is deprecated. Instead, use mapreduce.reduce.merge.inmem.threshold 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.job.map.memory.mb is deprecated. Instead, use mapreduce.map.memory.mb 
14/09/10 17:22:07 INFO Configuration.deprecation: topology.script.number.args is deprecated. Instead, use net.topology.script.number.args 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.reduce.slowstart.completed.maps is deprecated. Instead, use mapreduce.job.reduce.slowstart.completedmaps 
14/09/10 17:22:07 INFO Configuration.deprecation: dfs.umaskmode is deprecated. Instead, use fs.permissions.umask-mode 
14/09/10 17:22:07 INFO Configuration.deprecation: topology.node.switch.mapping.impl is deprecated. Instead, use net.topology.node.switch.mapping.impl 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.tasktracker.tasks.sleeptime-before-sigkill is deprecated. Instead, use mapreduce.tasktracker.tasks.sleeptimebeforesigkill 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.merge.recordsBeforeProgress is deprecated. Instead, use mapreduce.task.merge.progress.records 
14/09/10 17:22:07 INFO Configuration.deprecation: mapred.shuffle.connect.timeout is deprecated. Instead, use mapreduce.reduce.shuffle.connect.timeout 
14/09/10 17:22:08 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS 
14/09/10 17:22:08 INFO Configuration.deprecation: mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max 
14/09/10 17:22:08 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.max.objects is deprecated. Instead, use dfs.namenode.max.objects 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.data.dir is deprecated. Instead, use dfs.datanode.data.dir 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.name.dir is deprecated. Instead, use dfs.namenode.name.dir 
14/09/10 17:22:08 INFO Configuration.deprecation: fs.checkpoint.dir is deprecated. Instead, use dfs.namenode.checkpoint.dir 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.block.size is deprecated. Instead, use dfs.blocksize 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.access.time.precision is deprecated. Instead, use dfs.namenode.accesstime.precision 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.replication.min is deprecated. Instead, use dfs.namenode.replication.min 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.name.edits.dir is deprecated. Instead, use dfs.namenode.edits.dir 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.replication.considerLoad is deprecated. Instead, use dfs.namenode.replication.considerLoad 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.balance.bandwidthPerSec is deprecated. Instead, use dfs.datanode.balance.bandwidthPerSec 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.safemode.threshold.pct is deprecated. Instead, use dfs.namenode.safemode.threshold-pct 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.http.address is deprecated. Instead, use dfs.namenode.http-address 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.name.dir.restore is deprecated. Instead, use dfs.namenode.name.dir.restore 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.https.client.keystore.resource is deprecated. Instead, use dfs.client.https.keystore.resource 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.backup.address is deprecated. Instead, use dfs.namenode.backup.address 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.backup.http.address is deprecated. Instead, use dfs.namenode.backup.http-address 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.permissions is deprecated. Instead, use dfs.permissions.enabled 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.safemode.extension is deprecated. Instead, use dfs.namenode.safemode.extension 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.datanode.max.xcievers is deprecated. Instead, use dfs.datanode.max.transfer.threads 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.https.need.client.auth is deprecated. Instead, use dfs.client.https.need-auth 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.https.address is deprecated. Instead, use dfs.namenode.https-address 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.replication.interval is deprecated. Instead, use dfs.namenode.replication.interval 
14/09/10 17:22:08 INFO Configuration.deprecation: fs.checkpoint.edits.dir is deprecated. Instead, use dfs.namenode.checkpoint.edits.dir 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.write.packet.size is deprecated. Instead, use dfs.client-write-packet-size 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.permissions.supergroup is deprecated. Instead, use dfs.permissions.superusergroup 
14/09/10 17:22:08 INFO Configuration.deprecation: dfs.secondary.http.address is deprecated. Instead, use dfs.namenode.secondary.http-address 
14/09/10 17:22:08 INFO Configuration.deprecation: fs.checkpoint.period is deprecated. Instead, use dfs.namenode.checkpoint.period 
14/09/10 17:22:08 INFO client.RMProxy: Connecting to ResourceManager at /192.168.0.1:8032 
14/09/10 17:22:08 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 
14/09/10 17:22:08 INFO Configuration.deprecation: mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max 
14/09/10 17:22:08 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class 
14/09/10 17:22:08 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 
14/09/10 17:22:08 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS 
14/09/10 17:22:08 INFO Configuration.deprecation: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class 
14/09/10 17:22:08 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:host.name=xxx 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_55 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.55.x86_64/jre 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:java.class.path=[...] 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/hadoop/hadoop-2.2.0/lib/native 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-431.17.1.el6.x86_64 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:user.name=xxx 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/xxx 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/xxx/programas 
14/09/10 17:22:08 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0x18dddc42, quorum=localhost:2181, baseZNode=/hbase 
14/09/10 17:22:08 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x18dddc42 connecting to ZooKeeper ensemble=localhost:2181 
14/09/10 17:22:08 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 
14/09/10 17:22:08 INFO zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session 
14/09/10 17:22:08 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x14859c8510b004c, negotiated timeout = 90000 
14/09/10 17:22:09 INFO mapreduce.TableOutputFormat: Created table instance for s 
14/09/10 17:22:09 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 
14/09/10 17:22:11 INFO util.RegionSizeCalculator: Calculating region sizes for table "e". 
14/09/10 17:22:11 WARN mapreduce.TableInputFormatBase: Cannot resolve the host name for hadoop2/192.168.0.2 because of javax.naming.NameNotFoundException: DNS name not found [response code 3]; remaining name '2.0.168.192.in-addr.arpa' 
14/09/10 17:22:12 INFO mapreduce.JobSubmitter: number of splits:1 
14/09/10 17:22:12 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files 
14/09/10 17:22:12 INFO Configuration.deprecation: mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max 
14/09/10 17:22:12 INFO Configuration.deprecation: mapred.cache.files.timestamps is deprecated. Instead, use mapreduce.job.cache.files.timestamps 
14/09/10 17:22:12 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum 
14/09/10 17:22:12 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 
14/09/10 17:22:12 INFO Configuration.deprecation: mapred.job.classpath.files is deprecated. Instead, use mapreduce.job.classpath.files 
14/09/10 17:22:12 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name 
14/09/10 17:22:12 INFO Configuration.deprecation: mapred.cache.files.filesizes is deprecated. Instead, use mapreduce.job.cache.files.filesizes 
14/09/10 17:22:12 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS 
14/09/10 17:22:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1405689757338_0555 
14/09/10 18:26:53 INFO impl.YarnClientImpl: Submitted application application_1405689757338_0560 to ResourceManager at /192.168.0.1:8032 
14/09/10 18:26:53 INFO mapreduce.Job: The url to track the job: http://xxx.xxx.xxx:8088/proxy/application_1405689757338_0560/ 
14/09/10 18:26:53 INFO mapreduce.Job: Running job: job_1405689757338_0560 
14/09/10 18:27:18 INFO mapreduce.Job: Job job_1405689757338_0560 running in uber mode : false 
14/09/10 18:27:18 INFO mapreduce.Job: map 0% reduce 0% 
14/09/10 18:37:46 INFO mapreduce.Job: Task Id : attempt_1405689757338_0560_m_000000_0, Status : FAILED 
AttemptID:attempt_1405689757338_0560_m_000000_0 Timed out after 600 secs 
14/09/10 18:48:17 INFO mapreduce.Job: Task Id : attempt_1405689757338_0560_m_000000_1, Status : FAILED 
AttemptID:attempt_1405689757338_0560_m_000000_1 Timed out after 600 secs 
14/09/10 18:58:47 INFO mapreduce.Job: Task Id : attempt_1405689757338_0560_m_000000_2, Status : FAILED 
AttemptID:attempt_1405689757338_0560_m_000000_2 Timed out after 600 secs 
14/09/10 19:09:18 INFO mapreduce.Job: map 100% reduce 100% 
14/09/10 19:09:19 INFO mapreduce.Job: Job job_1405689757338_0560 failed with state FAILED due to: Task failed task_1405689757338_0560_m_000000 
Job failed as tasks failed. failedMaps:1 failedReduces:0 

14/09/10 19:09:19 INFO mapreduce.Job: Counters: 9 
    Job Counters 
     Failed map tasks=4 
     Launched map tasks=4 
     Other local map tasks=3 
     Rack-local map tasks=1 
     Total time spent by all maps in occupied slots (ms)=25139550 
     Total time spent by all reduces in occupied slots (ms)=0 
    Map-Reduce Framework 
     CPU time spent (ms)=0 
     Physical memory (bytes) snapshot=0 
     Virtual memory (bytes) snapshot=0 
Exception in thread "main" java.io.IOException: JOB ERROR. 
    at Principal.main(Principal.java:44) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

非常感谢。

来源

2014-09-10 brunneis

+0

以下链接可能有帮助 http://stackoverflow.com/questions/25180142/what-difference-between-execute-a-map-reduce-job-using-hadoop-and-java-command/25182075# 25182075 – 2014-09-11 07:10:13

回答

0

与您的hadoop安装相比，您的输出看起来像您有一个较旧的配置文件。如果您尚未自定义配置文件，则可能需要刷新;否则，你将不得不通过所有的警告，并确保它们不重要。

您的错误输出结束表示您正在运行的程序具有某种类型的本机编译组件。如果您将它提交给分布式映射/减少集群，系统如何知道如何调用您的本地库？本机库是否分发到所有节点，并位于相同的文件位置？您是否设置了java本地库路径，以便每次执行map/reduce任务时都将这个位置包含在每个系统中？

来源

2014-09-11 03:41:25 David

+0

我试过这个：'TableMapReduceUtil.addHBaseDependencyJars（config）;'和this：'TableMapReduceUtil.addDependencyJars（job）;'在我的代码上。但没有新东西。我在一个集群中运行程序。它没有基于它，所以管理员几天前安装它（0.98），并运行hadoop 2.2.0。我也用'-libjars ./mr_lib/*。jar'运行jar。我不知道通过设置java本地库路径是什么意思。谢谢。 – brunneis 2014-09-12 16:11:24

0

看起来，hadoop默认罐子不会从hadoop类路径中获取，请确保jar中添加的罐子在hadoop类路径中被引用。

来源

2014-09-12 10:09:24 Harry

+0

这意味着我使用的所有额外罐（hbase）都必须列在hadoop类路径中？谢谢。 – brunneis 2014-09-12 16:12:41

相关问题

 相关问题