Hadoop计数器文档？

Question

我MapReduce作业结束后，我收到了一大堆的Counter信息：

File System Counters 
       FILE: Number of bytes read=4386096368 
       FILE: Number of bytes written=8805370803 
       FILE: Number of read operations=0 
       FILE: Number of large read operations=0 
       FILE: Number of write operations=0 
       HDFS: Number of bytes read=54583718086 
       HDFS: Number of bytes written=4382090874 
       HDFS: Number of read operations=1479 
       HDFS: Number of large read operations=0 
       HDFS: Number of write operations=2 
     Job Counters 
       Launched map tasks=369 
       Launched reduce tasks=1 
       Data-local map tasks=369 
       Total time spent by all maps in occupied slots (ms)=34288552 
       Total time spent by all reduces in occupied slots (ms)=232084 
       Total time spent by all map tasks (ms)=8572138 
       Total time spent by all reduce tasks (ms)=58021 
       Total vcore-seconds taken by all map tasks=8572138 
       Total vcore-seconds taken by all reduce tasks=58021 
       Total megabyte-seconds taken by all map tasks=35111477248 
       Total megabyte-seconds taken by all reduce tasks=237654016 
     Map-Reduce Framework 
       Map input records=14753874 
       Map output records=666776 
       Map output bytes=4383426830 
       Map output materialized bytes=4386098552 
       Input split bytes=47970 
       Combine input records=0 
       Combine output records=0 
       Reduce input groups=1 
       Reduce shuffle bytes=4386098552 
       Reduce input records=666776 
       Reduce output records=666776 
       Spilled Records=1333552 
       Shuffled Maps =369 
       Failed Shuffles=0 
       Merged Map outputs=369 
       GC time elapsed (ms)=1121584 
       CPU time spent (ms)=23707900 
       Physical memory (bytes) snapshot=152915259392 
       Virtual memory (bytes) snapshot=2370755190784 
       Total committed heap usage (bytes)=126644912128 
     Shuffle Errors 
       BAD_ID=0 
       CONNECTION=0 
       IO_ERROR=0 
       WRONG_LENGTH=0 
       WRONG_MAP=0 
       WRONG_REDUCE=0 
     File Input Format Counters 
       Bytes Read=49449743227 
     File Output Format Counters 
       Bytes Written=4382090874 

我在哪里可以找到什么这些字段的意思解释？其中一些非常明显（Number of bytes read），但其他一些更模糊（Total time spent by all maps in occupied slots vs Total time spent by all map tasks）。

我发现了一个list of all the default counters，但我似乎无法找到他们的解释或描述。

我很惊讶，我似乎无法轻松找到有关此输出的文档。任何人都可以提供链接或解释？

Answer 1

Hadoop: The Definitive Guide的第8章（完整的PDF链接华盛顿州立大学），提供了计数器的详细信息，与MapReduce。这从225页开始，并在表8-1中列出。此资源的更新版本（第4版）可在Safari Books Online获得（您需要先登录）。