2016-08-16 75 views
4

销售激励类减速,MapReduce的,不显示错误,但没有给出所需的输出

package mr.map; 

import org.apache.hadoop.conf.Configuration; 
import org.apache.hadoop.fs.Path; 
import org.apache.hadoop.mapreduce.Job; 
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; 
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; 

import org.apache.hadoop.io.Text; 
import org.apache.hadoop.io.FloatWritable; 
//import org.apache.hadoop.mapreduce.Mapper; 
//import org.apache.hadoop.mapreduce.Reducer; 

public class SalesDriver 
{ 
    public static void main(String args[]) throws Exception 
    { 
     Configuration c=new Configuration(); 
     Job j=new Job(c,"Sales"); 

     j.setJarByClass(SalesDriver.class); 
     j.setMapperClass(SalesMapper.class); 
     j.setReducerClass(SalesReducer.class); 

     //j.setNumReduceTasks(0); 
     j.setOutputKeyClass(Text.class); 
     j.setOutputValueClass(FloatWritable.class); 

     Path in=new Path(args[0]); 
     Path out=new Path(args[1]); 

     FileInputFormat.addInputPath(j, in); 
     FileOutputFormat.setOutputPath(j, out); 

     System.exit(j.waitForCompletion(true)?0:1); 
    } 
} 

销售映射类

package mr.map; 

import java.io.IOException; 

//import org.apache.hadoop.io.IntWritable; 
import org.apache.hadoop.io.FloatWritable; 
import org.apache.hadoop.io.LongWritable; 
//import org.apache.hadoop.io.DoubleWritable; 
import org.apache.hadoop.io.Text; 
import org.apache.hadoop.mapreduce.Mapper; 

public class SalesMapper extends Mapper<LongWritable, Text, Text, FloatWritable> 
{ 
    public void map(LongWritable k, Text v, Context con) throws IOException, InterruptedException 
    { 
     String w[]=v.toString().split(" "); 
     String product=w[3]; 
     //String store=w[2]; 
     //float cost=Integer.parseInt(w[4]); 
     float costx = Float.parseFloat(w[4]); 

     //String newline= product+","+store; //","+costx; 
     //String newline = product; 
     con.write(new Text(product), new FloatWritable(costx)); 
    } 
} 

销售减速类

package mr.map; 

import java.io.IOException; 

import org.apache.hadoop.io.FloatWritable; 
//import org.apache.hadoop.io.IntWritable; 
//import org.apache.hadoop.io.DoubleWritable; 
import org.apache.hadoop.io.Text; 
import org.apache.hadoop.mapreduce.Reducer; 

public class SalesReducer extends Reducer<Text, FloatWritable, Text, FloatWritable> 
{ 
    public void reduce(Text k, Iterable<FloatWritable>vlist, Context con) throws IOException, InterruptedException 
    { 
     int tot=0; 
     for (FloatWritable v:vlist) 
     { 
      tot += v.get(); 
     } 
     //int total= (int)tot; 
     con.write(new Text(k), new FloatWritable(tot)); 
    } 
} 

结果MapReduce

Result of the MapReduce

我不明白为什么所有的结果都出现在一个大的浮点数和全部数字5.7480884E7。

下面是输入到映射精简程序的例子:

2012-01-01 09:00 San Jose Men's Clothing 214.05 Amex    
    2012-01-01 09:00 Fort Worth Women's Clothing 153.57 Visa   
    2012-01-01 09:00 San Diego Music 66.08 Cash      
    2012-01-01 09:00 Pittsburgh Pet Supplies 493.51 Discover   
    2012-01-01 09:00 Omaha Children's Clothing 235.63 MasterCard  
    2012-01-01 09:00 Stockton Men's Clothing 247.18 MasterCard   
    2012-01-01 09:00 Austin Cameras 379.6 Visa       
    2012-01-01 09:00 New York Consumer Electronics 296.8 Cash   
    2012-01-01 09:00 Corpus Christi Toys 25.38 Discover    
    2012-01-01 09:00 Fort Worth Toys 213.88 Visa      
    2012-01-01 09:00 Las Vegas Video Games 53.26 Visa     
    2012-01-01 09:00 Newark Video Games 39.75 Cash      
    2012-01-01 09:00 Austin Cameras 469.63 MasterCard     
    2012-01-01 09:00 Greensboro DVDs 290.82 MasterCard     
    2012-01-01 09:00 San Francisco Music 260.65 Discover    
    2012-01-01 09:00 Lincoln Garden 136.9 Visa       
    2012-01-01 09:00 Buffalo Women's Clothing 483.82 Visa    
    2012-01-01 09:00 San Jose Women's Clothing 215.82 Cash    
    2012-01-01 09:00 Boston Cameras 418.94 Amex      
    2012-01-01 09:00 Houston Baby 309.16 Visa       
    2012-01-01 09:00 Las Vegas Books 93.39 Visa      
    2012-01-01 09:00 Virginia Beach Children's Clothing 376.11 Amex 
    2012-01-01 09:01 Riverside Consumer Electronics 252.88 Cash  
    2012-01-01 09:01 Tulsa Baby 205.06 Visa       
    2012-01-01 09:01 Reno Crafts 88.25 Visa       
    2012-01-01 09:01 Chicago Books 31.08 Cash       
    2012-01-01 09:01 Fort Wayne Men's Clothing 370.55 Amex    
    2012-01-01 09:01 San Bernardino Consumer Electronics 170.2 Cash 
    2012-01-01 09:01 Madison Men's Clothing 16.78 Visa     
    2012-01-01 09:01 Austin Sporting Goods 327.75 Discover    
    2012-01-01 09:01 Portland CDs 108.69 Amex       
    2012-01-01 09:01 Riverside Sporting Goods 15.41 Discover   
    2012-01-01 09:01 Reno Toys 80.46 Visa        
    2012-01-01 09:01 Anchorage Music 298.86 MasterCard  
+0

这个输出映射缩减programBaby \t 5.7480884E7 书籍\t 5.743978E7 光盘\t 5.7400252E7 相机\t 5.728862E7 儿童服装\t 5.7612936E7 计算机\t 5.7303832E7 消费电子\t 5.744192E7 工艺品\t 5.7407532E7 DVDs \t 5.763812E7 Garden \t 5.7528848E7 健康与美容\t 5.7469112E7 男士服装\t 5.7609916E7 音乐\t 5.7484752E7 宠物用品\t 5.7186328E7 体育用品\t 5.7587608E7 玩具\t 5.7452464E7 视频游戏\t 5.750184E7 女装\t 5。7423576E7 – habeebsiddique

回答

0

变化减速器的输出值类型设置为文本,并转换浮动到预期的格式字符串。

String.format("%f",tot) 

按照以下职位详细信息格式化数字:
with scientific
without scientific notation

减速机:

public class SalesReducer extends Reducer<Text, FloatWritable, Text, Text> 
{ 
    public void reduce(Text k, Iterable<FloatWritable>vlist, Context con) throws IOException, InterruptedException 
    { 
     float tot=0; 
     for (FloatWritable v:vlist) 
     { 
      tot += v.get(); 
     } 
     //int total= (int)tot; 
     con.write(new Text(k), new Text(String.format("%f",tot))); 
    } 
} 
+0

我遵循上面给出的转换为字符串。但我得到的错误是“java.util.IllegalFormatConversionException:f!= java.lang.Integer”。我用“%s”替换了“%f”。该程序运行没有错误,但输出是57400252,而不是5.7400252E7,即没有小数位 – habeebsiddique

+0

我发布它给你提供线索有关的问题。你可以做的是改变tot的数据类型为float。使用DecimalFormat api来实现所需的格式。 –

+0

我也改变了tot类型来浮动,但它给出了相同的结果。 – habeebsiddique

0

您存储int变量花车和的值。
现在第一件事是int不能在小数点后精确处理浮点值。其次,如果行数非常高,则总和值可能会超出可接受的范围 int

请尝试将int变量从int更改为 float double

double tot=0; 
+0

我改变了类型浮动。它给出了和以前一样的结果。 – habeebsiddique

+0

您的输入数据集包含您试图在空间上分离的可变长度记录。 ** 2012-01-01 09:00圣何塞男士服装214.05美国运通** 这有在W [6]位置的数值,而 ** 2012-01-01 09:01雷诺玩具80.46 Visa ** 这已经在w [4]了。如果制图人员正在计算字符串的正确位置,我不确定。 –

+0

这些字段由制表符间隔分开,我在映射器中使用制表符间隔符。该映射器正在给出正确的输出。 – habeebsiddique