猪替换命令

Question

我有一个文件，它有一个像

11/16/2015,"others (phone,health,etc.)",cont'd attempts,"resource,inc.",dg 

我想删除逗号目前只有内部双引号的数据。

预期结果

11/162015,"others(phone health etc.)",cont'd attempts,"resource inc.",dg 

到目前为止我试过

Foreach a generate replace ($1,',',''); 
Foreach a generate regex_extract($1,'[\,]+',1); 

但他们没有工作。

Answer 1

所有的弗里斯特使用REGULAR EXP分隔字段的元组，然后应用更换

试试这个代码：

a = load '<path>' as line; 

b = foreach a generate FLATTEN(REGEX_EXTRACT_ALL(line,'(.*)[,]["](.*)["][,](.*)[,]["](.*)["][,](.*)')); 

c = foreach b generate $0,REPLACE($1,',',' '),$2,REPLACE($3,',',' '),$4; 
dump c; 

Answer 2

可以使用UDF是可以实现的，它可以看看在每个元组中的所有字段都通过。

import java.util.HashMap; 
import java.util.Iterator; 
import java.util.ArrayList; 
import java.io.IOException; 
import java.lang.Long; 
import java.lang.Exception; 

import org.apache.pig.EvalFunc; 
import org.apache.pig.data.Tuple; 
import org.apache.pig.data.DataBag; 
import org.apache.pig.data.BagFactory; 
import org.apache.pig.data.DataType; 
import org.apache.pig.impl.logicalLayer.schema.Schema; 
import org.apache.pig.impl.logicalLayer.FrontendException; 
import org.apache.pig.backend.executionengine.ExecException; 

public class CommaRemove extends EvalFunc<DataBag> { 

    @Override 
    public DataBag exec(Tuple input) throws IOException { 
     if (input == null || input.size() == 0) { 
      return null; 
     } 

     try { 
      int inputSize = input.size(); 
      Tuple output = TupleFactory.getInstance().newTuple(inputSize); 
      for(int i = 0; i < inputSize ; i++) 
      { 
       output.set(i, input.get(i).replace(',','')); 
      } 
      return output; 
     } catch (Exception e) { 
      System.err.println("Failed to process input; error - " + e.getMessage()); 
      return null; 
     } 
    } 
}