回答
必须动态地生成基于剩余字符串的长度零的数目,所以我不认为它可能在本地猪。
这在UDF中是非常有可能的。
input.txt中
11111
222222222
33
org.apache.hadoop.util.NativeCodeLoader
apachepig
PigScript:
REGISTER leftformat.jar;
A = LOAD 'input.txt' USING PigStorage() AS(f1:chararray);
B = FOREACH A GENERATE format.LEFTPAD(f1);
DUMP B;
输出:
(0000000000000000000000000000000000011111)
(0000000000000000000000000000000222222222)
(0000000000000000000000000000000000000033)
(0org.apache.hadoop.util.NativeCodeLoader)
(0000000000000000000000000000000apachepig)
UDF代码:的下面Java类文件被编译并作为产生leftformat.jar
LEFTPAD.java
package format;
import java.io.IOException;
import org.apache.commons.lang.StringUtils;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
public class LEFTPAD extends EvalFunc<String> {
@Override
public String exec(Tuple arg) throws IOException {
try
{
String input = (String)arg.get(0);
return StringUtils.leftPad(input, 40, "0");
}
catch(Exception e)
{
throw new IOException("Caught exception while processing the input row ", e);
}
}
}
UPDATE:
1.Download 4 jar files from the below link(apache-commons-lang.jar,piggybank.jar, pig-0.11.0.jar and hadoop-common-2.6.0-cdh5.4.5)
http://www.java2s.com/Code/Jar/a/Downloadapachecommonslangjar.htm
http://www.java2s.com/Code/Jar/p/Downloadpiggybankjar.htm
http://www.java2s.com/Code/Jar/p/Downloadpig0110jar.htm
2. Set all the 3 jar files to your class path
>> export CLASSPATH=/tmp/pig-0.11.1.jar:/tmp/piggybank.jar:/tmp/apache-commons-lang.jar
3. Create directory name format
>>mkdir format
4. Compile your LEFTPAD.java and make sure all the three jars are included in the class path otherwise compilation issue will come
>>javac LEFTPAD.java
5. Move the class file to format folder
>>mv LEFTPAD.class format
6. Create jar file name leftformat.jar
>>jar -cf leftformat.jar format/
7. jar file will be created, include into your pig script
Example from command line:
$ mkdir format
$ javac LEFTPAD.java
$ mv LEFTPAD.class format/
$ jar -cf leftformat.jar format/
$ ls
LEFTPAD.java format input.txt leftformat.jar script.pig
感谢您的回答。我在创建jar文件时遇到了问题。我已经创建了LEFTPAD.class(上面),并创建了带有followong的创建和扩展txt:Main-Class:LEFTPAD,然后运行以下命令:jar cfm leftformat.jar extension.txt LEFTPAD.class。 Jar创建成功,但从猪脚本中调用时,我收到错误消息 – clairvoyant 2014-11-17 10:05:37
错误消息:<行113,列24>无法生成逻辑计划。嵌套异常:org.apache.pig.backend.executionengine.ExecException:错误1070:无法使用imports解析format.LEFTPAD:[,java.lang。,org.apache.pig.builtin。,org.apache.pig.impl .builtin。] – clairvoyant 2014-11-17 10:09:54
尝试像这样,创建一个文件夹名称“format”并编译你的java文件LEFTPAD.java。 LEFTPAD.class应该存在于“格式”文件夹中。然后像这样创建jar文件“jar -cf leftformat.jar format /”。之后,将这个罐子包含在你的猪脚本中。如果您遇到一些问题,请让我知道。 – 2014-11-17 10:43:59
也许使用正则表达式? – clairvoyant 2014-11-14 15:15:04
你能贴一些例子吗? – 2014-11-14 15:50:03