我正在学习如何从/向hdfs读取/写入文件。无法从hadoop检索文件hdfs
这是我用读码:
import java.io.InputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
public class FileSystemCat {
public static void main (String [] args) throws Exception {
String uri = "/user/hadoop/file.txt";
Configuration conf = new Configuration();
conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml"));
conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml"));
FileSystem fs = FileSystem.get(URI.create(uri),conf);
InputStream in = null;
try{
in = fs.open(new Path(uri));
IOUtils.copyBytes(in, System.out, 4096,false);
}finally{
IOUtils.closeStream(in);
}
}
}
该文件是存在的
不过,我得到以下运行我的代码eclipse
Exception in thread "main" java.io.FileNotFoundException: File /user/hadoop/file.txt does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:511)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:724)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:501)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:137)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
at hadoop.FileSystemCat.main(FileSystemCat.java:22)
我用作路径二者文件:///user/hadoop/file.txt和HDFS:///user/hadoop/file.txt
对于后者的误差是稍有不同:
Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
芯-site.xml中
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost/</value>
</property>
</configuration>
HDFS-site.xml中
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///usr/local/hadoop_store/hdfs/namenode/</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///usr/local/hadoop_store/hdfs/datanode/,file:///mnt/hadoop/hadoop_store/hdfs/datanode/</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>
有什么顾虑?
感谢
我编辑您的提示后做
hadoop fs -ls /
。它仍然不起作用。请看看hdfs-site.xls和core-site.xml – QGA你如何运行应用程序?使用'hadoop jar'? – Balduz
我在构建路径中添加了hadoop文件夹和子文件夹中包含的所有jar文件(我知道不好的做法:)) – QGA