使用Redis运行Storm时出错：java.lang.NoClassDefFoundError：Lredis/clients/jedis/Jedis

Question

My Storm类使用Redis队列来收集数据。

我试图通过

storm jar jar_file_name.jar Topology_name configuration_file 

运行我的风暴罐子但我得到以下异常：

Exception in thread "main" java.lang.NoClassDefFoundError: Lredis/clients/jedis/Jedis; 
    at java.lang.Class.getDeclaredFields0(Native Method) 
    at java.lang.Class.privateGetDeclaredFields(Class.java:2397) 
    at java.lang.Class.getDeclaredField(Class.java:1946) 
    at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1659) 
    at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:72) 
    at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:480) 
    at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:468) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:468) 
    at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:365) 
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1133) 
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) 
    at backtype.storm.utils.Utils.serialize(Utils.java:77) 
    at backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:111) 
    at OutlierPredictor.main(OutlierPredictor.java:98) 
Caused by: java.lang.ClassNotFoundException: redis.clients.jedis.Jedis 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 

我一直在使用

javac -classpath $HADOOP_CORE:$HBASE_CLASSPATH:/usr/local/hadoop- 2.2.0/redis_jar/commons-pool-1.5.5.jar:/usr/local/hadoop-2.2.0/redis_jar/jedis-2.1.0.jar:/usr/local/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar -d dir_name/ dir_name/Javafile.java 

命令行编译。

我在单个节点设置中执行此操作。

到底哪里出问题了？

Answer 1

您需要将所有代码和依赖关系打包到一个jar中。

参考Storm tutorial，拓扑结构部分，

Running a topology is straightforward. First, you package all your code and dependencies into a single jar. Then, you run a command like the following:

storm jar all-my-code.jar backtype.storm.MyTopology arg1 arg2

您可能需要使用一些包装工具一样，OneJAR，JarJar或ANT和创建一个包含所有文件和相关性的罐子。请参阅这些因此发布

1. Merging Multiple Jars in to a Single Jar
2. Easiest way to merge a release into one JAR file

Answer 2

谢谢约翰的回答。

但我找到了另一种解决方案。

我没有包括jedis-2.1.0.jar，在我的$ STORM_HOME/lib目录/ commons-pool的-1.5.5.jar

注：这两个文件都在那里的$ REDIS_CORE。

Answer 3

1）在依赖性

<dependency> 
    <groupId>redis.clients</groupId> 
    <artifactId>jedis</artifactId> 
    <version>2.9.0</version> 
</dependency> 

2）使用装配插件的所有依赖打包成一个jar文件

<plugin> 
     <groupId>org.apache.maven.plugins</groupId> 
     <artifactId>maven-jar-plugin</artifactId> 
     <version>2.6</version> 
     <configuration> 
      <manifest> 
       <addClasspath>true</addClasspath> 
       <mainClass>your main class</mainClass> 
      </manifest> 
     </configuration> 
    </plugin> 
    <plugin> 
     <groupId>org.apache.maven.plugins</groupId> 
     <artifactId>maven-assembly-plugin</artifactId> 
     <configuration> 
      <archive> 
       <manifest> 
        <mainClass>your main class</mainClass> 
       </manifest> 
      </archive> 
      <descriptorRefs> 
       <descriptorRef>jar-with-dependencies</descriptorRef> 
      </descriptorRefs> 
     </configuration> 
     <executions> 
      <execution> 
       <id>make-assembly</id> 
       <phase>package</phase> 
       <goals> 
        <goal>single</goal> 
       </goals> 
      </execution> 
     </executions> 
    </plugin> 

3）忽略包风暴核添加jedis

<dependency> 
    <groupId>org.apache.storm</groupId> 
    <artifactId>storm-core</artifactId> 
    <version>1.1.1</version> 
    <scope>provided</scope> 
</dependency>