VertexRDD给我类型不匹配错误

Question

我正在运行下面的代码，试图在Apache Spark中的GraphX中创建图。

import org.apache.spark.SparkConf 

import org.apache.spark.SparkContext 

import org.apache.spark.graphx.GraphLoader 

import org.apache.spark.graphx.Graph 

import org.apache.spark.rdd.RDD 
import org.apache.spark.graphx.VertexId 

//loads file from the array 

val lines = sc.textFile("hdfs://moonshot-ha-nameservice/data/google-plus/2309.graph"); 

//maps lines and takes the first 21 characters of each line which is the node. 

val result = lines.map(line => line.substring(0,20)) 

//creates a new variable with each node followed by a long . 

val result2 = result.map(word => (word,1L).toLong) 

//where i am getting an error 

val vertexRDD: RDD[(Long,Long)] = sc.parallelize(result2) 

我收到以下错误：

error: type mismatch; 

found : org.apache.spark.rdd.RDD[(Long, Long)] 

required: Seq[?] 

Error occurred in an application involving default arguments. 
     val vertexRDD: RDD[(Long, Long)] = sc.parallelize(result2) 

Answer 1

首先，您的地图可以简化为以下代码：现在

val vertexRDD: RDD[(Long, Long)] = 
    lines.map(line => (line.substring(0, 17).toLong, 1L)) 

，你的错误：你不能用RDD拨打sc.parallelize。您的vertexRDD已经由result2定义。然后，您可以创建RESULT2您的图形和你EdgesRDD：

val g = Graph(result2, edgesRDD) 

，或者，如果采用我的建议：

val g = Graph(vertexRDD, edgesRDD)