决策树星火负荷数据 - LabelledPoint

更改标签我尝试做在https://spark.apache.org/docs/latest/mllib-decision-tree.html 决策树星火负荷数据 - LabelledPoint

在火花决策树的例子中，我从http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html#a1a

数据集是在LIBSVM格式下载A1A数据集，其中两个班有标签+1.0 -1.0 当我尝试

import org.apache.spark.mllib.tree.DecisionTree 
import org.apache.spark.mllib.tree.model.DecisionTreeModel 
import org.apache.spark.mllib.util.MLUtils 

// Load and parse the data file. 
val data = MLUtils.loadLibSVMFile(sc, "/user/cloudera/testDT/a1a.t") 
// Split the data into training and test sets (30% held out for testing) 
val splits = data.randomSplit(Array(0.7, 0.3)) 
val (trainingData, testData) = (splits(0), splits(1)) 

// Train a DecisionTree model. 
// Empty categoricalFeaturesInfo indicates all features are continuous. 
val numClasses = 2 
val categoricalFeaturesInfo = Map[Int, Int]() 
val impurity = "gini" 
val maxDepth = 5 
val maxBins = 32 

val model = DecisionTree.trainClassifier(trainingData, numClasses, categoricalFeaturesInfo, 
| impurity, maxDepth, maxBins)

我得到：

java.lang.IllegalArgumentException: GiniAggregator given label -1.0 but requires label is non-negative.

所以我试图将标签-1.0更改为0.0。我想是这样

def changeLabel(a: org.apache.spark.mllib.regression.LabeledPoint) = 
{ if (a.label == -1.0) {a.label = 0.0} }

我在哪里得到的错误：

reassignment to val

所以我的问题是这样的：我怎样才能改变我的数据的标签？或者是否有解决方法，使DecisionTree.trainClassifier（）能够处理带有负面标签的数据？

来源

2017-09-01 Michail N

TL; DR你不能辞职，一个Product类的值参数，即使是可能的（声明为var），你永远不会应在星火修改的地方的数据。

如何：

def changeLabel(a: org.apache.spark.mllib.regression.LabeledPoint) = 
    if (a.label == -1.0) a.copy(label = 0.0) else a

scala> changeLabel(LabeledPoint(-1.0, Vectors.dense(1.0, 2.0, 3.0))) 
res1: org.apache.spark.mllib.regression.LabeledPoint = (0.0,[1.0,2.0,3.0]) 

scala> changeLabel(LabeledPoint(1.0, Vectors.dense(1.0, 2.0, 3.0))) 
res2: org.apache.spark.mllib.regression.LabeledPoint = (1.0,[1.0,2.0,3.0])

来源

2017-09-01 11:19:26 user8371915

决策树星火负荷数据 - LabelledPoint

回答

相关问题