使用插入符号预测多个模型的统计信息

Question

我正在尝试使用包caret获取各种培训模型的预测统计信息。下面是说明我需要一个例子：

library(caret) 

# Training: 
# ... Get X and Y for training a binary classification problem. 
# ... X is input (2000, 5) Y is output (2000,1) ... 

tmp <- createDataPartition(Y, p = 3/4, times = 3, list = TRUE, groups = min(5, length(Y))) 

myCtrl <- trainControl(method = "boot", index = tmp, timingSamps = 2, classProbs = TRUE, summaryFunction = twoClassSummary) 

RFmodel <- train(X,Y,method='rf',trControl=myCtrl,tuneLength=1, metric="ROC") 
SVMmodel <- train(X,Y,method='svmRadial',trControl=myCtrl,tuneLength=3, metric="ROC") 
KNNmodel <- train(X,Y,method='knn',trControl=myCtrl,tuneLength=10, metric="ROC") 
NNmodel <- train(X,Y,method='nnet',trControl=myCtrl,tuneLength=3, trace = FALSE, metric="ROC") 

# resamps reports ROC, Sens, Spec for all models 
resamps <- resamples(list(RF = RFmodel, KNN = KNNmodel, NN = NNmodel, SVM = SVMmodel)) 

# Prediction: 
# ... Collect X_pred (7000, 5) and Y_pred (7000,1) ... 
testPred <- predict(list(RF = RFmodel, KNN = KNNmodel, NN = NNmodel, SVM = SVMmodel), Xtst, type="prob") 

我怎样才能得到的预测（ROC等）从X_kand Y_ pred我的4款车型的统计数据？

Answer 1

#Make a list of all the models 
all.models <- list(model1, model2, model3, model4, model5, model6) 
names(all.models) <- sapply(all.models, function(x) x$method) 
sort(sapply(all.models, function(x) min(x$results$RMSE))) 

上面的代码不是我的，如果我没记错的话。

# Table 

# CORRELATIONS 
correlations = c(
cor(predict(model1,newdata=TD),Y), 
cor(predict(model2,newdata=TD),Y), 
cor(predict(model3,newdata=TD),Y), 
cor(predict(model4,newdata=TD),Y), 
cor(predict(model5,newdata=TD),Y), 

RMSE = as.numeric(sapply(all.models, function(x) min(x$results$RMSE))) 

names=c('General Linear Model','Random Forests','Artificial Neural Networks','Logistic/multinomial regression','K nearest neighbors', 'Support Vector Machines') 

matrix(c(names,correlations,RMSE),ncol=3) 

希望这会有所帮助。我知道这不是ROC，但是这是一些预测的统计数据。