NotFittedError：TfidfVectorizer - 词汇未安装

我正在尝试使用scikit-learn/pandas构建一个情感分析器。建立和评估模型的工作，但试图分类新的示例文本不。NotFittedError：TfidfVectorizer - 词汇未安装

我的代码：

import csv 
import pandas as pd 
import numpy as np 
from sklearn.model_selection import train_test_split 
from sklearn.feature_extraction.text import TfidfVectorizer 
from sklearn.naive_bayes import BernoulliNB 
from sklearn.metrics import classification_report 
from sklearn.metrics import accuracy_score 

infile = 'Sentiment_Analysis_Dataset.csv' 
data = "SentimentText" 
labels = "Sentiment" 


class Classifier(): 
    def __init__(self): 
     self.train_set, self.test_set = self.load_data() 
     self.counts, self.test_counts = self.vectorize() 
     self.classifier = self.train_model() 

    def load_data(self): 

     df = pd.read_csv(infile, header=0, error_bad_lines=False) 
     train_set, test_set = train_test_split(df, test_size=.3) 
     return train_set, test_set 

    def train_model(self): 
     classifier = BernoulliNB() 
     targets = self.train_set[labels] 
     classifier.fit(self.counts, targets) 
     return classifier 


    def vectorize(self): 

     vectorizer = TfidfVectorizer(min_df=5, 
           max_df = 0.8, 
           sublinear_tf=True, 
           ngram_range = (1,2), 
           use_idf=True) 
     counts = vectorizer.fit_transform(self.train_set[data]) 
     test_counts = vectorizer.transform(self.test_set[data]) 

     return counts, test_counts 

    def evaluate(self): 
     test_counts,test_set = self.test_counts, self.test_set 
     predictions = self.classifier.predict(test_counts) 
     print (classification_report(test_set[labels], predictions)) 
     print ("The accuracy score is {:.2%}".format(accuracy_score(test_set[labels], predictions))) 


    def classify(self, input): 
     input_text = input 

     input_vectorizer = TfidfVectorizer(min_df=5, 
           max_df = 0.8, 
           sublinear_tf=True, 
           ngram_range = (1,2), 
           use_idf=True) 
     input_counts = input_vectorizer.transform(input_text) 
     predictions = self.classifier.predict(input_counts) 
     print(predictions) 

myModel = Classifier() 

text = ['I like this I feel good about it', 'give me 5 dollars'] 

myModel.classify(text) 
myModel.evaluate()

错误：

Traceback (most recent call last): 
    File "sentiment.py", line 74, in <module> 
    myModel.classify(text) 
    File "sentiment.py", line 66, in classify 
    input_counts = input_vectorizer.transform(input_text) 
    File "/home/rachel/Sentiment/ENV/lib/python3.5/site-packages/sklearn/feature_extraction/text.py", line 1380, in transform 
    X = super(TfidfVectorizer, self).transform(raw_documents) 
    File "/home/rachel/Sentiment/ENV/lib/python3.5/site-packages/sklearn/feature_extraction/text.py", line 890, in transform 
    self._check_vocabulary() 
    File "/home/rachel/Sentiment/ENV/lib/python3.5/site-packages/sklearn/feature_extraction/text.py", line 278, in _check_vocabulary 
    check_is_fitted(self, 'vocabulary_', msg=msg), 
    File "/home/rachel/Sentiment/ENV/lib/python3.5/site-packages/sklearn/utils/validation.py", line 690, in check_is_fitted 
    raise _NotFittedError(msg % {'name': type(estimator).__name__}) 
sklearn.exceptions.NotFittedError: TfidfVectorizer - Vocabulary wasn't fitted.

我不知道这个问题可能是什么。在我的分类方法中，我创建了一个全新的矢量化器来处理要分类的文本，与用于从模型创建训练和测试数据的矢量化器分离。

感谢

来源

2017-05-26 killer_manatee

不管怎样，在你的'classify'功能，创建一个新的矢量化的对象，然后调用'transform'以往任何时候都装了。 –

添加到@AryaMcCarthy的答案，这个类中的整个分类功能是误导性的。构造函数允许传入输入数据。那么为什么在分类时再次通过它？ –

你装一个矢量器，但你把它扔掉，因为它不存在过去的vectorize功能的寿命。相反，保存您的模型vectorize它被改造后：

self._vectorizer = vectorizer

在 classify功能

然后，不创建一个新的矢量器。相反，使用你会装到训练数据的一个：

input_counts = self._vectorizer.transform(input_text)

来源

2017-05-26 05:09:14

NotFittedError：TfidfVectorizer - 词汇未安装

回答

相关问题