Load pickle NotFittedError: CountVectorizer - Vocabulary wasn't fitted
Load pickle NotFittedError: CountVectorizer - Vocabulary wasn't fitted
我正在尝试使用 scikit 机器对垃圾邮件进行分类 learning.once 我将矢量化器和分类器都转储到 respective.pkl 文件中并在 temp.py 中导入 tem 以进行预测我收到此错误:
raise NotFittedError(msg % {'name': type(estimator).__name__})
NotFittedError: CountVectorizer - Vocabulary wasn't fitted
一旦我构建了一个模型,就用名称(my_model.pkl)保存了模型,(vectorizer.pkl)并重新启动了我的内核,但是当我加载保存的模型时(sample.pkl)在对示例文本进行预测期间,它给出了相同的 Volcubary not found 错误。
app.py:
import pandas as pd
df = pd.read_csv('spam.csv', encoding="latin-1")
#Drop the columns not needed
df.drop(['Unnamed: 2', 'Unnamed: 3', 'Unnamed: 4'], axis=1, inplace=True)
#Create a new column label which has the same values as v1 then set the ham and spam values to 0 and 1 which is the standard format for our prediction
df['label'] = df['v1'].map({'ham': 0, 'spam': 1})
#Create a new column having the same values as v2 column
df['message'] = df['v2']
#Now drop the v1 and v2
df.drop(['v1', 'v2'], axis=1, inplace=True)
#print(df.head(10))
from sklearn.feature_extraction.text import CountVectorizer
bow_transformer = CountVectorizer().fit_transform(df['message'])
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import classification_report
#Split the data
X_train, X_test, y_train, y_test = train_test_split(bow_transformer, df['label'], test_size=0.33, random_state=42)
#Naive Bayes Classifier
clf = MultinomialNB()
clf.fit(X_train,y_train)
clf.score(X_test,y_test)
y_pred = clf.predict(X_test)
print(classification_report(y_test, y_pred))
pickle.dump(bow_transformer, open("vector.pkl", "wb"))
pickle.dump(clf, open("my_model.pkl", "wb"))
temp.py:::我正在这个文件中做预测
from sklearn.feature_extraction.text import CountVectorizer
cv=CountVectorizer()
vectorizer = pickle.load(open("my_model.pkl", "rb"))
selector = pickle.load(open("vector.pkl", "rb"))
test_set=["heloo how are u"]
new_test=cv.transform(test_set)
在您的 app.py
中,您正在腌制 document-term matrix 而不是向量化器
pickle.dump(bow_transformer, open("vector.pkl", "wb"))
其中 bow_transformer 是
bow_transformer = CountVectorizer().fit_transform(df['message'])
在你的 temp.py
中,当你解开它时,你只有文档术语 matrix.The 正确的方法来腌制它是:
bow_transformer = CountVectorizer().fit(df['message'])
bow_transformer_dtm = bow_transformer.transform(df['message'])
现在您可以使用
腌制您的 bow_transformer
pickle.dump(bow_transformer, open("vector.pkl", "wb"))
这将是一个转换器而不是文档术语矩阵。
并且在您的 temp.py
中,您可以解开它并如下图所示使用它:
selector = pickle.load(open("vector.pkl", "rb"))
test_set=["heloo how are u"]
new_test=selector.transform(test_set)
希望对您有所帮助!
我正在尝试使用 scikit 机器对垃圾邮件进行分类 learning.once 我将矢量化器和分类器都转储到 respective.pkl 文件中并在 temp.py 中导入 tem 以进行预测我收到此错误:
raise NotFittedError(msg % {'name': type(estimator).__name__})
NotFittedError: CountVectorizer - Vocabulary wasn't fitted
一旦我构建了一个模型,就用名称(my_model.pkl)保存了模型,(vectorizer.pkl)并重新启动了我的内核,但是当我加载保存的模型时(sample.pkl)在对示例文本进行预测期间,它给出了相同的 Volcubary not found 错误。
app.py:
import pandas as pd
df = pd.read_csv('spam.csv', encoding="latin-1")
#Drop the columns not needed
df.drop(['Unnamed: 2', 'Unnamed: 3', 'Unnamed: 4'], axis=1, inplace=True)
#Create a new column label which has the same values as v1 then set the ham and spam values to 0 and 1 which is the standard format for our prediction
df['label'] = df['v1'].map({'ham': 0, 'spam': 1})
#Create a new column having the same values as v2 column
df['message'] = df['v2']
#Now drop the v1 and v2
df.drop(['v1', 'v2'], axis=1, inplace=True)
#print(df.head(10))
from sklearn.feature_extraction.text import CountVectorizer
bow_transformer = CountVectorizer().fit_transform(df['message'])
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import classification_report
#Split the data
X_train, X_test, y_train, y_test = train_test_split(bow_transformer, df['label'], test_size=0.33, random_state=42)
#Naive Bayes Classifier
clf = MultinomialNB()
clf.fit(X_train,y_train)
clf.score(X_test,y_test)
y_pred = clf.predict(X_test)
print(classification_report(y_test, y_pred))
pickle.dump(bow_transformer, open("vector.pkl", "wb"))
pickle.dump(clf, open("my_model.pkl", "wb"))
temp.py:::我正在这个文件中做预测
from sklearn.feature_extraction.text import CountVectorizer
cv=CountVectorizer()
vectorizer = pickle.load(open("my_model.pkl", "rb"))
selector = pickle.load(open("vector.pkl", "rb"))
test_set=["heloo how are u"]
new_test=cv.transform(test_set)
在您的 app.py
中,您正在腌制 document-term matrix 而不是向量化器
pickle.dump(bow_transformer, open("vector.pkl", "wb"))
其中 bow_transformer 是
bow_transformer = CountVectorizer().fit_transform(df['message'])
在你的 temp.py
中,当你解开它时,你只有文档术语 matrix.The 正确的方法来腌制它是:
bow_transformer = CountVectorizer().fit(df['message'])
bow_transformer_dtm = bow_transformer.transform(df['message'])
现在您可以使用
腌制您的bow_transformer
pickle.dump(bow_transformer, open("vector.pkl", "wb"))
这将是一个转换器而不是文档术语矩阵。
并且在您的 temp.py
中,您可以解开它并如下图所示使用它:
selector = pickle.load(open("vector.pkl", "rb"))
test_set=["heloo how are u"]
new_test=selector.transform(test_set)
希望对您有所帮助!