如何评估使用示例 SequenceClassification 代码训练的 CNTK 模型

Question

我使用 https://github.com/Microsoft/CNTK/blob/v2.0/Examples/SequenceClassification/SimpleExample/Python/SequenceClassification.py 中给出的代码来训练模型。我如何评价它？

Answer 1

如果您想评估 Python 中的模型，请参阅第 here. If you want use your model in other languages, e.g. C++/C#, you can find details in the Model Evalaution 页。

谢谢，

Answer 2

我是通过以下方式获取的：

import cntk as C
from cntk.ops.functions import load_model # Note this
...
...
# saved the model after epochs
for i in range(500):
    mb = reader.next_minibatch(minibatch_size, input_map=input_map)
    trainer.train_minibatch(mb)
    classifier_output.save("model.dnn") # Note this
...
...
# loading the model
model = load_model("model.dnn") # Note this

# converted sentence to numbers and given as sequence
predScores = model(C.Value.one_hot([[1,238,4,4990,7223,1357,2]], 50466)) # Note this
predClass = np.argmax(predScores)
print(predClass)

其中 [[1,238,4,4990,7223,1357,2]] 是词汇表中单词索引的序列（基本上是训练发生的那种序列，50466 是词汇表的大小。

Answer 3

当您在 CNTK 中训练您的模型时，您不太可能不需要使用 create_reader/Minibatch 工具。主要是因为 test/production 文件通常很小。模型评估实际上非常简单：

import cntk as C
import pandas as pd
import numpy as np

model = C.load_model(path_to_where_the_model_is_saved) # load your CNTK model

ds = pd.read_csv(filename, delimiter=",") # load your data of course
                                          # we are assuming all data come 
                                          # together in a single matrix

X = ds.values[:,0:28].astype('float32') # ensures the right type for CNTK
Y = ds.values[:,28].astype('float32')   # last column is the label

X= X / 255 # perform any necessary transformation if any

pred = model(X) # evaluate your test data

pred[pred > 0.5]=1
pred[pred!=1]=0
maxa=np.mean(Y==pred)

print("Accuracy {} ".format(maxa*100.0))

如何评估使用示例 SequenceClassification 代码训练的 CNTK 模型

How do I evaluate a CNTK model trained using sample SequenceClassification code

python

cntk