Spacy - ValueError: Can't read file: models/model-best/accuracy.json
Spacy - ValueError: Can't read file: models/model-best/accuracy.json
我想在瑞典 UD Treebank 上训练一个 Spacy 模型。
为此，我按照 spacy 页面上的说明进行操作：
https://spacy.io/usage/training#spacy-train-cli
训练本身 运行s，很好，但最后它试图打开一个由于某种原因不存在的文件。至少不在这个位置。
USER@Ubuntu18:~/spacy_models/sv$ python -m spacy train sv models talbanken-json/sv_talbanken-ud-train.json talbanken-json/sv_talbanken-ud-dev.json
⚠ Output directory is not empty
This can lead to unintended side effects when saving the model. Please use an
empty directory or a different path instead. If the specified output path
doesn't exist, the directory will be created for you.
Training pipeline: ['tagger', 'parser', 'ner']
Starting with blank model 'sv'
Counting training words (limit=0)
Itn Tag Loss Tag % Dep Loss UAS LAS NER Loss NER P NER R NER F Token % CPU WPS
--- --------- -------- --------- ------ ------ --------- ------ ------ ------ ------- -------
1 23722.240 87.792 74889.173 67.796 56.740 0.000 0.000 0.000 0.000 100.000 15740
....
30 780.531 93.508 13930.092 81.295 75.906 0.000 0.000 0.000 0.000 100.000 15569
✔ Saved model to output directory
models/model-final
Traceback (most recent call last):
File "/home/USER/miniconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/USER/miniconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/USER/miniconda3/lib/python3.7/site-packages/spacy/__main__.py", line 33, in <module>
plac.call(commands[command], sys.argv[1:])
File "/home/USER/miniconda3/lib/python3.7/site-packages/plac_core.py", line 328, in call
cmd, result = parser.consume(arglist)
File "/home/USER/miniconda3/lib/python3.7/site-packages/plac_core.py", line 207, in consume
return cmd, self.func(*(args + varargs + extraopts), **kwargs)
File "/home/USER/miniconda3/lib/python3.7/site-packages/spacy/cli/train.py", line 497, in train
best_model_path = _collate_best_model(meta, output_path, nlp.pipe_names)
File "/home/USER/miniconda3/lib/python3.7/site-packages/spacy/cli/train.py", line 559, in _collate_best_model
bests[component] = _find_best(output_path, component)
File "/home/USER/miniconda3/lib/python3.7/site-packages/spacy/cli/train.py", line 578, in _find_best
accs = srsly.read_json(epoch_model / "accuracy.json")
File "/home/USER/miniconda3/lib/python3.7/site-packages/srsly/_json_api.py", line 50, in read_json
file_path = force_path(location)
File "/home/USER/miniconda3/lib/python3.7/site-packages/srsly/util.py", line 21, in force_path
raise ValueError("Can't read file: {}".format(location))
ValueError: Can't read file: models/model-best/accuracy.json
相应的文件夹本身仅包含一个 meta.json 文件：
$ ls models/model-best/
meta.json ner parser tagger tokenizer vocab
幸运的是模型本身的训练并没有受到这个问题的影响。如果我可以 运行 它不会崩溃，那就太好了。
有什么办法可以解决这个问题吗？
注意脚本显示的警告并从空输出目录开始：
⚠ Output directory is not empty
This can lead to unintended side effects when saving the model. Please use an
empty directory or a different path instead. If the specified output path
doesn't exist, the directory will be created for you.
我注意到某些实体不存在于有效数据集中，但存在于训练数据集中。这就是我解决上述问题的方法。
我想在瑞典 UD Treebank 上训练一个 Spacy 模型。
为此，我按照 spacy 页面上的说明进行操作： https://spacy.io/usage/training#spacy-train-cli
训练本身 运行s，很好，但最后它试图打开一个由于某种原因不存在的文件。至少不在这个位置。
USER@Ubuntu18:~/spacy_models/sv$ python -m spacy train sv models talbanken-json/sv_talbanken-ud-train.json talbanken-json/sv_talbanken-ud-dev.json
⚠ Output directory is not empty
This can lead to unintended side effects when saving the model. Please use an
empty directory or a different path instead. If the specified output path
doesn't exist, the directory will be created for you.
Training pipeline: ['tagger', 'parser', 'ner']
Starting with blank model 'sv'
Counting training words (limit=0)
Itn Tag Loss Tag % Dep Loss UAS LAS NER Loss NER P NER R NER F Token % CPU WPS
--- --------- -------- --------- ------ ------ --------- ------ ------ ------ ------- -------
1 23722.240 87.792 74889.173 67.796 56.740 0.000 0.000 0.000 0.000 100.000 15740
....
30 780.531 93.508 13930.092 81.295 75.906 0.000 0.000 0.000 0.000 100.000 15569
✔ Saved model to output directory
models/model-final
Traceback (most recent call last):
File "/home/USER/miniconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/USER/miniconda3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/USER/miniconda3/lib/python3.7/site-packages/spacy/__main__.py", line 33, in <module>
plac.call(commands[command], sys.argv[1:])
File "/home/USER/miniconda3/lib/python3.7/site-packages/plac_core.py", line 328, in call
cmd, result = parser.consume(arglist)
File "/home/USER/miniconda3/lib/python3.7/site-packages/plac_core.py", line 207, in consume
return cmd, self.func(*(args + varargs + extraopts), **kwargs)
File "/home/USER/miniconda3/lib/python3.7/site-packages/spacy/cli/train.py", line 497, in train
best_model_path = _collate_best_model(meta, output_path, nlp.pipe_names)
File "/home/USER/miniconda3/lib/python3.7/site-packages/spacy/cli/train.py", line 559, in _collate_best_model
bests[component] = _find_best(output_path, component)
File "/home/USER/miniconda3/lib/python3.7/site-packages/spacy/cli/train.py", line 578, in _find_best
accs = srsly.read_json(epoch_model / "accuracy.json")
File "/home/USER/miniconda3/lib/python3.7/site-packages/srsly/_json_api.py", line 50, in read_json
file_path = force_path(location)
File "/home/USER/miniconda3/lib/python3.7/site-packages/srsly/util.py", line 21, in force_path
raise ValueError("Can't read file: {}".format(location))
ValueError: Can't read file: models/model-best/accuracy.json
相应的文件夹本身仅包含一个 meta.json 文件：
$ ls models/model-best/
meta.json ner parser tagger tokenizer vocab
幸运的是模型本身的训练并没有受到这个问题的影响。如果我可以 运行 它不会崩溃，那就太好了。
有什么办法可以解决这个问题吗？
注意脚本显示的警告并从空输出目录开始：
⚠ Output directory is not empty
This can lead to unintended side effects when saving the model. Please use an
empty directory or a different path instead. If the specified output path
doesn't exist, the directory will be created for you.
我注意到某些实体不存在于有效数据集中，但存在于训练数据集中。这就是我解决上述问题的方法。