spacy 2.2.3 FileNotFoundError: [Errno 2] No such file or directory: 'thinc\\neural\\_custom_kernels.cu' in pyinstaller
spacy 2.2.3 FileNotFoundError: [Errno 2] No such file or directory: 'thinc\\neural\\_custom_kernels.cu' in pyinstaller
我正在尝试使用 pyinstaller 创建一个可执行文件。我在执行问题时遇到以下问题。
File "test_env2_live\main.py", line 2, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "test_env2_live\controller\main.py", line 2, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "test_env2_live\controller\inflow\job_controller.py", line 175, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\spacy\__init__.py", line 10, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\thinc\neural\__init__.py", line 4, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\thinc\neural\_classes\model.py", line 11, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\thinc\neural\train.py", line 7, in <module>
File "optimizers.pyx", line 14, in init thinc.neural.optimizers
File "ops.pyx", line 24, in init thinc.neural.ops
ImportError: cannot import name _custom_kernels
所以我在构建可执行文件时添加了--hiddenimport thinc.neural._custom_kernels。下面是构建可执行文件的代码
pyinstaller main.py --hiddenimport preshed.maps --hiddenimport srsly.msgpack.util --hidden-import="sklearn.utils._cython_blas" --hidden-import="sklearn.neighbors.typedefs" --hidden-import="sklearn.neighbors.quad_tree" --hidden-import="sklearn.tree._utils" --hidden-import cymem.cymem --hidden-import thinc.linalg --hidden-import murmurhash.mrmr --hidden-import cytoolz.utils --hidden-import cytoolz._signatures --hidden-import spacy.strings --hidden-import spacy.morphology --hidden-import spacy.lexeme --hidden-import spacy.tokens --hidden-import spacy.gold --hidden-import spacy.tokens.underscore --hidden-import spacy.parts_of_speech --hidden-import dill --hidden-import spacy.tokens.printers --hidden-import spacy.tokens._retokenize --hidden-import spacy.syntax --hidden-import spacy.syntax.stateclass --hidden-import spacy.syntax.transition_system --hidden-import spacy.syntax.nonproj --hidden-import spacy.syntax.nn_parser --hidden-import spacy.syntax.arc_eager --hidden-import thinc.extra.search --hidden-import spacy.syntax._beam_utils --hidden-import spacy.syntax.ner --hidden-import thinc.neural._classes.difference --hidden-import spacy.vocab --hidden-import spacy.lemmatizer --hidden-import spacy._ml --hidden-import spacy.lang.en --hiddenimport thinc.neural._custom_kernels
但是现在可执行文件出现以下错误
File "site-packages\thinc\neural\train.py", line 7, in <module>
File "optimizers.pyx", line 14, in init thinc.neural.optimizers
File "ops.pyx", line 24, in init thinc.neural.ops
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\thinc\neural\_custom_kernels.py", line 36, in <module>
File "pathlib.py", line 1183, in open
File "pathlib.py", line 1037, in _opener
File "pathlib.py", line 387, in wrapped
FileNotFoundError: [Errno 2] No such file or directory: 'D:\rajesh\python\console\test_env2_live\dist\main\thinc\neural\_custom_kernels.cu'
不确定为什么会出现 FileNotFoundError 错误。谁能帮我？
配置：Python3.6.7，conda 4.8.0，spacy v2.2.3，Windows10
我可以成功运行 低版本spacy(2.0.18) 的可执行文件。但它在 2.2.3 版本中给出了错误。
提前致谢
FileNotFound 错误是因为 PyInstaller 没有正确打包 thinc； thinc 需要一个钩子。我发现包含 from spacy import * 的脚本将与下面的挂钩文件一起使用。我使用的命令是：
pyinstaller test-spacy.py --additional-hooks-dir=.
之所以有效，是因为我在与 test-spacy.py 脚本相同的目录中为 spacy（及其子模块）添加了一个挂钩。只需将以下文本复制到名为 hook-spacy.py 的文件中，该文件与您的脚本位于同一目录中。
# HOOK FILE FOR SPACY
from PyInstaller.utils.hooks import collect_all
# ----------------------------- SPACY -----------------------------
data = collect_all('spacy')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
# ----------------------------- THINC -----------------------------
data = collect_all('thinc')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- CYMEM -----------------------------
data = collect_all('cymem')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- PRESHED -----------------------------
data = collect_all('preshed')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- BLIS -----------------------------
data = collect_all('blis')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# This hook file is a bit of a hack - really, all of the libraries should be in seperate hook files. (Eg hook-blis.py with the blis part of the hook)
如果您需要更多帮助，请在评论中提问。
我调整了 .spec 文件并添加了具有以下详细信息的 main.spec 文件并且它有效。
# -*- mode: python ; coding: utf-8 -*-
import PyInstaller
datas = []
datas.extend(PyInstaller.utils.hooks.collect_data_files('spacy.lang', include_py_files = True))
datas.extend(PyInstaller.utils.hooks.collect_data_files('thinc'))
block_cipher = None
a = Analysis(['main.py'],
pathex=['D:\rajesh\python\console\live'],
binaries=[],
datas=datas,
hiddenimports = [
'spacy.kb',
'spacy.lexeme',
'spacy.matcher._schemas',
'spacy.morphology',
'spacy.parts_of_speech',
'spacy.syntax._beam_utils',
'spacy.syntax._parser_model',
'spacy.syntax.arc_eager',
'spacy.syntax.ner',
'spacy.syntax.nn_parser',
'spacy.syntax.stateclass',
'spacy.syntax.transition_system',
'spacy.tokens._retokenize',
'spacy.tokens.morphanalysis',
'spacy.tokens.underscore',
'spacy._align',
'blis',
'blis.py',
'cymem',
'cymem.cymem',
'murmurhash',
'murmurhash.mrmr',
'preshed.maps',
'srsly.msgpack.util',
'thinc.extra.search',
'thinc.linalg',
'thinc.neural._aligned_alloc',
'thinc.neural._custom_kernels',
'sklearn.utils._cython_blas',
'sklearn.neighbors.typedefs',
'sklearn.neighbors.quad_tree',
'sklearn.tree._utils'
],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
[],
exclude_binaries=True,
name='main',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True )
coll = COLLECT(exe,
a.binaries,/
a.zipfiles,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name='main')
我同意@Legorooj。添加到他的回答中，最后我不得不为 srsly.msgpack.util 添加另一个隐藏的导入。同样在我的脚本中，我使用了 "en_core_web_sm":
import en_core_web_sm
nlp = en_core_web_sm.load()
所以我也为“en_core_web_sm”添加了一个钩子文件。 hook-spacy.py 文件变为：
# HOOK FILE FOR SPACY
from PyInstaller.utils.hooks import collect_all
# ----------------------------- SPACY -----------------------------
data = collect_all('spacy')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
# ----------------------------- THINC -----------------------------
data = collect_all('thinc')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- CYMEM -----------------------------
data = collect_all('cymem')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- PRESHED -----------------------------
data = collect_all('preshed')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- BLIS -----------------------------
data = collect_all('blis')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- OTHER ----------------------------
hiddenimports += ['srsly.msgpack.util']
hook-en_core_web_sm 文件是：
from PyInstaller.utils.hooks import collect_data_files
datas = collect_data_files("en_core_web_sm")
# -*- mode: python ; coding: utf-8 -*-
block_cipher = None
import PyInstaller
# HOOK FILE FOR SPACY
from PyInstaller.utils.hooks import collect_all
from PyInstaller.utils.hooks import collect_data_files
# ----------------------------- SPACY -----------------------------
data = collect_all('spacy')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
# ----------------------------- THINC -----------------------------
data = collect_all('thinc')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- CYMEM -----------------------------
data = collect_all('cymem')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- PRESHED -----------------------------
data = collect_all('preshed')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- BLIS -----------------------------
data = collect_all('blis')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- OTHER ----------------------------
hiddenimports += ['srsly.msgpack.util']
# ----------------------------- Sacy Model -------------------------
datas += collect_data_files("en_core_web_sm")
a = Analysis(['ttest_spacy.py'],
pathex=['PATH/TO/PYFILE'],
binaries=binaries,
datas=datas,
hiddenimports = hiddenimports,
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
[],
exclude_binaries=True,
name='ttest_spacy',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True )
coll = COLLECT(exe,
a.binaries,
a.zipfiles,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name='ttest_spacy')
我正在尝试使用 pyinstaller 创建一个可执行文件。我在执行问题时遇到以下问题。
File "test_env2_live\main.py", line 2, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "test_env2_live\controller\main.py", line 2, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "test_env2_live\controller\inflow\job_controller.py", line 175, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\spacy\__init__.py", line 10, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\thinc\neural\__init__.py", line 4, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\thinc\neural\_classes\model.py", line 11, in <module>
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\thinc\neural\train.py", line 7, in <module>
File "optimizers.pyx", line 14, in init thinc.neural.optimizers
File "ops.pyx", line 24, in init thinc.neural.ops
ImportError: cannot import name _custom_kernels
所以我在构建可执行文件时添加了--hiddenimport thinc.neural._custom_kernels。下面是构建可执行文件的代码
pyinstaller main.py --hiddenimport preshed.maps --hiddenimport srsly.msgpack.util --hidden-import="sklearn.utils._cython_blas" --hidden-import="sklearn.neighbors.typedefs" --hidden-import="sklearn.neighbors.quad_tree" --hidden-import="sklearn.tree._utils" --hidden-import cymem.cymem --hidden-import thinc.linalg --hidden-import murmurhash.mrmr --hidden-import cytoolz.utils --hidden-import cytoolz._signatures --hidden-import spacy.strings --hidden-import spacy.morphology --hidden-import spacy.lexeme --hidden-import spacy.tokens --hidden-import spacy.gold --hidden-import spacy.tokens.underscore --hidden-import spacy.parts_of_speech --hidden-import dill --hidden-import spacy.tokens.printers --hidden-import spacy.tokens._retokenize --hidden-import spacy.syntax --hidden-import spacy.syntax.stateclass --hidden-import spacy.syntax.transition_system --hidden-import spacy.syntax.nonproj --hidden-import spacy.syntax.nn_parser --hidden-import spacy.syntax.arc_eager --hidden-import thinc.extra.search --hidden-import spacy.syntax._beam_utils --hidden-import spacy.syntax.ner --hidden-import thinc.neural._classes.difference --hidden-import spacy.vocab --hidden-import spacy.lemmatizer --hidden-import spacy._ml --hidden-import spacy.lang.en --hiddenimport thinc.neural._custom_kernels
但是现在可执行文件出现以下错误
File "site-packages\thinc\neural\train.py", line 7, in <module>
File "optimizers.pyx", line 14, in init thinc.neural.optimizers
File "ops.pyx", line 24, in init thinc.neural.ops
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "C:\Users\rajesh.das\AppData\Local\Continuum\anaconda3\envs\test_env2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "site-packages\thinc\neural\_custom_kernels.py", line 36, in <module>
File "pathlib.py", line 1183, in open
File "pathlib.py", line 1037, in _opener
File "pathlib.py", line 387, in wrapped
FileNotFoundError: [Errno 2] No such file or directory: 'D:\rajesh\python\console\test_env2_live\dist\main\thinc\neural\_custom_kernels.cu'
不确定为什么会出现 FileNotFoundError 错误。谁能帮我？
配置：Python3.6.7，conda 4.8.0，spacy v2.2.3，Windows10
我可以成功运行 低版本spacy(2.0.18) 的可执行文件。但它在 2.2.3 版本中给出了错误。
提前致谢
FileNotFound 错误是因为 PyInstaller 没有正确打包 thinc； thinc 需要一个钩子。我发现包含 from spacy import * 的脚本将与下面的挂钩文件一起使用。我使用的命令是：
pyinstaller test-spacy.py --additional-hooks-dir=.
之所以有效，是因为我在与 test-spacy.py 脚本相同的目录中为 spacy（及其子模块）添加了一个挂钩。只需将以下文本复制到名为 hook-spacy.py 的文件中，该文件与您的脚本位于同一目录中。
# HOOK FILE FOR SPACY
from PyInstaller.utils.hooks import collect_all
# ----------------------------- SPACY -----------------------------
data = collect_all('spacy')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
# ----------------------------- THINC -----------------------------
data = collect_all('thinc')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- CYMEM -----------------------------
data = collect_all('cymem')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- PRESHED -----------------------------
data = collect_all('preshed')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- BLIS -----------------------------
data = collect_all('blis')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# This hook file is a bit of a hack - really, all of the libraries should be in seperate hook files. (Eg hook-blis.py with the blis part of the hook)
如果您需要更多帮助，请在评论中提问。
我调整了 .spec 文件并添加了具有以下详细信息的 main.spec 文件并且它有效。
# -*- mode: python ; coding: utf-8 -*-
import PyInstaller
datas = []
datas.extend(PyInstaller.utils.hooks.collect_data_files('spacy.lang', include_py_files = True))
datas.extend(PyInstaller.utils.hooks.collect_data_files('thinc'))
block_cipher = None
a = Analysis(['main.py'],
pathex=['D:\rajesh\python\console\live'],
binaries=[],
datas=datas,
hiddenimports = [
'spacy.kb',
'spacy.lexeme',
'spacy.matcher._schemas',
'spacy.morphology',
'spacy.parts_of_speech',
'spacy.syntax._beam_utils',
'spacy.syntax._parser_model',
'spacy.syntax.arc_eager',
'spacy.syntax.ner',
'spacy.syntax.nn_parser',
'spacy.syntax.stateclass',
'spacy.syntax.transition_system',
'spacy.tokens._retokenize',
'spacy.tokens.morphanalysis',
'spacy.tokens.underscore',
'spacy._align',
'blis',
'blis.py',
'cymem',
'cymem.cymem',
'murmurhash',
'murmurhash.mrmr',
'preshed.maps',
'srsly.msgpack.util',
'thinc.extra.search',
'thinc.linalg',
'thinc.neural._aligned_alloc',
'thinc.neural._custom_kernels',
'sklearn.utils._cython_blas',
'sklearn.neighbors.typedefs',
'sklearn.neighbors.quad_tree',
'sklearn.tree._utils'
],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
[],
exclude_binaries=True,
name='main',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True )
coll = COLLECT(exe,
a.binaries,/
a.zipfiles,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name='main')
我同意@Legorooj。添加到他的回答中，最后我不得不为 srsly.msgpack.util 添加另一个隐藏的导入。同样在我的脚本中，我使用了 "en_core_web_sm":
import en_core_web_sm
nlp = en_core_web_sm.load()
所以我也为“en_core_web_sm”添加了一个钩子文件。 hook-spacy.py 文件变为：
# HOOK FILE FOR SPACY
from PyInstaller.utils.hooks import collect_all
# ----------------------------- SPACY -----------------------------
data = collect_all('spacy')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
# ----------------------------- THINC -----------------------------
data = collect_all('thinc')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- CYMEM -----------------------------
data = collect_all('cymem')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- PRESHED -----------------------------
data = collect_all('preshed')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- BLIS -----------------------------
data = collect_all('blis')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- OTHER ----------------------------
hiddenimports += ['srsly.msgpack.util']
hook-en_core_web_sm 文件是：
from PyInstaller.utils.hooks import collect_data_files
datas = collect_data_files("en_core_web_sm")
# -*- mode: python ; coding: utf-8 -*-
block_cipher = None
import PyInstaller
# HOOK FILE FOR SPACY
from PyInstaller.utils.hooks import collect_all
from PyInstaller.utils.hooks import collect_data_files
# ----------------------------- SPACY -----------------------------
data = collect_all('spacy')
datas = data[0]
binaries = data[1]
hiddenimports = data[2]
# ----------------------------- THINC -----------------------------
data = collect_all('thinc')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- CYMEM -----------------------------
data = collect_all('cymem')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- PRESHED -----------------------------
data = collect_all('preshed')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- BLIS -----------------------------
data = collect_all('blis')
datas += data[0]
binaries += data[1]
hiddenimports += data[2]
# ----------------------------- OTHER ----------------------------
hiddenimports += ['srsly.msgpack.util']
# ----------------------------- Sacy Model -------------------------
datas += collect_data_files("en_core_web_sm")
a = Analysis(['ttest_spacy.py'],
pathex=['PATH/TO/PYFILE'],
binaries=binaries,
datas=datas,
hiddenimports = hiddenimports,
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
[],
exclude_binaries=True,
name='ttest_spacy',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True )
coll = COLLECT(exe,
a.binaries,
a.zipfiles,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name='ttest_spacy')