为什么在通过无服务器部署时 docker 找不到我自己的包？

Question

我想将我的 python 包部署到亚马逊并通过 lambda 使其可用。为此，我正在尝试无服务器。

当我尝试部署我的包时，我收到以下错误消息：

SLS_DEBUG=* serverless deploy --stage dev --aws-profile default
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command requirements
Serverless: Load command requirements:clean
Serverless: Load command requirements:install
Serverless: Load command requirements:cleanCache
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Generated requirements from /home/ola/projects/lambda/tagdoc/requirements.txt in /home/ola/projects/lambda/tagdoc/.serverless/requirements.txt...
Serverless: Installing requirements from /home/ola/projects/lambda/tagdoc/.serverless/requirements/requirements.txt ...
Serverless: Docker Image: lambci/lambda:build-python3.6
Requirement 'pkgg-0.1.0.tar.gz' looks like a filename, but the file does not exist
Processing ./pkgg-0.1.0.tar.gz
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/var/task/pkgg-0.1.0.tar.gz'


  Error --------------------------------------------------

  null

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

Error: null
    at installRequirements (/home/ola/projects/lambda/tagdoc/node_modules/serverless-python-requirements/lib/pip.js:262:11)
    at installRequirementsIfNeeded (/home/ola/projects/lambda/tagdoc/node_modules/serverless-python-requirements/lib/pip.js:448:3)
    at ServerlessPythonRequirements.installAllRequirements (/home/ola/projects/lambda/tagdoc/node_modules/serverless-python-requirements/lib/pip.js:527:29)
From previous event:
    at PluginManager.invoke (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:390:22)
    at PluginManager.spawn (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:408:17)
    at Deploy.BbPromise.bind.then.then (/usr/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:123:50)
From previous event:
    at Object.before:deploy:deploy [as hook] (/usr/lib/node_modules/serverless/lib/plugins/deploy/deploy.js:113:10)
    at BbPromise.reduce (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:390:55)
From previous event:
    at PluginManager.invoke (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:390:22)
    at PluginManager.run (/usr/lib/node_modules/serverless/lib/classes/PluginManager.js:421:17)
    at variables.populateService.then.then (/usr/lib/node_modules/serverless/lib/Serverless.js:157:33)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)
From previous event:
    at Serverless.run (/usr/lib/node_modules/serverless/lib/Serverless.js:144:8)
    at serverless.init.then (/usr/lib/node_modules/serverless/bin/serverless:43:50)
    at <anonymous>

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information -----------------------------
     OS:                     linux
     Node Version:           8.15.0
     Serverless Version:     1.30.1

我的 serverless.yml 文件如下所示：

# Welcome to Serverless!
#
# This file is the main config file for your service.
# It's very minimal at this point and uses default values.
# You can always add more config options for more control.
# We've included some commented out config examples here.
# Just uncomment any of them to get that config option.
#
# For full config options, check the docs:
#    docs.serverless.com
#
# Happy Coding!

service: tagdoc # NOTE: update this with your service name

# You can pin your service to only deploy with a specific Serverless version
# Check out our docs for more details
# frameworkVersion: "=X.X.X"

provider:
  name: aws
  runtime: python3.6
  region: eu-central-1
# you can overwrite defaults here
#  stage: dev

plugins:
  - serverless-python-requirements

# You need to have Docker installed to be able to set dockerizePip:
# true or dockerizePip: non-linux. Alternatively, you can set
# dockerizePip: false, and it will not use Docker packaging. But,
# Docker packaging is essential if you need to build native packages
# that are part of your dependencies like Psycopg2, NumPy, Pandas, etc
custom:
  pythonRequirements:
    dockerizePip: true

# you can define service wide environment variables here
#  environment:
#    variable1: value1

# you can add packaging information here
package:
  include:
    - ./nltk_data/*

functions:
  ttxt:
    handler: handler.ttxt
    events:
      - http:
          path: /ttxt
          method: get
          integration: lambda
          request:
            template:
              application/json: '{ "txt" : "$input.params(''txt'')" }'
          response:
            headers:
  turl:
    handler: handler.turl
    events:
      - http:
          path: /turl
          method: get
          integration: lambda
          request:
            template:
              application/json: '{ "url" : "$input.params(''url'')" }'
          response:
            headers:
              Content-Type: "'application/json'"

最重要的是，该目录包含所有必需的文件，特别是内置的 python 包：

tree -L 2 lambda/
lambda/
└── tagdoc
    ├── pkgg-0.1.0.tar.gz
    ├── handler.py
    ├── nltk_data
    ├── node_modules
    ├── package.json
    ├── package-lock.json
    ├── README.md
    ├── requirements.txt
    ├── serverless.yml
    └── srv

4 directories, 8 files

我目前是运行 Debian 9.5。我还通过 Github 与朋友运行共享了代码，同样 OS 并且他似乎可以部署该程序包。因此，它看起来与我的无服务器 / docker 的本地设置有关。但我不知道如何解决这个问题。任何帮助将不胜感激。

requirements.txt只包含一行：

cat requirements.txt 
pkgg-0.1.0.tar.gz

现在包本身有一些依赖性。通过仅包含包，它从包的 setup.py 文件中获取依赖项。

编辑

如其中一条评论所述。这是我的 .gitignore

#Compiled source#
#################
*.pyc

# no data files #
################
*.csv

#Log files#
#################
*.log

#swap files#
###############
*.swp
*.*~
\#*\#
.\#*

#django migration directory#
############################
venv*
srv*
theme
collectedstatic
stunnel

# Elastic Beanstalk Files
.elasticbeanstalk/*
!.elasticbeanstalk/*.cfg.yml
!.elasticbeanstalk/*.global.yml

#Serverless package
.serverless

这就是我的 .serverless 的样子。所有requirements.txt的内容都和上面的一模一样

tree -L 3 .serverless/
.serverless/
├── cloudformation-template-create-stack.json
├── requirements
│   └── requirements.txt
└── requirements.txt

1 directory, 3 files

Answer 1

项目 lambci/lambda describes itself as Images that (very closely) mimic the live AWS Lambda environment, if you want to make sure the environments is 100% compatible, spin up an small t2.micro instance on EC2 , connect via SSH and setup your project there, run pip install -r requirements.txton your shell, download all libraries and dispose the instance. That is exactly what Lambda will do next time it provision the environment 用于运行您的代码。

Answer 2

考虑按照此无服务器 blog post

中所述在 docker 容器中构建

Answer 3

dockerizePip: true => pip 将运行在容器中，默认情况下不与您的主机 OS 共享文件系统，但它可以下载 python包来自网络。看起来插件 serverless-python-requirements 没有将您的本地文件正确挂载到容器中，因此 pip 无法看到您的本地包 (pkgg-0.1.0.tar.gz)。

恕我直言，最好的解决方案是不要在 requirements.txt.

中使用本地文件

交叉引用：https://github.com/UnitedIncome/serverless-python-requirements/issues/258

Answer 4

问题似乎在 serverless-python-requirements 内。看起来 4.1.1 版可以完美运行，而 4.2.5 版则不然。我在 github

上提出了一个问题

为什么在通过无服务器部署时 docker 找不到我自己的包？

why can't docker find my own package when deploying via serverless?

docker

aws-lambda

serverless-framework