(Python) ImportError: cannot import name 'SemLock' from '_multiprocessing'

Question

我正在尝试在 Red Hat Linux 共享服务器上构建一个 Python 3.8 解释器。编译器构建和执行程序，但那些需要 Posix 命名信号量操作的程序除外。例如，concurrent.futures 模块中的 ThreadPoolExecutor class 或 multiprocessing 模块中的 Queue class。例外情况是：

Traceback (most recent call last):
  File "/home/friendv0/Python3/lib/python3.8/multiprocessing/synchronize.py", line 28, in <module>
    from _multiprocessing import SemLock, sem_unlink
ImportError: cannot import name 'SemLock' from '_multiprocessing' (/home/friendv0/Python3/lib/python3.8/lib-dynload/_multiprocessing.cpython-38-x86_64-linux-gnu.so)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./temp.py", line 53, in <module>
    main()
  File "./temp.py", line 30, in main
    tasks_to_accomplish = Queue()
  File "/home/friendv0/Python3/lib/python3.8/multiprocessing/context.py", line 103, in Queue
    return Queue(maxsize, ctx=self.get_context())
  File "/home/friendv0/Python3/lib/python3.8/multiprocessing/queues.py", line 39, in __init__
    from .synchronize import SEM_VALUE_MAX as maxsize
  File "/home/friendv0/Python3/lib/python3.8/multiprocessing/synchronize.py", line 30, in <module>
    raise ImportError("This platform lacks a functioning sem_open" +
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.

但我知道平台上有一个正常运行的 sem_open 实现。我从 Github 下载，编译并成功执行了以下 C 程序（仅显示部分）：

#include <stdio.h>          /* printf()                 */
#include <stdlib.h>         /* exit(), malloc(), free() */
#include <unistd.h>
#include <sys/types.h>      /* key_t, sem_t, pid_t      */
#include <sys/wait.h>
#include <sys/shm.h>        /* shmat(), IPC_RMID        */
#include <errno.h>          /* errno, ECHILD            */
#include <semaphore.h>      /* sem_open(), sem_destroy(), sem_wait().. */
#include <fcntl.h>          /* O_CREAT, O_EXEC          */


int main (int argc, char **argv){
    int i;                        /*      loop variables          */
    key_t shmkey;                 /*      shared memory key       */
    int shmid;                    /*      shared memory id        */
    sem_t *sem;                   /*      synch semaphore         *//*shared */
    pid_t pid;                    /*      fork pid                */
    int *p;                       /*      shared variable         *//*shared */
    unsigned int n;               /*      fork count              */
    unsigned int value;           /*      semaphore value         */

    /* initialize a shared variable in shared memory */
    shmkey = ftok ("/dev/null", 5);       /* valid directory name and a number */

    /*
      code elided
    */

    /* initialize semaphores for shared processes */
    sem = sem_open ("pSem", O_CREAT | O_EXCL, 0644, value);
    /* name of semaphore is "pSem", semaphore is reached using this name */
    sem_unlink ("pSem");

上面的程序是用 -pthread 标志编译的。

我检查了我的 Python 安装。来自 config.log:

configure:11488: checking for sem_open
configure:11488: gcc -pthread -o conftest  -I /home1/friendv0/local/lib/libffi-3.2.1/include -L/home/friendv0/openssl/lib -Wl,-rpath,/home/friendv0/openssl/lib conftest.c -lpthread -ldl  >&5
configure:11488: $? = 0
configure:11488: result: yes

并且来自 Makefile：

CC=             gcc -pthread
CXX=            g++ -pthread

我什至进行了清理和重建以确保使用了正确的编译器标志。我唯一能想到的是它不是 loading/linking 正确的 C 运行时库。

有人知道是什么原因造成的吗？

Answer 1

首先，我从Github下载的使用信号量的C程序实际上并没有运行；它从未检查 sem_open 中的 return 值，即 SEM_FAILED.

我更仔细地检查了配置日志，但遗漏了以下内容：

configure:14565: checking whether POSIX semaphores are enabled
configure:14594: gcc -pthread -o conftest  -I /home1/friendv0/local/lib/libffi-3.2.1/include -L/home/friendv0/local/lib64 conftest.c -lcrypt -lpthread -ldl  -lutil -lm >&5
configure:14594: $? = 0
configure:14594: ./conftest
sem_open: Permission denied
configure:14594: $? = 1
configure: program exited with status 1
configure: failed program was:
| /* confdefs.h */
| #define _GNU_SOURCE 1
| #define _NETBSD_SOURCE 1
| #define __BSD_VISIBLE 1

/* many lines elided */

| /* end confdefs.h.  */
|
| #include <unistd.h>
| #include <fcntl.h>
| #include <stdio.h>
| #include <semaphore.h>
| #include <sys/stat.h>
|
| int main(void) {
|   sem_t *a = sem_open("/autoconf", O_CREAT, S_IRUSR|S_IWUSR, 0);
|   if (a == SEM_FAILED) {
|     perror("sem_open");
|     return 1;
|   }
|   sem_close(a);
|   sem_unlink("/autoconf");
|   return 0;
| }
|

在做了一些研究之后，上面的 Permission denied 错误可能是由于没有对 /dev/shm 的写访问权限引起的（命名信号量使用共享内存，我是在共享服务器上）。该目录的当前属性是：

drwxr-xr-x  2 root root          40 Jul 23 02:20 shm

棘手的一点是说服我的 ISP 向用户授予写入权限不是安全问题。

(Python) ImportError: cannot import name 'SemLock' from '_multiprocessing'

(Python) ImportError: cannot import name 'SemLock' from '_multiprocessing'

python

build

python-3.x

python-multiprocessing