JNA并发问题

Question

我正在尝试访问由 JNA 加载的本机 (fortran) 库 (mylib.so)。该库由 Spark-Job 并行访问。到目前为止，我还没有同步方法调用（也没有同步库），因为对共享库的调用是我计算中的瓶颈，它们必须运行并行。

我收到以下错误：

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007ffbcb5f8dcd, pid=58569, tid=140708155152128
#
# JRE version: Java(TM) SE Runtime Environment (8.0_60-b27) (build 1.8.0_60-b27)
*** Error in `/usr/java/jdk1.8.0_60/jre/bin/java': double free or corruption (!prev): 0x0000000001b756d0 ***
*** Error in `/usr/java/jdk1.8.0_60/jre/bin/java': free(): corrupted unsorted chunks: 0x0000000001b75010 ***
*** Error in `/usr/java/jdk1.8.0_60/jre/bin/java': double free or corruption (!prev): 0x0000000001b756d0 ***
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libc.so.6+0x38dcd]======= Backtrace: =========
======= Backtrace: =========
======= Backtrace: =========
/lib64/libc.so.6(+0x7d053)[0x7ffbcb63d053]
/lib64/libc.so.6(+0x7d053)[0x7ffbcb63d053]
/lib64/libc.so.6(+0x7d053)[0x7ffbcb63d053]
/lib64/libc.so.6(+0x38e90)[0x7ffbcb5f8e90]
/usr/java/jdk1.8.0_60/jre/lib/amd64/server/libjvm.so(+0x5d43f9)[0x7ffbcabba3f9]
/lib64/libc.so.6(+0x38e90)[0x7ffbcb5f8e90]
/lib64/libc.so.6(+0x38eb5)[0x7ffbcb5f8eb5]
/lib64/libc.so.6(+0x38e69)[0x7ffbcb5f8e69]
/lib64/libc.so.6(+0x38eb5)[0x7ffbcb5f8eb5]
  __run_exit_handlers+0x3d
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
/opt/usr/local/lib/mylib.so(for_exit+0x19)[0x7ff9285b0fa9]
/lib64/libc.so.6(+0x38eb5)[0x7ffbcb5f8eb5]
/opt/usr/local/lib/mylib.so(for_exit+0x19)[0x7ff9285b0fa9]
/opt/usr/local/lib/mylib.so(for__once_private+0x27)[0x7ff9285660b7]
/opt/usr/local/lib/mylib.so(for__once_private+0x27)[0x7ff9285660b7]
/opt/usr/local/lib/mylib.so(for__acquire_lun+0x814)[0x7ff92855ea04]
/opt/usr/local/lib/mylib.so(for__acquire_lun+0x814)[0x7ff92855ea04]
/opt/usr/local/lib/mylib.so(for_write_int_fmt+0x9c)[0x7ff92857f83c]
/opt/usr/local/lib/mylib.so(for__acquire_lun+0x814)[0x7ff92855ea04]
/opt/usr/local/lib/mylib.so(for_write_int_fmt+0x9c)[0x7ff92857f83c]
/opt/usr/local/lib/mylib.so(seterr_+0x1c8)[0x7ff928515328]
/opt/usr/local/lib/mylib.so(for_write_int_fmt+0x9c)[0x7ff92857f83c]
/opt/usr/local/lib/mylib.so(seterr_+0x1c8)[0x7ff928515328]
/opt/usr/local/lib/mylib.so(ddl2sf_+0x322)[0x7ff92850eb02]
/opt/usr/local/lib/mylib.so(seterr_+0x1c8)[0x7ff928515328]
/opt/usr/local/lib/mylib.so(entsrc_+0x80)[0x7ff92850edb0]
/opt/usr/local/lib/mylib.so(bspline_+0x52c)[0x7ff92850d66c]
/opt/usr/local/lib/mylib.so(ddl2sf_+0x2e7)[0x7ff92850eac7]
/opt/usr/local/lib/mylib.so(enter_+0x55)[0x7ff92850ecd5]
/opt/usr/local/lib/mylib.so(rspbsp_+0x4a)[0x7ff928523cda]
/opt/usr/local/lib/mylib.so(bspline_+0x52c)[0x7ff92850d66c]
/opt/usr/local/lib/mylib.so(ddl2sf_+0x16d)[0x7ff92850e94d]
/tmp/jna--1845237309/jna4302334124297214663.tmp(ffi_call_unix64+0x4c)[0x7ff92909465c]
/tmp/jna--1845237309/jna4302334124297214663.tmp(ffi_call+0x1d4)[0x7ff929094164]
/opt/usr/local/lib/mylib.so(bspline_+0x52c)[0x7ff92850d66c]
/opt/usr/local/lib/mylib.so(rspbsp_+0x4a)[0x7ff928523cda]
/tmp/jna--1845237309/jna4302334124297214663.tmp(+0x5870)[0x7ff929087870]
/tmp/jna--1845237309/jna4302334124297214663.tmp(Java_com_sun_jna_Native_invokeVoid+0x22)[0x7ff92908a462]
[0x7ffbb5015994]

据我所知，本机库不是线程安全的？从 stacktracke 看来，实际问题是 libc.so，还是我自己的库 mylib.so？

如果问题出在我自己的库中，是否可以通过制作共享对象的多个物理副本（例如每个线程一个）来解决这个问题？

Answer 1

这通常表示您的共享库或其使用存在问题。如果您没有将库设计为线程安全的，那么它很可能不是。单个进程无法多次加载本机库，因此没有简单的解决方案。有几个途径：

从您的本机库分配多个对象，并为每个 java 线程分配不同的对象。如果您的本机库在内部使用静态数据结构，则这可能无法实现。
通过公开它的 classes/methods 同步对本机库的访问。这样，一次只有一个 java 线程可以访问它们。但是，根据您的库和用例，这可能对性能没有帮助——所有操作仍可能在一个地方。
使库线程安全。这在母语中可能比 Java 困难得多。此外，一些算法不太适合并行化。

JNA并发问题

JNA concurrency issues

concurrency

jna