为什么在 运行 一个 pandas 操作时我会收到 dask 警告？
Why am I getting dask warnings when running a pandas operation?
我有一个同时具有 pandas 和 dask 操作的笔记本。
当我还没有启动客户端时，一切都如预期的那样。但是一旦我启动 dask.distributed 客户端，我就会在我正在 运行 pandas 操作的单元格中收到警告，例如pd.read_parquet('my_file')
当我开始工作时，我得到了准确的 Nanny 行数。
警告示例：
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.26s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.38s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.38s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.38s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.37s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Scheduler for 1.37s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.36s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
我想知道为什么，以及如何让它们停止。
此警告表示 Dask 工作进程长时间无响应。这很糟糕，因为工作人员将无法向其他工作人员提供数据，无法与调度程序通信等。这不正常，即使 运行 宁计算，因为这些计算正在 运行单独的线程。
造成这个问题的主要原因有两个：
- 您的任务 运行 不释放 GIL 的功能。这几天 罕见 （大多数 pandas 操作释放 GIL）但可能会发生。我相信 read_parquet 的所有变体都发布了 GIL
- 如果这种情况只发生一次并且只在启动时发生，那么这是一个已在
distributed.__version__ == '1.21.3' 左右修复的错误。您可能想要升级。
您还可以通过增加 ~/.dask/config.yaml 文件中允许的最大滴答时间来消除警告
tick-maximum-delay: 10 s
我有一个同时具有 pandas 和 dask 操作的笔记本。
当我还没有启动客户端时，一切都如预期的那样。但是一旦我启动 dask.distributed 客户端，我就会在我正在 运行 pandas 操作的单元格中收到警告，例如pd.read_parquet('my_file')
当我开始工作时，我得到了准确的 Nanny 行数。
警告示例：
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.26s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.38s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.38s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.38s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.37s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Scheduler for 1.37s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
distributed.core - WARNING - Event loop was unresponsive in Nanny for 1.36s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
我想知道为什么，以及如何让它们停止。
此警告表示 Dask 工作进程长时间无响应。这很糟糕，因为工作人员将无法向其他工作人员提供数据，无法与调度程序通信等。这不正常，即使 运行 宁计算，因为这些计算正在 运行单独的线程。
造成这个问题的主要原因有两个：
- 您的任务 运行 不释放 GIL 的功能。这几天 罕见 （大多数 pandas 操作释放 GIL）但可能会发生。我相信 read_parquet 的所有变体都发布了 GIL
- 如果这种情况只发生一次并且只在启动时发生，那么这是一个已在
distributed.__version__ == '1.21.3'左右修复的错误。您可能想要升级。
您还可以通过增加 ~/.dask/config.yaml 文件中允许的最大滴答时间来消除警告
tick-maximum-delay: 10 s