Dask 是否允许使用 Dask 系列进行索引？

Question

我看到的行为看起来像是 Dask 中的错误，但我想确保我没有做错什么。

我有一个名为 labeled_texts 的 Dask 数据框。它包含一个名为 "text" 的列。我计算了一个名为 label_rows 的 Dask 系列，它包含布尔值并且与 labeled_texts 具有相同的长度。我用它来索引 labeled_texts 并从那个较小的数据框中得到 "text" 列，就像这样。

labeled_text[label_rows]["text"].compute()

当我运行上述行时，我在 Dask/Pandas 代码中得到 KeyError: 'text' 方式。但是，以下命令有效

labeled_text[label_rows].compute()["text"]
labeled_text[label_rows.compute()]["text"]

我认为所有三个命令都应该产生相同的结果，第一个应该不会导致错误。这是正确的吗？

不幸的是，我无法在此处提出可以 post 的最小重现方案。这个问题在一个特定的集群上一直发生，但是相同的代码和数据运行ning 在不同的机器上工作正常。（这进一步让我认为这是一个 Dask 错误。）

如果没有更好的重现场景，我不指望任何人能够为我解决这个问题。我只是想确保我没有做错什么。

这是完整的堆栈跟踪。

  Traceback (most recent call last):

  ...my code that ultimately calls compute()...

    File "/usr/local/lib/python3.6/site-packages/dask/base.py", line 175, in compute
      (result,) = compute(self, traverse=False, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/dask/base.py", line 446, in compute
      results = schedule(dsk, keys, **kwargs)
    File "/usr/local/lib/python3.6/site-packages/distributed/client.py", line 2510, in get
      results = self.gather(packed, asynchronous=asynchronous, direct=direct)
    File "/usr/local/lib/python3.6/site-packages/distributed/client.py", line 1812, in gather
      asynchronous=asynchronous,
    File "/usr/local/lib/python3.6/site-packages/distributed/client.py", line 753, in sync
      self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
    File "/usr/local/lib/python3.6/site-packages/distributed/utils.py", line 337, in sync
      six.reraise(*error[0])
    File "/usr/local/lib/python3.6/site-packages/six.py", line 693, in reraise
      raise value
    File "/usr/local/lib/python3.6/site-packages/distributed/utils.py", line 322, in f
      result[0] = yield future
    File "/usr/local/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
      value = future.result()
    File "/usr/local/lib/python3.6/site-packages/distributed/client.py", line 1668, in _gather
      six.reraise(type(exception), exception, traceback)
    File "/usr/local/lib/python3.6/site-packages/six.py", line 692, in reraise
      raise value.with_traceback(tb)
    File "/usr/local/lib/python3.6/site-packages/dask/optimization.py", line 1059, in __call__
      return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
    File "/usr/local/lib/python3.6/site-packages/dask/core.py", line 149, in get
      result = _execute_task(task, cache)
    File "/usr/local/lib/python3.6/site-packages/dask/core.py", line 119, in _execute_task
      return func(*args2)
    File "/usr/local/lib/python3.6/site-packages/pandas/core/frame.py", line 2980, in __getitem__
      indexer = self.columns.get_loc(key)
    File "/usr/local/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2899, in get_loc
      return self._engine.get_loc(self._maybe_cast_indexer(key))
    File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc
    File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
    File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item
    File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
  KeyError: 'text'

Answer 1

没有什么特别突出的。正如你所建议的，我建议尝试提供一个最小的 reproducver。

Dask 是否允许使用 Dask 系列进行索引？

Does Dask allow indexing with a Dask series?

python

dask

dask-distributed