从 Numpy 结果分配 Koalas 列

Question

尝试复制 Databricks-Koalas 中的 Pandas 功能在 Pandas:

df = pd.DataFrame({'a': [450, 1, 26],
                   'b': [1, 450, 70],
                  })
thresh = [x for x in range(26)] # create a list 1 to 25
df["c"] = np.where((df.a.isin(thresh) | df.b.isin(thresh)), 1, 0) # find the values within the threshold and flag column 'c'
df
# returns
Out[32]: 
     a    b  c
0  450    1  1
1    1  450  1
2   26   70  0

在考拉中：

df = ks.DataFrame({'a': [450, 1, 26],
                   'b': [1, 450, 70],
                  })

thresh = [x for x in range(26)] # create a list 1 to 25
df = df.assign(c=np.where((df.a.isin(thresh) | df.b.isin(thresh)), 1, 0)) # find the values within the threshold and flag column 'c'
# returns
PandasNotImplementedError: The method `pd.Series.__iter__()` is not implemented. If you want to collect your data as an NumPy array, use 'to_numpy()' instead.

我该如何正确使用 to_numpy 或将 Numpy 结果包装在 ks.Series() 中以便 assign() 获取结果？

df = df.assign(c=ks.Series(np.where((df.a.isin(thresh) | df.b.isin(thresh)), 1, 0))) 给出与上面相同的错误。

有没有办法在考拉中复制 pandas 功能？

Answer 1

要在 ks.DataFrame 中执行您在此处执行的操作，您不需要 np.where，但您可以使用 astype:

df = df.assign(c= (df.a.isin(thresh) | df.b.isin(thresh)).astype(int) )
df
     a    b  c
0  450    1  1
1    1  450  1
2   26   70  0

从 Numpy 结果分配 Koalas 列

Assign Koalas Column from Numpy Result

python

numpy

pandas

databricks