scipy 版本可以更改 scipy.interpolate.griddata 结果吗？
Can the scipy version change scipy.interpolate.griddata results?
我的问题从 scipy 1.2.3 及其函数 scipy.interpolate.griddata 开始，它执行插值并为我提供参考数据集。 （我对三次 2d 插值感兴趣，请参见下面的测试用例）
将 scipy 更新为 scipy 1.5.2 后，我无法生成与之前完全相同的结果...不同之处在于不可忽略。
通过测试在我的 anaconda 发行版中可用的 scipy 的先前版本，如果我安装 scipy 1.3.2.[=16=，我会生成准确的初始插值结果]
所以我认为 griddata 或其子组件之一是在 scipy 1.3.2.
之后更新的
但是我在 Scipy 发行说明中找不到关于它的任何解释：Scipy.org Release Notes, nothing in the history for scipy/scipy/interpolate/ndgriddata.py on GitHub History ndgriddata, nothing in the history for scipy/scipy/interpolate/interpnd.pyx on GitHub History interpnd。也许我没有看到明显的东西？
有没有人遇到过这个问题：更新 scipy 改变了 scipy.interpolate.griddata 给出的结果？
为了做一个测试用例，我借用了一些代码：（非常感谢）
from scipy.interpolate import griddata
import numpy as np
# scipy 1.2.3 or (scipy 1.3.2) reference dataset
z_griddata_scipy_1_2_3 = np.array([[1.22464680e-16, 2.99260075e-02, 4.64921877e-02, 3.63387200e-02,
-1.17334278e-02, -4.10790167e-02, -3.53276896e-02, -1.32599029e-02,
6.57516828e-03, 1.46193750e-02, 1.29942167e-02, 4.60176170e-03,
-1.02398072e-02, -3.13455739e-02, -3.89274672e-02, -1.15549286e-02,
3.59960447e-02, 4.60537630e-02, 2.96438015e-02, 1.22464680e-16],
[3.06593878e-01, 2.94590471e-01, 2.55311166e-01, 1.72704804e-01,
6.75755257e-02, -8.71796149e-02, -1.69793095e-01, -2.16754270e-01,
-2.45929090e-01, -2.64204208e-01, -2.83893302e-01, -2.86038057e-01,
-2.52505900e-01, -1.93389278e-01, -9.70877464e-02, 6.22252315e-02,
1.64062151e-01, 2.49498113e-01, 2.91797267e-01, 3.07425460e-01]])
# auxiliary function for mesh generation
def gimme_mesh(n):
minval = -1
maxval = 1
# produce an asymmetric shape in order to catch issues with transpositions
return np.meshgrid(np.linspace(minval, maxval, n), np.linspace(minval, maxval, n+1))
# set up underlying test functions, vectorized
def fun_smooth(x, y):
return np.cos(np.pi * x)*np.sin(np.pi * y)
def test_griddata_cubic():
# sparse input mesh, 6x7 in shape
N_sparse = 6
x_sparse, y_sparse = gimme_mesh(N_sparse)
z_sparse_smooth = fun_smooth(x_sparse, y_sparse)
# dense output mesh, 20x21 in shape
N_dense = 20
x_dense, y_dense = gimme_mesh(N_dense)
z_griddata_scipy_test = griddata(np.array([x_sparse.ravel(), y_sparse.ravel()]).T,
z_sparse_smooth.ravel(),
(x_dense, y_dense),
method='cubic')
try:
np.testing.assert_almost_equal(z_griddata_scipy_1_2_3, z_griddata_scipy_test[:2], decimal=5)
except AssertionError as err:
print (err)
if __name__ == '__main__':
"""
"""
test_griddata_cubic()
我电脑上的测试结果Windows7,Python3.7,scipy1.5.2:
Arrays are not almost equal to 5 decimals
Mismatched elements: 38 / 40 (95%)
Max absolute difference: 0.03821737
Max relative difference: 0.67726368
x: array([[ 1.22465e-16, 2.99260e-02, 4.64922e-02, 3.63387e-02,
-1.17334e-02, -4.10790e-02, -3.53277e-02, -1.32599e-02,
6.57517e-03, 1.46194e-02, 1.29942e-02, 4.60176e-03,...
y: array([[ 1.22465e-16, 2.97398e-02, 4.62030e-02, 3.61127e-02,
-1.15711e-02, -3.85005e-02, -3.03032e-02, -9.36536e-03,
3.92018e-03, 1.17290e-02, 1.37729e-02, 6.40206e-03,...
我可以观察到差异是不可忽略的！
由于版本 1.4.0 的行为已经不同，运行 对 1.3.2 和 1.4.0 之间的所有 commits 进行二分，并从每个源代码构建 SciPy提交 + 运行 测试脚本，有助于缩小相关更改的范围。
结果是 these commits that apparently introduced a new qh_new_qhull_scipy algorithm. This algorithm in turn is used by the griddata function. According to this commit 引入了行为更改，qh_new_qhull 之前已修补，因此您获得的新结果很可能是正确的。
以下是缩小到相关提交的详细步骤。
准备步骤：
- 创建一个新的虚拟环境。
- 获取SciPy源代码：
git clone https://github.com/scipy/scipy.git；和 mv scipy scipy-source.
- 安装 building SciPy from source (+
pip install tempita) 所需的依赖项。
- 将 OP 的代码复制并粘贴到文件
test.py 中，然后删除 np.testing.assert_almost_equal. 周围的 try/except
- 运行 下面的
main 脚本（这会花很长时间）。
主脚本：
from pathlib import Path
import shlex
import subprocess
import sys
def log(msg, *, level=0, newline=False):
indent = ' ' * level
print(f'{indent}{msg}', end='\n' if newline else '', flush=True)
def run(cmd, **kwargs):
if cmd.startswith('git'):
kwargs['cwd'] = 'scipy-source'
kwargs.update(check=True, capture_output=True, text=True)
return subprocess.run(shlex.split(cmd), **kwargs).stdout.strip()
def run_and_log(cmd, **kwargs):
exc, *args = shlex.split(cmd)
exc = Path(exc).name
log(f'{" ".join([exc, *args])}: ', level=1)
text = run(cmd, **kwargs).strip()
if text:
text = text.splitlines()[-1]
log(text, newline=True)
return text
python = sys.executable
ancestor = run('git merge-base v1.3.2 v1.4.0')
commits = run(f'git --no-pager log --pretty=oneline --reverse --ancestry-path {ancestor}..v1.4.0').splitlines()
commits = [x.split(' ', maxsplit=1)[0] for x in commits]
log(f'Scanning {len(commits)} commits', newline=True)
low, high = 0, len(commits)
index = (low + high) // 2
while 0 < index < len(commits):
commit = commits[index]
log(f'{commit[:8]} ({index})', newline=True)
run_and_log(f'{python} -m pip uninstall -y scipy')
run_and_log(f'git checkout {commit}')
try:
run_and_log(f'{python} -m pip install .', cwd='scipy-source')
except subprocess.CalledProcessError:
log('build failed', newline=True)
index += 1
continue
try:
run_and_log(f'{python} test.py')
except subprocess.CalledProcessError:
log('failed', newline=True)
high = index
else:
low = index
if high - low <= 1:
break
index = (low + high) // 2
它产生以下输出：
$ python main.py
Scanning 1281 commits
19f4c290 (640)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+18219f5
git checkout 19f4c2900d6c62d1e56c2faecd8a2b1d584c094e:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+19f4c29
python test.py: failed
983e83e5 (320)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+19f4c29
git checkout 983e83e549a1c6f04b4657d2396dc47ab4b8d0e1:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+983e83e
python test.py:
a3276047 (480)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+983e83e
git checkout a3276047bb3493eeab6dac5148615cc8010faac0:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+a327604
python test.py: failed
35f86bc3 (400)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+a327604
git checkout 35f86bc33016dc88fc4340e4dc3f23bf86e4f311:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+35f86bc
python test.py: failed
7b0345e9 (360)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+35f86bc
git checkout 7b0345e9c038d7c1082ec0097f1ed4a626734bf4:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+7b0345e
python test.py: failed
f9525c3c (340)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+7b0345e
git checkout f9525c3ce7a9b63afda23b0af2ad32c7fbcedaa9:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+f9525c3
python test.py:
4a71ecf7 (350)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+f9525c3
git checkout 4a71ecf72be8321eb9a5faf7307f4c18cb0cb500:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+4a71ecf
python test.py: failed
9b7879b8 (345)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+4a71ecf
git checkout 9b7879b832aa52131383999fc7dd0dfa3575a4c2:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+9b7879b
python test.py: failed
5481d175 (342)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+9b7879b
git checkout 5481d175bb29d7162ac9eed18bb9181a2b566f20:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+5481d17
python test.py:
f2854466 (343)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+5481d17
git checkout f28544666cbc74c86c21fc2b320bc31f9fbd1c2c:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+f285446
python test.py:
18219f5a (344)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+f285446
git checkout 18219f5a1971be9a3fd6d590aeb496a419e8cd0e:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+18219f5
python test.py: failed
我的问题从 scipy 1.2.3 及其函数 scipy.interpolate.griddata 开始，它执行插值并为我提供参考数据集。 （我对三次 2d 插值感兴趣，请参见下面的测试用例）
将 scipy 更新为 scipy 1.5.2 后，我无法生成与之前完全相同的结果...不同之处在于不可忽略。
通过测试在我的 anaconda 发行版中可用的 scipy 的先前版本，如果我安装 scipy 1.3.2.[=16=，我会生成准确的初始插值结果]
所以我认为 griddata 或其子组件之一是在 scipy 1.3.2.
之后更新的但是我在 Scipy 发行说明中找不到关于它的任何解释：Scipy.org Release Notes, nothing in the history for scipy/scipy/interpolate/ndgriddata.py on GitHub History ndgriddata, nothing in the history for scipy/scipy/interpolate/interpnd.pyx on GitHub History interpnd。也许我没有看到明显的东西？
有没有人遇到过这个问题：更新 scipy 改变了 scipy.interpolate.griddata 给出的结果？
为了做一个测试用例，我借用了一些代码：
from scipy.interpolate import griddata
import numpy as np
# scipy 1.2.3 or (scipy 1.3.2) reference dataset
z_griddata_scipy_1_2_3 = np.array([[1.22464680e-16, 2.99260075e-02, 4.64921877e-02, 3.63387200e-02,
-1.17334278e-02, -4.10790167e-02, -3.53276896e-02, -1.32599029e-02,
6.57516828e-03, 1.46193750e-02, 1.29942167e-02, 4.60176170e-03,
-1.02398072e-02, -3.13455739e-02, -3.89274672e-02, -1.15549286e-02,
3.59960447e-02, 4.60537630e-02, 2.96438015e-02, 1.22464680e-16],
[3.06593878e-01, 2.94590471e-01, 2.55311166e-01, 1.72704804e-01,
6.75755257e-02, -8.71796149e-02, -1.69793095e-01, -2.16754270e-01,
-2.45929090e-01, -2.64204208e-01, -2.83893302e-01, -2.86038057e-01,
-2.52505900e-01, -1.93389278e-01, -9.70877464e-02, 6.22252315e-02,
1.64062151e-01, 2.49498113e-01, 2.91797267e-01, 3.07425460e-01]])
# auxiliary function for mesh generation
def gimme_mesh(n):
minval = -1
maxval = 1
# produce an asymmetric shape in order to catch issues with transpositions
return np.meshgrid(np.linspace(minval, maxval, n), np.linspace(minval, maxval, n+1))
# set up underlying test functions, vectorized
def fun_smooth(x, y):
return np.cos(np.pi * x)*np.sin(np.pi * y)
def test_griddata_cubic():
# sparse input mesh, 6x7 in shape
N_sparse = 6
x_sparse, y_sparse = gimme_mesh(N_sparse)
z_sparse_smooth = fun_smooth(x_sparse, y_sparse)
# dense output mesh, 20x21 in shape
N_dense = 20
x_dense, y_dense = gimme_mesh(N_dense)
z_griddata_scipy_test = griddata(np.array([x_sparse.ravel(), y_sparse.ravel()]).T,
z_sparse_smooth.ravel(),
(x_dense, y_dense),
method='cubic')
try:
np.testing.assert_almost_equal(z_griddata_scipy_1_2_3, z_griddata_scipy_test[:2], decimal=5)
except AssertionError as err:
print (err)
if __name__ == '__main__':
"""
"""
test_griddata_cubic()
我电脑上的测试结果Windows7,Python3.7,scipy1.5.2:
Arrays are not almost equal to 5 decimals
Mismatched elements: 38 / 40 (95%)
Max absolute difference: 0.03821737
Max relative difference: 0.67726368
x: array([[ 1.22465e-16, 2.99260e-02, 4.64922e-02, 3.63387e-02,
-1.17334e-02, -4.10790e-02, -3.53277e-02, -1.32599e-02,
6.57517e-03, 1.46194e-02, 1.29942e-02, 4.60176e-03,...
y: array([[ 1.22465e-16, 2.97398e-02, 4.62030e-02, 3.61127e-02,
-1.15711e-02, -3.85005e-02, -3.03032e-02, -9.36536e-03,
3.92018e-03, 1.17290e-02, 1.37729e-02, 6.40206e-03,...
我可以观察到差异是不可忽略的！
由于版本 1.4.0 的行为已经不同，运行 对 1.3.2 和 1.4.0 之间的所有 commits 进行二分，并从每个源代码构建 SciPy提交 + 运行 测试脚本，有助于缩小相关更改的范围。
结果是 these commits that apparently introduced a new qh_new_qhull_scipy algorithm. This algorithm in turn is used by the griddata function. According to this commit 引入了行为更改，qh_new_qhull 之前已修补，因此您获得的新结果很可能是正确的。
以下是缩小到相关提交的详细步骤。
准备步骤：
- 创建一个新的虚拟环境。
- 获取SciPy源代码：
git clone https://github.com/scipy/scipy.git；和mv scipy scipy-source. - 安装 building SciPy from source (+
pip install tempita) 所需的依赖项。 - 将 OP 的代码复制并粘贴到文件
test.py中，然后删除np.testing.assert_almost_equal. 周围的 - 运行 下面的
main脚本（这会花很长时间）。
try/except
主脚本：
from pathlib import Path
import shlex
import subprocess
import sys
def log(msg, *, level=0, newline=False):
indent = ' ' * level
print(f'{indent}{msg}', end='\n' if newline else '', flush=True)
def run(cmd, **kwargs):
if cmd.startswith('git'):
kwargs['cwd'] = 'scipy-source'
kwargs.update(check=True, capture_output=True, text=True)
return subprocess.run(shlex.split(cmd), **kwargs).stdout.strip()
def run_and_log(cmd, **kwargs):
exc, *args = shlex.split(cmd)
exc = Path(exc).name
log(f'{" ".join([exc, *args])}: ', level=1)
text = run(cmd, **kwargs).strip()
if text:
text = text.splitlines()[-1]
log(text, newline=True)
return text
python = sys.executable
ancestor = run('git merge-base v1.3.2 v1.4.0')
commits = run(f'git --no-pager log --pretty=oneline --reverse --ancestry-path {ancestor}..v1.4.0').splitlines()
commits = [x.split(' ', maxsplit=1)[0] for x in commits]
log(f'Scanning {len(commits)} commits', newline=True)
low, high = 0, len(commits)
index = (low + high) // 2
while 0 < index < len(commits):
commit = commits[index]
log(f'{commit[:8]} ({index})', newline=True)
run_and_log(f'{python} -m pip uninstall -y scipy')
run_and_log(f'git checkout {commit}')
try:
run_and_log(f'{python} -m pip install .', cwd='scipy-source')
except subprocess.CalledProcessError:
log('build failed', newline=True)
index += 1
continue
try:
run_and_log(f'{python} test.py')
except subprocess.CalledProcessError:
log('failed', newline=True)
high = index
else:
low = index
if high - low <= 1:
break
index = (low + high) // 2
它产生以下输出：
$ python main.py
Scanning 1281 commits
19f4c290 (640)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+18219f5
git checkout 19f4c2900d6c62d1e56c2faecd8a2b1d584c094e:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+19f4c29
python test.py: failed
983e83e5 (320)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+19f4c29
git checkout 983e83e549a1c6f04b4657d2396dc47ab4b8d0e1:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+983e83e
python test.py:
a3276047 (480)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+983e83e
git checkout a3276047bb3493eeab6dac5148615cc8010faac0:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+a327604
python test.py: failed
35f86bc3 (400)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+a327604
git checkout 35f86bc33016dc88fc4340e4dc3f23bf86e4f311:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+35f86bc
python test.py: failed
7b0345e9 (360)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+35f86bc
git checkout 7b0345e9c038d7c1082ec0097f1ed4a626734bf4:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+7b0345e
python test.py: failed
f9525c3c (340)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+7b0345e
git checkout f9525c3ce7a9b63afda23b0af2ad32c7fbcedaa9:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+f9525c3
python test.py:
4a71ecf7 (350)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+f9525c3
git checkout 4a71ecf72be8321eb9a5faf7307f4c18cb0cb500:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+4a71ecf
python test.py: failed
9b7879b8 (345)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+4a71ecf
git checkout 9b7879b832aa52131383999fc7dd0dfa3575a4c2:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+9b7879b
python test.py: failed
5481d175 (342)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+9b7879b
git checkout 5481d175bb29d7162ac9eed18bb9181a2b566f20:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+5481d17
python test.py:
f2854466 (343)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+5481d17
git checkout f28544666cbc74c86c21fc2b320bc31f9fbd1c2c:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+f285446
python test.py:
18219f5a (344)
python -m pip uninstall -y scipy: Successfully uninstalled scipy-1.4.0.dev0+f285446
git checkout 18219f5a1971be9a3fd6d590aeb496a419e8cd0e:
python -m pip install .: Successfully installed scipy-1.4.0.dev0+18219f5
python test.py: failed