scikit-learn：K-Means 和 MiniBatchKMeans 聚类算法的比较
scikit-learn: Comparison of the K-Means and MiniBatchKMeans clustering algorithms
我正在阅读 Clustering. They have an example comparing K-Means and MiniBatchKMeans 上的 scikit-learn 用户指南。
我对示例中的以下代码有点困惑：
# We want to have the same colors for the same cluster from the
# MiniBatchKMeans and the KMeans algorithm. Let's pair the cluster centers per
# closest one.
k_means_cluster_centers = np.sort(k_means.cluster_centers_, axis=0)
mbk_means_cluster_centers = np.sort(mbk.cluster_centers_, axis=0)
k_means_labels = pairwise_distances_argmin(X, k_means_cluster_centers)
mbk_means_labels = pairwise_distances_argmin(X, mbk_means_cluster_centers)
order = pairwise_distances_argmin(k_means_cluster_centers,
mbk_means_cluster_centers)
排序前后k-means聚类中心的值为：
k_means.cluster_centers_
array([[ 1.07705469, -1.06730994],
[-1.07159013, -1.00648645],
[ 0.96700708, 1.01837274]])
k_means_cluster_centers
array([[-1.07159013, -1.06730994],
[ 0.96700708, -1.00648645],
[ 1.07705469, 1.01837274]])
有三个中心，所以我假设每一行都是一个中心的 xy 坐标。
我不确定为什么他们在将每个点与最近的中心配对之前使用 np.sort()，因为这会扭曲中心的 x/y 坐标。也许他们只是想按 x 轴或 y 轴排序？
I am not sure why we use np.sort() here.
答案在评论中 - 但是，它的实现方式存在错误，请参见下文。
# We want to have the same colors for the same cluster from the
# MiniBatchKMeans and the KMeans algorithm. Let's pair the cluster centers per
# closest one.
配对在示例代码的下方两行完成：
k_means_cluster_centers = np.sort(k_means.cluster_centers_, axis=0)
mbk_means_cluster_centers = np.sort(mbk.cluster_centers_, axis=0)
(...)
mbk_means_cluster_centers)
order = pairwise_distances_argmin(k_means_cluster_centers,
mbk_means_cluster_centers)
在代码中，order 有效地用作查找 table 以获取 mbk_means_cluster_centers 中对应于 k_means_cluster_centers 的集群。
my_members = mbk_means_labels == order[k]
cluster_center = mbk_means_cluster_centers[order[k]]
It distorts the coordinate of calculated cluster centers.
（根据评论中的讨论更新）
的确，通过使用np.sort(..., axis=0)，中心坐标搞混了。正确的排序方式是使用 np.lexsort，像这样
arr = k_means.cluster_centers_
k_means_cluster_centers = arr[np.lexsort((arr[:, 0], arr[:, 1]))]
arr = mbk.cluster_centers_
mbk_means_cluster_center = arr[np.lexsort((arr[:, 0], arr[:, 1]))]
实际上这会改变示例的结果：
使用sort(..., axis=0)
使用np.lexsort
我认为你是对的。像本例中那样进行排序会混淆点的 x 和 y 坐标。它在示例中起作用的事实或多或少是巧合。
我们有 x 坐标 [1, -1, 1] 和 y 坐标 [1, -1, -1]。排序后它们变成 [-1, 1, 1] 和 [-1, -1, 1]，它们构成了我们最初的三对：
# original | sorted
# [ 1, -1] | [-1, -1]
# [-1, -1] | [ 1, -1]
# [ 1, 1] | [ 1, 1]
在下面观察使用四个集群时这是如何分解的。在这种情况下，我们有：
# original | sorted
# [-1, -1] | [-1, -1]
# [-1, 1] | [-1, -1]
# [ 1, -1] | [ 1, 1]
# [ 1, 1] | [ 1, 1]
不相同的点。
修改后的示例代码：
print(__doc__)
import time
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import MiniBatchKMeans, KMeans
from sklearn.metrics.pairwise import pairwise_distances_argmin
from sklearn.datasets.samples_generator import make_blobs
# #############################################################################
# Generate sample data
np.random.seed(0)
batch_size = 45
centers = [[1, 1], [-1, -1], [1, -1], [-1, 1]]
n_clusters = len(centers)
X, labels_true = make_blobs(n_samples=3000, centers=centers, cluster_std=0.7)
# #############################################################################
# Compute clustering with Means
k_means = KMeans(init='k-means++', n_clusters=4, n_init=10)
t0 = time.time()
k_means.fit(X)
t_batch = time.time() - t0
# #############################################################################
# Compute clustering with MiniBatchKMeans
mbk = MiniBatchKMeans(init='k-means++', n_clusters=4, batch_size=batch_size,
n_init=10, max_no_improvement=10, verbose=0)
t0 = time.time()
mbk.fit(X)
t_mini_batch = time.time() - t0
# #############################################################################
# Plot result
fig = plt.figure(figsize=(8, 3))
fig.subplots_adjust(left=0.02, right=0.98, bottom=0.05, top=0.9)
colors = ['#4EACC5', '#FF9C34', '#4E9A06', '#123456']
# We want to have the same colors for the same cluster from the
# MiniBatchKMeans and the KMeans algorithm. Let's pair the cluster centers per
# closest one.
k_means_cluster_centers = np.sort(k_means.cluster_centers_, axis=0)
mbk_means_cluster_centers = np.sort(mbk.cluster_centers_, axis=0)
k_means_labels = pairwise_distances_argmin(X, k_means_cluster_centers)
mbk_means_labels = pairwise_distances_argmin(X, mbk_means_cluster_centers)
order = pairwise_distances_argmin(k_means_cluster_centers,
mbk_means_cluster_centers)
# KMeans
ax = fig.add_subplot(1, 3, 1)
for k, col in zip(range(n_clusters), colors):
my_members = k_means_labels == k
cluster_center = k_means_cluster_centers[k]
ax.plot(X[my_members, 0], X[my_members, 1], 'w',
markerfacecolor=col, marker='.')
ax.plot(cluster_center[0], cluster_center[1], 'o', markerfacecolor=col,
markeredgecolor='k', markersize=6)
ax.set_title('KMeans')
ax.set_xticks(())
ax.set_yticks(())
plt.text(-3.5, 1.8, 'train time: %.2fs\ninertia: %f' % (
t_batch, k_means.inertia_))
# MiniBatchKMeans
ax = fig.add_subplot(1, 3, 2)
for k, col in zip(range(n_clusters), colors):
my_members = mbk_means_labels == order[k]
cluster_center = mbk_means_cluster_centers[order[k]]
ax.plot(X[my_members, 0], X[my_members, 1], 'w',
markerfacecolor=col, marker='.')
ax.plot(cluster_center[0], cluster_center[1], 'o', markerfacecolor=col,
markeredgecolor='k', markersize=6)
ax.set_title('MiniBatchKMeans')
ax.set_xticks(())
ax.set_yticks(())
plt.text(-3.5, 1.8, 'train time: %.2fs\ninertia: %f' %
(t_mini_batch, mbk.inertia_))
# Initialise the different array to all False
different = (mbk_means_labels == 4)
ax = fig.add_subplot(1, 3, 3)
for k in range(n_clusters):
different += ((k_means_labels == k) != (mbk_means_labels == order[k]))
identic = np.logical_not(different)
ax.plot(X[identic, 0], X[identic, 1], 'w',
markerfacecolor='#bbbbbb', marker='.')
ax.plot(X[different, 0], X[different, 1], 'w',
markerfacecolor='m', marker='.')
ax.set_title('Difference')
ax.set_xticks(())
ax.set_yticks(())
plt.show()
更合适的顺序可能如下所示：
# order cluster centers by their x and y coordinates, weighted by 1 and 0.1 respectively
k_order = np.argsort(k_means.cluster_centers_[:, 0] + k_means.cluster_centers_[:, 1]*0.1)
mbk_order = np.argsort(mbk.cluster_centers_[:, 0] + mbk.cluster_centers_[:, 1]*0.1)
k_means_cluster_centers = k_means.cluster_centers_[k_order]
mbk_means_cluster_centers = mbk.cluster_centers_[mbk_order]
但是，正确的方法是首先对齐聚类中心，然后施加（任意）顺序。这应该可以完成工作：
mbk_order = pairwise_distances_argmin(k_means.cluster_centers_, mbk.cluster_centers_)
k_means_cluster_centers = k_means.cluster_centers_
mbk_means_cluster_centers = mbk.cluster_centers_[mbk_order]
我正在阅读 Clustering. They have an example comparing K-Means and MiniBatchKMeans 上的 scikit-learn 用户指南。
我对示例中的以下代码有点困惑：
# We want to have the same colors for the same cluster from the
# MiniBatchKMeans and the KMeans algorithm. Let's pair the cluster centers per
# closest one.
k_means_cluster_centers = np.sort(k_means.cluster_centers_, axis=0)
mbk_means_cluster_centers = np.sort(mbk.cluster_centers_, axis=0)
k_means_labels = pairwise_distances_argmin(X, k_means_cluster_centers)
mbk_means_labels = pairwise_distances_argmin(X, mbk_means_cluster_centers)
order = pairwise_distances_argmin(k_means_cluster_centers,
mbk_means_cluster_centers)
排序前后k-means聚类中心的值为：
k_means.cluster_centers_
array([[ 1.07705469, -1.06730994],
[-1.07159013, -1.00648645],
[ 0.96700708, 1.01837274]])
k_means_cluster_centers
array([[-1.07159013, -1.06730994],
[ 0.96700708, -1.00648645],
[ 1.07705469, 1.01837274]])
有三个中心，所以我假设每一行都是一个中心的 xy 坐标。
我不确定为什么他们在将每个点与最近的中心配对之前使用 np.sort()，因为这会扭曲中心的 x/y 坐标。也许他们只是想按 x 轴或 y 轴排序？
I am not sure why we use np.sort() here.
答案在评论中 - 但是，它的实现方式存在错误，请参见下文。
# We want to have the same colors for the same cluster from the
# MiniBatchKMeans and the KMeans algorithm. Let's pair the cluster centers per
# closest one.
配对在示例代码的下方两行完成：
k_means_cluster_centers = np.sort(k_means.cluster_centers_, axis=0)
mbk_means_cluster_centers = np.sort(mbk.cluster_centers_, axis=0)
(...)
mbk_means_cluster_centers)
order = pairwise_distances_argmin(k_means_cluster_centers,
mbk_means_cluster_centers)
在代码中，order 有效地用作查找 table 以获取 mbk_means_cluster_centers 中对应于 k_means_cluster_centers 的集群。
my_members = mbk_means_labels == order[k]
cluster_center = mbk_means_cluster_centers[order[k]]
It distorts the coordinate of calculated cluster centers.
（根据评论中的讨论更新）
的确，通过使用np.sort(..., axis=0)，中心坐标搞混了。正确的排序方式是使用 np.lexsort，像这样
arr = k_means.cluster_centers_
k_means_cluster_centers = arr[np.lexsort((arr[:, 0], arr[:, 1]))]
arr = mbk.cluster_centers_
mbk_means_cluster_center = arr[np.lexsort((arr[:, 0], arr[:, 1]))]
实际上这会改变示例的结果：
使用sort(..., axis=0)
使用np.lexsort
我认为你是对的。像本例中那样进行排序会混淆点的 x 和 y 坐标。它在示例中起作用的事实或多或少是巧合。
我们有 x 坐标 [1, -1, 1] 和 y 坐标 [1, -1, -1]。排序后它们变成 [-1, 1, 1] 和 [-1, -1, 1]，它们构成了我们最初的三对：
# original | sorted
# [ 1, -1] | [-1, -1]
# [-1, -1] | [ 1, -1]
# [ 1, 1] | [ 1, 1]
在下面观察使用四个集群时这是如何分解的。在这种情况下，我们有：
# original | sorted
# [-1, -1] | [-1, -1]
# [-1, 1] | [-1, -1]
# [ 1, -1] | [ 1, 1]
# [ 1, 1] | [ 1, 1]
不相同的点。
修改后的示例代码：
print(__doc__)
import time
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import MiniBatchKMeans, KMeans
from sklearn.metrics.pairwise import pairwise_distances_argmin
from sklearn.datasets.samples_generator import make_blobs
# #############################################################################
# Generate sample data
np.random.seed(0)
batch_size = 45
centers = [[1, 1], [-1, -1], [1, -1], [-1, 1]]
n_clusters = len(centers)
X, labels_true = make_blobs(n_samples=3000, centers=centers, cluster_std=0.7)
# #############################################################################
# Compute clustering with Means
k_means = KMeans(init='k-means++', n_clusters=4, n_init=10)
t0 = time.time()
k_means.fit(X)
t_batch = time.time() - t0
# #############################################################################
# Compute clustering with MiniBatchKMeans
mbk = MiniBatchKMeans(init='k-means++', n_clusters=4, batch_size=batch_size,
n_init=10, max_no_improvement=10, verbose=0)
t0 = time.time()
mbk.fit(X)
t_mini_batch = time.time() - t0
# #############################################################################
# Plot result
fig = plt.figure(figsize=(8, 3))
fig.subplots_adjust(left=0.02, right=0.98, bottom=0.05, top=0.9)
colors = ['#4EACC5', '#FF9C34', '#4E9A06', '#123456']
# We want to have the same colors for the same cluster from the
# MiniBatchKMeans and the KMeans algorithm. Let's pair the cluster centers per
# closest one.
k_means_cluster_centers = np.sort(k_means.cluster_centers_, axis=0)
mbk_means_cluster_centers = np.sort(mbk.cluster_centers_, axis=0)
k_means_labels = pairwise_distances_argmin(X, k_means_cluster_centers)
mbk_means_labels = pairwise_distances_argmin(X, mbk_means_cluster_centers)
order = pairwise_distances_argmin(k_means_cluster_centers,
mbk_means_cluster_centers)
# KMeans
ax = fig.add_subplot(1, 3, 1)
for k, col in zip(range(n_clusters), colors):
my_members = k_means_labels == k
cluster_center = k_means_cluster_centers[k]
ax.plot(X[my_members, 0], X[my_members, 1], 'w',
markerfacecolor=col, marker='.')
ax.plot(cluster_center[0], cluster_center[1], 'o', markerfacecolor=col,
markeredgecolor='k', markersize=6)
ax.set_title('KMeans')
ax.set_xticks(())
ax.set_yticks(())
plt.text(-3.5, 1.8, 'train time: %.2fs\ninertia: %f' % (
t_batch, k_means.inertia_))
# MiniBatchKMeans
ax = fig.add_subplot(1, 3, 2)
for k, col in zip(range(n_clusters), colors):
my_members = mbk_means_labels == order[k]
cluster_center = mbk_means_cluster_centers[order[k]]
ax.plot(X[my_members, 0], X[my_members, 1], 'w',
markerfacecolor=col, marker='.')
ax.plot(cluster_center[0], cluster_center[1], 'o', markerfacecolor=col,
markeredgecolor='k', markersize=6)
ax.set_title('MiniBatchKMeans')
ax.set_xticks(())
ax.set_yticks(())
plt.text(-3.5, 1.8, 'train time: %.2fs\ninertia: %f' %
(t_mini_batch, mbk.inertia_))
# Initialise the different array to all False
different = (mbk_means_labels == 4)
ax = fig.add_subplot(1, 3, 3)
for k in range(n_clusters):
different += ((k_means_labels == k) != (mbk_means_labels == order[k]))
identic = np.logical_not(different)
ax.plot(X[identic, 0], X[identic, 1], 'w',
markerfacecolor='#bbbbbb', marker='.')
ax.plot(X[different, 0], X[different, 1], 'w',
markerfacecolor='m', marker='.')
ax.set_title('Difference')
ax.set_xticks(())
ax.set_yticks(())
plt.show()
更合适的顺序可能如下所示：
# order cluster centers by their x and y coordinates, weighted by 1 and 0.1 respectively
k_order = np.argsort(k_means.cluster_centers_[:, 0] + k_means.cluster_centers_[:, 1]*0.1)
mbk_order = np.argsort(mbk.cluster_centers_[:, 0] + mbk.cluster_centers_[:, 1]*0.1)
k_means_cluster_centers = k_means.cluster_centers_[k_order]
mbk_means_cluster_centers = mbk.cluster_centers_[mbk_order]
但是，正确的方法是首先对齐聚类中心，然后施加（任意）顺序。这应该可以完成工作：
mbk_order = pairwise_distances_argmin(k_means.cluster_centers_, mbk.cluster_centers_)
k_means_cluster_centers = k_means.cluster_centers_
mbk_means_cluster_centers = mbk.cluster_centers_[mbk_order]