当我添加 class 权重时，训练模型会出现 ValueError

Question

我在使用 multi class U-Net 分割时遇到了数据模型训练时的值错误。我的 multi class 模型分为 4 classes.

训练模型代码：

from simple_multi_unet_model import multi_unet_model #Uses softmax 

from tensorflow.keras.utils import normalize
import os
import glob
import cv2
import numpy as np
from matplotlib import pyplot as plt
from sklearn.model_selection import train_test_split
import tensorflow as tf



#Resizing images, if needed
SIZE_X = 128 
SIZE_Y = 128
n_classes=4 #Number of classes for segmentation

#Capture training image info as a list
train_images = []

directory_path = '/home/Documents/Multiclass/images/'
list_of_files  = sorted( filter( os.path.isfile, glob.glob(directory_path + '*.png', recursive=True) ) )

 

for img_path in list_of_files:
        img = cv2.imread(img_path, 0)       
        img = cv2.resize(img, (SIZE_Y, SIZE_X))
        train_images.append(img)
       
#Convert list to array for machine learning processing        
train_images = np.array(train_images)

#Capture mask/label info as a list
train_masks = []
labels_path =  '/home/Documents/Multiclass/labels/'
list_of_labels = sorted( filter( os.path.isfile, glob.glob(labels_path + '*.png', recursive=True) ) )

 
for mask_path in list_of_labels:
        mask = cv2.imread(mask_path, 0)       
        mask = cv2.resize(mask, (SIZE_Y, SIZE_X), interpolation = cv2.INTER_NEAREST)  #Otherwise ground truth changes due to interpolation
        train_masks.append(mask)
        
#Convert list to array for machine learning processing          
train_masks = np.array(train_masks)

###############################################
#Encode labels... but multi dim array so need to flatten, encode and reshape
from sklearn.preprocessing import LabelEncoder
labelencoder = LabelEncoder()
n, h, w = train_masks.shape
train_masks_reshaped = train_masks.reshape(-1,1)
train_masks_reshaped_encoded = labelencoder.fit_transform(train_masks_reshaped)
train_masks_encoded_original_shape = train_masks_reshaped_encoded.reshape(n, h, w)

np.unique(train_masks_encoded_original_shape)

#################################################
train_images = np.expand_dims(train_images, axis=3)
train_images = normalize(train_images, axis=1)
train_masks_input = np.expand_dims(train_masks_encoded_original_shape, axis=3)

#Create a subset of data for quick testing
#Picking 10% for testing and remaining for training
from sklearn.model_selection import train_test_split
x_train, X_test, y_train, y_test = train_test_split(train_images, train_masks_input, test_size = 0.10, random_state = 0)


print("Class values in the dataset are ... ", np.unique(y_train))  # 0 is the background/few unlabeled 

from tensorflow.keras.utils import to_categorical
train_masks_cat = to_categorical(y_train, num_classes=n_classes)
y_train_cat = train_masks_cat.reshape((y_train.shape[0], y_train.shape[1], y_train.shape[2], n_classes))



test_masks_cat = to_categorical(y_test, num_classes=n_classes)
y_test_cat = test_masks_cat.reshape((y_test.shape[0], y_test.shape[1], y_test.shape[2], n_classes))

   
###############################################################
from sklearn.utils import class_weight
class_weights = class_weight.compute_class_weight('balanced',
                                                 np.unique(train_masks_reshaped_encoded),
                                                 train_masks_reshaped_encoded)
                                                

print("Class weights are...:", class_weights)


IMG_HEIGHT = x_train.shape[1]
IMG_WIDTH  = x_train.shape[2]
IMG_CHANNELS = x_train.shape[3]


def get_model():
    return multi_unet_model(n_classes=n_classes, IMG_HEIGHT=IMG_HEIGHT, IMG_WIDTH=IMG_WIDTH, IMG_CHANNELS=IMG_CHANNELS)

model = get_model()
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()

#If starting with pre-trained weights. 
#model.load_weights('???.hdf5')

history = model.fit(x_train, y_train_cat, 
                    batch_size = 16, 
                    verbose=1, 
                    epochs=100, 
                    validation_data=(X_test, y_test_cat), 
                    class_weight=class_weights,
                    shuffle=False)

我使用以下方法来定义 class 权重：

from sklearn.utils import class_weight
class_weights = class_weight.compute_class_weight('balanced',
                                                 np.unique(train_masks_reshaped_encoded),
                                                 train_masks_reshaped_encoded)
                                                

print("Class weights are...:", class_weights)

结果是class_weights : 0.276965 ,13.5112 ,5.80929,6.97915。

我在训练模型时遇到 ValueError。我怎么可能解决它？如果您认为我的方法不可行，请提出使用 class 权重的更好方法。

  File "/home/anaconda3/lib/python3.8/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 1185, in _configure_dataset_and_inferred_steps
    if class_weight:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Answer 1

我对这个错误信息的理解是 numpy 不知道如果 any 元素为真，是否将数组计算为 True，或者作为True 仅当所有元素为真时。因此，返回 ValueError，因为布尔值评估在这方面不明确。

因此，在评估数组时，您应该使用 a.any() 或 a.all()，如所附的错误消息所示。

当您尝试在布尔上下文中评估 class 权重时，错误很可能是从代码中的其他地方发生的（不是共享的？）。

Answer 2

我遇到了同样的问题，但是用这个解决了！！！

你必须把它压缩成字典

试试下面的代码：

从sklearn.utils导入class_weight

class_weights = dict(zip(np.unique(train_masks_reshaped_encoded), class_weight.compute_class_weight('balanced', np.unique(train_masks_reshaped_encoded), train_masks_reshaped_encoded)))

当我添加 class 权重时，训练模型会出现 ValueError

Training the model gives ValueError when I add class weights

python

image-segmentation

conv-neural-network

keras

tensorflow