您如何将测试数据与训练数据分开

Question

大家好，目前正在学习深度学习和机器学习

我也在研究代码的同时阅读了一些github解释

但没有解释他们如何从这段代码中分离测试数据（底部有评论的部分# declare data for training and validation, if you want, you can seperate testset from this

# 1. Creating Datasets
# define temporary empty list for load
data = []
label = []
Totalnb = 0

# Load Dataset
for i in range(n_labels):
    nb = 0
    # Counting datasets in each labels
    for root, dirs, files in os.walk('Progress/DataLatihBaru/' + str(i+1)): # set directory
        for name in dirs:
            nb = nb + 1
    print(i,"Label number of Dataset is:",nb)
    Totalnb = Totalnb + nb
    # by Counting size, cross subfolder and read image data, resize image, and append list 
    for j in range(nb):
        temp = []
        for k in range(timesteps):
            # name = 'NormalizedCascaded/' + str(i+1) + '/' + str(j+1) + '/' + str(k+1) + '.jpg'
            name = 'Progress/DataLatihBaru/' + str(i+1) + '/' + str(j+1) + '/' + 'a (' + str(k+1) + ')' + '.jpg'
            img = cv2.imread(name)
            res = cv2.resize(img, dsize=(img_col, img_row), interpolation=cv2.INTER_CUBIC)
            temp.append(res)
        label.append(i)        
        data.append(temp)
print("Total Number of Data is",Totalnb)

# Convert List to numpy array, for Keras use
Train_label = np.eye(n_labels)[label] # One-hot encoding by np array function
Train_data = np.array(data)
print("Dataset shape is",Train_data.shape, "(size, timestep, column, row, channel)")
print("Label shape is",Train_label.shape,"(size, label onehot vector)")
# shuffling dataset for input fit function
# if don`t, can`t train model entirely
x = np.arange(Train_label.shape[0])
np.random.shuffle(x)
# same order shuffle is needed
Train_label = Train_label[x]
Train_data = Train_data[x]

# declare data for training and validation, if you want, you can seperate testset from this
X_train=Train_data[0:Totalnb,:]
Y_train=Train_label[0:Totalnb]

任何人都可以帮助我，以便我可以通过一些解释来理解我应该如何将数据分离到测试中的那一部分吗？

非常感谢！

Answer 1

由于 Train_label 和 Train_data 已经打乱，您可以简单地更改训练-测试-拆分的范围。

train_size = 0.8
X_train=Train_data[:int(Totalnb * TRAIN_RATIO),:]
Y_train=Train_label[:int(Totalnb * TRAIN_RATIO)]
X_test=Train_data[int(Totalnb * TRAIN_RATIO):,:]
Y_test=Train_label[int(Totalnb * TRAIN_RATIO):]

或者，使用 sklearn 中的 train_test_split：

from sklearn.model_selection import train_test_split
X_train, X_test, Y_train, Y_test = train_test_split(Train_data, Train_label, test_size=0.2)

您如何将测试数据与训练数据分开

How do you divide your test data from train data

python

numpy

deep-learning

tensorflow