将 Tensorflow 输入管道更改为 "Datasets" - 错误

Question

直到现在我在 tensorflow 中使用这样的管道：

queue_filenames = tf.train.string_input_producer(data)
reader = tf.FixedLengthRecordReader(record_bytes=4*4)

class Record(object):
    pass
result = Record()
result.ley, value = reader.read(queue_filenames)
record = tf.decode_raw(value, tf.float32)
image = tf.reshape(tf.strided_slice(record,[0],[1]),[1])
label = tf.reshape(tf.strided_slice(record,[1],[4]),[3])

x, y = tf.train.shuffle_batch([image, label],
                              batch_size=batch_size,
                              capacity=batch_size*3,
                              min_after_dequeue=batch_size*2)

但现在我想更改为 "dataset"-thing。我写了这个：

dataset = tf.data.FixedLengthRecordDataset(filenames=data,
                                           record_bytes=4*4)
dataset.map(_generate_x_y)
dataset.shuffle(buffer_size=batch_size*2)
dataset.batch(batch_size=batch_size)
dataset.repeat()
iterator = dataset.make_one_shot_iterator()
x, y = iterator.get_next()

与：

def _generate_x_y(sample):
    features = {"x": tf.FixedLenFeature([1], tf.float32),
                "y": tf.FixedLenFeature([3], tf.float32)}
    parsed_features = tf.parse_single_example(sample,features)
    return parsed_features["x"], parsed_features["y"]

我的图表是这样的：

y_ = network(x)

和：

loss = tf.losses.softmax_cross_entropy(y,y_)
train_step = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(loss=loss)

我的会话是：

with tf.Session(graph=graph_train) as sess:
    tf.global_variables_initializer().run()
    for i in range(100):
        _, = sess.run([train_step])

它在旧管道上运行良好，但在新数据集上出现以下错误：

File "C:/***/main.py", line 49, in <module>
x, y = iterator.get_next()
  File "C:\***\python\framework\ops.py", line 396, in __iter__
"`Tensor` objects are not iterable when eager execution is not "
TypeError: `Tensor` objects are not iterable when eager execution is not enabled. To iterate over this tensor use `tf.map_fn`.

感谢您的帮助:-)

Answer 1

一个可能导致问题的明显问题是您没有使用转换后的数据集。基本上，而不是

dataset = tf.data.FixedLengthRecordDataset(filenames=data,
                                           record_bytes=4*4)
dataset.map(_generate_x_y)
dataset.shuffle(buffer_size=batch_size*2)

你应该做的：

dataset = tf.data.FixedLengthRecordDataset(filenames=data,
                                           record_bytes=4*4)
dataset = dataset.map(_generate_x_y)
dataset = dataset.shuffle(buffer_size=batch_size*2)

每个数据集操作 returns 一个新的、转换的数据集。原始对象未被 map 和 shuffle.

等操作修改

将 Tensorflow 输入管道更改为 "Datasets" - 错误

Changing Tensorflow Input Pipeline to "Datasets" - Error

pipeline

dataset

tensorflow