tensorflow隐藏状态似乎没有改变

Question

我试图按照这个RNN tutorial on medium一样，重构它，因为我一直在。当我运行我的代码时，它似乎工作，但是当我试图打印current state变量以查看神经网络内发生了什么时，我得到了全部1 s。那是预期的行为？由于某种原因，州没有更新？据我所知，current state应该包含隐藏层中所有批次的最新值，所以绝对不应该全部为1 s。任何帮助将不胜感激。

def __train_minibatch__(self, batch_num, sess, current_state): 
    """ 
    Trains one minibatch. 

    :type batch_num: int 
    :param batch_num: the current batch number. 

    :type sess: tensorflow Session 
    :param sess: the session during which training occurs. 

    :type current_state: numpy matrix (array of arrays) 
    :param current_state: the current hidden state 

    :type return: (float, numpy matrix) 
    :param return: (the calculated loss for this minibatch, the updated hidden state) 
    """ 
    start_index = batch_num * self.settings.truncate 
    end_index = start_index + self.settings.truncate 

    batch_x = self.x_train_batches[:, start_index:end_index] 
    batch_y = self.y_train_batches[:, start_index:end_index] 
    total_loss, train_step, current_state, predictions_series = sess.run(
     [self.total_loss_fun, self.train_step_fun, self.current_state, self.predictions_series], 
     feed_dict={ 
      self.batch_x_placeholder:batch_x, 
      self.batch_y_placeholder:batch_y, 
      self.hidden_state:current_state 
     }) 
    return total_loss, current_state, predictions_series 
# End of __train_minibatch__() 

def __train_epoch__(self, epoch_num, sess, current_state, loss_list): 
    """ 
    Trains one full epoch. 

    :type epoch_num: int 
    :param epoch_num: the number of the current epoch. 

    :type sess: tensorflow Session 
    :param sess: the session during training occurs. 

    :type current_state: numpy matrix 
    :param current_state: the current hidden state. 

    :type loss_list: list of floats 
    :param loss_list: holds the losses incurred during training. 

    :type return: (float, numpy matrix) 
    :param return: (the latest incurred lost, the latest hidden state) 
    """ 
    self.logger.info("Starting epoch: %d" % (epoch_num)) 

    for batch_num in range(self.num_batches): 
     # Debug log outside of function to reduce number of arguments. 
     self.logger.debug("Training minibatch : ", batch_num, " | ", "epoch : ", epoch_num + 1) 
     total_loss, current_state, predictions_series = self.__train_minibatch__(batch_num, sess, current_state) 
     loss_list.append(total_loss) 
    # End of batch training 

    self.logger.info("Finished epoch: %d | loss: %f" % (epoch_num, total_loss)) 
    return total_loss, current_state, predictions_series 
# End of __train_epoch__() 

def train(self): 
    """ 
    Trains the given model on the given dataset, and saves the losses incurred 
    at the end of each epoch to a plot image. 
    """ 
    self.logger.info("Started training the model.") 
    self.__unstack_variables__() 
    self.__create_functions__() 
    with tf.Session() as sess: 
     sess.run(tf.global_variables_initializer()) 
     loss_list = [] 

     current_state = np.zeros((self.settings.batch_size, self.settings.hidden_size), dtype=float) 
     for epoch_idx in range(1, self.settings.epochs + 1): 
      total_loss, current_state, predictions_series = self.__train_epoch__(epoch_idx, sess, current_state, loss_list) 
      print("Shape: ", current_state.shape, " | Current output: ", current_state) 
      # End of epoch training 

    self.logger.info("Finished training the model. Final loss: %f" % total_loss) 
    self.__plot__(loss_list) 
    self.generate_output() 
# End of train() 

更新

完成second part of the tutorial和使用后，内置RNN API，问题就消失了，这意味着要么有毛病我用我的current_state变量，或改变方式tensorflow API导致了一些古怪的事情发生（虽然我很确定它是前者）。如果有人有明确的答案，请留下问题。

Answer 1

首先，你应该确保“它看起来工作”是真实的，你的测试错误真的变低了。

我的一个假设是，最后一批在最后被零损坏，因为数据total_series_length/batch_size的长度不是truncated_backprop_length的倍数。 （我没有检查它发生了什么，它充满了零。本教程中的代码太旧了，无法在我的tf版本上运行，而且我们也没有你的代码。）这个最后的微型批处理在最后只有零可能导致最后的current_state汇聚到所有的。在任何其他小批量current_state不会是所有的。

您可以尝试每次运行sess.run时在__train_minibatch__中打印current_state。或者可能只是每1000个小批量打印一次。