GridSearch在Keras + TensorFlow导致资源枯竭

Question

我知道这个错误是经常性的，我明白什么能导致它。 例如，运行这个模型的150×150 163个图像给我的错误（但它不是很清楚，我为什么设置的batch_size Keras似乎仍然试图在同一时间分配所有图像的GPU）：

model = Sequential() 
    model.add(Conv2D(64, kernel_size=(6, 6), activation='relu', input_shape=input_shape, padding='same', name='b1_conv')) 
    model.add(MaxPooling2D(pool_size=(2, 2), name='b1_poll')) 
    model.add(Conv2D(128, kernel_size=(6, 6), activation='relu', padding='same', name='b2_conv')) 
    model.add(MaxPooling2D(pool_size=(2, 2), name='b2_pool')) 
    model.add(Conv2D(256, kernel_size=(6, 6), activation='relu', padding='same', name='b3_conv')) 
    model.add(MaxPooling2D(pool_size=(2, 2), name='b3_pool')) 
    model.add(Flatten()) 
    model.add(Dense(500, activation='relu', name='fc1')) 
    model.add(Dropout(0.5)) 
    model.add(Dense(500, activation='relu', name='fc2')) 
    model.add(Dropout(0.5)) 
    model.add(Dense(n_targets, activation='softmax', name='prediction')) 
    model.compile(optimizer=optim, loss='categorical_crossentropy', metrics=['accuracy']) 

鉴于此，我将图像大小缩小至30x30（导致精度下降，如预期的那样）。但是，在此模型中运行网格搜索资源耗尽。

model = KerasClassifier(build_fn=create_model, verbose=0) 

# grid initial weight, batch size and optimizer 
sgd = optimizers.SGD(lr=0.0005) 
rms = optimizers.RMSprop(lr=0.0005) 
adag = optimizers.Adagrad(lr=0.0005) 
adad = optimizers.Adadelta(lr=0.0005) 
adam = optimizers.Adam(lr=0.0005) 
adamm = optimizers.Adamax(lr=0.0005) 
nadam = optimizers.Nadam(lr=0.0005) 

optimizers = [sgd, rms, adag, adad, adam, adamm, nadam] 
init = ['glorot_uniform', 'normal', 'uniform', 'he_normal'] 
batches = [32, 64, 128] 
param_grid = dict(optim=optimizers, batch_size=batches, init=init) 
grid = GridSearchCV(estimator=model, param_grid=param_grid) 
grid_result = grid.fit(X_train, y_train) 

# summarize results 
print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_)) 

我不知道是否有可能通过网格搜索中使用的每种组合之前“干净”的东西（不知道我说清楚了，这是所有新的给我）。

编辑

使用fit_generator也给了我同样的错误：

def generator(features, labels, batch_size): 
    # Create empty arrays to contain batch of features and labels# 
    batch_features = np.zeros((batch_size, size, size, 1)) 
    batch_labels = np.zeros((batch_size, n_targets)) 
    while True: 
     for i in range(batch_size): 
      # choose random index in features 
      index = np.random.choice(len(features),1) 
      batch_features[i] = features[index] 
      batch_labels[i] = labels[index] 
     yield batch_features, batch_labels 

sgd = optimizers.SGD(lr=0.0005) 
rms = optimizers.RMSprop(lr=0.0005) 
adag = optimizers.Adagrad(lr=0.0005) 
adad = optimizers.Adadelta(lr=0.0005) 
adam = optimizers.Adam(lr=0.0005) 
adamm = optimizers.Adamax(lr=0.0005) 
nadam = optimizers.Nadam(lr=0.0005) 

optim = [rms, adag, adad, adam, adamm, nadam] 
init = ['normal', 'uniform', 'he_normal'] 

combinations = [(a, b) for a in optim for b in init] 
for combination in combinations: 
    init = combination[1] 
    optim = combination[0] 
    model = create_model(init=init, optim=optim) 
    model.fit_generator(generator(X_train, y_train, batch_size=32), 
         steps_per_epoch=X_train.shape[0] // 32, 
         epochs=100, verbose=0, validation_data=(X_test, y_test)) 
    scores = model.model.evaluate(X_test, y_test, verbose=0) 
    print("%s: %.2f%% Model %s %s" % (model.model.metrics_names[1], scores[1]*100, optim, init)) 

Answer 1

你应该发电机+ yield工作，他们从内存丢弃他们已经使用的数据。看看我的answer到类似的问题。

Answer 2

您有K作为tensorflow后端培训/评估

K.clear_session() 

的每次运行后清除tensorflow会议。