Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share
menu search
person
Welcome To Ask or Share your Answers For Others

Categories

As the title clearly describes the situation I'm experiencing, despite employing Dropout, MaxPooling, EarlyStopping and Regularizers, my CNN model is still overfitting. Also, I've experimented with various learning_rate, dropout_rate, and L1/L2 regularization weight decay. How can I further prevent overfitting?

Here is the model (using Keras on TensorFlow backend):

batch_size = 128
num_epochs = 200
weight_decay = 1e-3
num_filters = 32 * 2
n_kernel_size = 5
num_classes = 3
activation_fn = 'relu'
nb_units = 128
last_dense_units = 128
n_lr = 0.001
n_momentum = 0.99
n_dr = 0.00001
dropout_rate = 0.8

model.add(Embedding(nb_words, EMBEDDING_DIM, input_length=max_seq_len))
model.add(Dropout(dropout_rate))
model.add(Conv1D(num_filters, n_kernel_size, padding='same', activation=activation_fn,
                 kernel_regularizer=regularizers.l2(weight_decay)))
model.add(MaxPooling1D())
model.add(GlobalMaxPooling1D())
model.add(Dense(128, activation=activation_fn, kernel_regularizer=regularizers.l2(weight_decay)))
model.add(Dropout(dropout_rate))
model.add(Dense(num_classes, activation='softmax'))

adam = Adam(lr=n_lr, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=n_dr)
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['acc'])

early_stopping = EarlyStopping(
    monitor='val_loss',
    patience=3,
    mode='min',
    verbose=1,
    restore_best_weights=True
)

model.fit(...)

Here's the accuracy plots of training and validation: plot

question from:https://stackoverflow.com/questions/65908417/despite-employing-dropout-maxpooling-early-stopping-and-regularizers-my-cnn

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
1.1k views
Welcome To Ask or Share your Answers For Others

1 Answer

There are still overfitting methods to try:

Your model does seem to be overfitting by about 10%. But how much overfitting is too much overfitting? I would look to this post and related conversation so you can best evaluate your specific situation.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
thumb_up_alt 0 like thumb_down_alt 0 dislike
Welcome to ShenZhenJia Knowledge Sharing Community for programmer and developer-Open, Learning and Share

548k questions

547k answers

4 comments

86.3k users

...