Callbacks in Python

Training a neural network for a fixed number of epochs is a gamble. Set epochs too low and the model never converges; set it too high and, past some point, it starts memorizing the training set instead of generalizing — the training loss keeps falling while the validation loss creeps back up. If you commit to a hard-coded epochs=1000 and walk away, you have no way to react to that inflection point; you either babysit the terminal output or burn hundreds of epochs of GPU time training a model that is actively getting worse on real data.

Keras solves this with callbacks: objects that hook into the training loop and run automatically at defined points — after a batch, after an epoch, at the start or end of training — without you having to rewrite model.fit() yourself. Instead of guessing a fixed epoch count, you delegate the decision of when to stop, when to save, and when to adjust the learning rate to code that inspects the actual metrics as they arrive.

This reframes epochs=1000 not as a target to hit but as a safety ceiling: the callback, not the epoch counter, decides when training has actually finished.

A Callback in Keras is a plain object passed to the callbacks argument of model.fit() that Keras invokes at specific points in the training loop — most commonly on_epoch_end, but also on_batch_end, on_train_begin, and on_train_end. Each call gives the callback access to the current logs dictionary (loss, val_loss, accuracy, and any other tracked metrics), so it can inspect training progress in real time.

The import shown here, from tensorflow.keras.callbacks import EarlyStopping, pulls in one of several built-in callbacks Keras ships with. You don't have to write the monitoring logic yourself — EarlyStopping, ModelCheckpoint, ReduceLROnPlateau, and others already implement the common patterns (stop training, save weights, adjust the learning rate) based on a metric you name.

Because callbacks run automatically inside the fit() loop, they require no changes to your model architecture or training data — you attach behavior to the training process itself, orthogonal to what the model actually computes.

A callback is not a network call and it isn't part of the mathematics of training — it doesn't compute gradients, and it doesn't talk to any external server. It is a Python object with hook methods (on_epoch_end, on_batch_begin, and so on) that Keras calls automatically as training proceeds, purely to observe and react to what's happening.

That distinction matters because it tells you what a callback can and can't do. It can read the current metrics and issue commands like 'stop training' or 'save the model to disk,' but it never participates in the forward or backward pass — the loss computation and backpropagation happen exactly the same whether or not you attach any callbacks at all.

In other words, callbacks are a monitoring and control layer bolted onto model.fit(), not a modification to the model's math. That's why you can add, remove, or swap callbacks freely without changing what the model learns — only when and whether training continues.

EarlyStopping is the callback most people reach for first because it directly attacks the overfitting problem from the start of this lesson. You give it a metric to watch — typically val_loss, since validation loss is what tells you whether the model generalizes rather than just memorizes — and after every epoch it compares the new value to the best one seen so far.

In early_stop = EarlyStopping(monitor="val_loss", patience=5), the callback isn't watching training loss, which almost always keeps decreasing; it's watching the held-out validation set, which is what reveals overfitting the moment the model starts fitting noise in the training data instead of the underlying pattern.

Once that callback is passed into model.fit(X, y, epochs=1000, callbacks=[early_stop]), epochs=1000 stops being a promise and becomes a ceiling — training stops the moment EarlyStopping decides val_loss is no longer improving, however many epochs that takes.

patience=5 doesn't mean training stops after 5 epochs — it means EarlyStopping tolerates 5 consecutive epochs of no improvement in val_loss before it gives up. Validation loss rarely improves in a perfectly smooth line; it dips, plateaus, and sometimes ticks up for a few epochs before finding a new low. A patience of 0 would abort training the instant it hit any such bump, often far too early.

Setting patience higher gives the model room to push through those temporary plateaus and keep looking for a better minimum, at the cost of running a few more (possibly wasted) epochs before stopping. Setting it too low risks stopping right before a breakthrough; setting it too high defeats the purpose of early stopping in the first place, since you end up training deep into the overfitting region anyway.

Choosing a good patience value is a practical trade-off: small datasets with noisy validation metrics usually need a larger patience, while large, stable datasets can often get away with patience=2 or 3.

Restore Best Weights is the piece of the EarlyStopping puzzle that closes a gap patience alone leaves open: knowing when to stop is only half the problem, because by the time patience epochs have passed without improvement, the model's weights in memory belong to the epoch training stopped at — not the epoch that actually performed best.

This matters for the same reason patience exists: validation loss doesn't move in a straight line, so the last epoch before stopping is very rarely the best one. Without a mechanism to reach back and recover those earlier, better weights, EarlyStopping would correctly detect overfitting and still hand you an overfitted model.

The fix lives in a single argument on the same EarlyStopping callback, and understanding exactly which epoch's weights end up in your model — and why — is the difference between a model that quietly ships an overfitted checkpoint and one that ships its best-performing version.

Walk through the numbers: if the best val_loss was recorded at epoch 50 and patience=5, EarlyStopping keeps training through epochs 51, 52, 53, 54, and 55 waiting to see if val_loss improves again. When it doesn't, training halts at epoch 55 — but the model object in memory has already continued updating its weights through all five of those extra epochs.

That means the weights sitting in the model right after fit() returns are the epoch-55 weights, not the epoch-50 weights that actually had the lowest validation loss. Those five extra epochs happened precisely because the model was no longer improving, which is often exactly when it starts drifting into overfitting territory.

So by default, EarlyStopping's own patience mechanism guarantees you finish training with weights that are, at best, no better than your peak — and at worst, measurably more overfit than the checkpoint you actually wanted.

The fix is a single boolean: EarlyStopping(monitor="val_loss", patience=5, restore_best_weights=True). With this flag set, Keras tracks the best-performing weights internally throughout training and, the moment it decides to stop, rolls the model back to that snapshot before returning control to you — so the epoch-55 example resolves to the epoch-50 weights automatically.

Without restore_best_weights=True (its default is False), you'd need to solve the same problem manually with a separate ModelCheckpoint(save_best_only=True) callback, saving the best model to disk on every improvement and reloading it after training finishes. That works, but it's an extra step people forget, and it's strictly more moving parts than a single keyword argument.

The practical rule: whenever you use EarlyStopping, pass restore_best_weights=True in the same call. There's essentially no scenario where you want to keep the overfitted weights from the final, non-improving epochs instead of your model's actual best epoch.

Put together, EarlyStopping(monitor="val_loss", patience=5, restore_best_weights=True) plus a ModelCheckpoint(save_best_only=True) callback gives you a training loop that stops itself at the right time, keeps the best-performing weights in memory, and persists that same best model to disk — with no manual monitoring of the training logs required.

These two callbacks are usually the first pair to reach for in any real training run: EarlyStopping protects your GPU time and prevents shipping an overfitted model, while ModelCheckpoint protects you against crashes, interruptions, and simply losing track of which saved file was actually the best one.

From here, the same callback mechanism extends to other automated behaviors during training — logging metrics, adjusting the learning rate on a plateau, or writing custom callbacks of your own — all built on the same on_epoch_end hook you've just seen EarlyStopping and ModelCheckpoint use.

What happens if you tell Keras to train for 1000 epochs, but the model starts overfitting at epoch 50? Do you just waste 950 epochs of GPU time?

To solve this, Keras uses "Callbacks". These are functions that execute automatically at the end of every single epoch to check on the model.

What is a "Callback" in the context of Keras model.fit()?

• →A network request to Google servers.
• →A utility function that runs automatically at specific points during training (like the end of an epoch) to monitor metrics and take automated actions.
• →The function that calculates the gradients.

The most powerful callback is EarlyStopping. It watches the val_loss. If the validation loss stops improving, it aborts the training instantly.

In EarlyStopping(monitor="val_loss", patience=5), what does patience=5 mean?

• →It waits 5 minutes before starting.
• →It allows the model to train for 5 more epochs after the val_loss stops improving, just in case it's a temporary plateau, before finally killing the training loop.
• →It stops training exactly at epoch 5.

Another critical callback is ModelCheckpoint. Training a massive model can take days. What if the server crashes? Checkpoint saves the weights to disk periodically.

Why should you always pass save_best_only=True to the ModelCheckpoint callback?

• →To save RAM on the GPU.
• →Because you only want to save the model to the hard drive when it actually improves on the validation set, ensuring you don't overwrite a great model with an overfitted one.
• →It is required to use EarlyStopping.

Now, prepare yourself. We are about to enter the ADA Defense Protocol. Ensure you understand Restore Best Weights.

If EarlyStopping triggers at epoch 55 (because patience=5, meaning the best epoch was 50), the model in memory currently holds the overfitted weights of epoch 55.

ADA DEFENSE: When using EarlyStopping with a patience of 5, the training stops at epoch 55. However, the best weights were found at epoch 50. How do you ensure your final model actually contains the weights from epoch 50?

• →It happens automatically; Keras always reverts to the best epoch.
• →You must pass restore_best_weights=True to the EarlyStopping callback, otherwise Keras will leave the corrupted/overfitted weights from epoch 55 in memory.
• →You have to train the model a second time for exactly 50 epochs.

Threat neutralized. Model integrity secured. Module 03 complete.

Simulate Real EarlyStopping. Finish should_stop_early(): count consecutive epochs with no improvement.