Tuesday, November 17, 2026
HomeData ScienceContinual Learning – Catastrophic Forgetting, EWC, Progressive Networks and Replay Methods

Continual Learning – Catastrophic Forgetting, EWC, Progressive Networks and Replay Methods

Table of Content

📋 KEY INSIGHTS

  • Continual learning (CL) — also called lifelong learning or incremental learning — studies how neural networks can learn a sequence of tasks T_1, T_2, …, T_n without forgetting previously learned knowledge when trained on a new task. The central challenge is catastrophic forgetting (McCloskey & Cohen, 1989): the standard gradient descent update for task T_k overwrites the weight configurations that encoded task T_{k-1}, because the gradient step is computed only on T_k data with no memory of or regularisation toward the previous task. Catastrophic forgetting is not a theoretical curiosity — it is a fundamental obstacle to deploying ML systems in environments where data distributions shift over time, new classes appear, and old tasks must remain solvable.
  • The three canonical CL scenarios, defined by task identity information at test time: (1) Task-Incremental Learning (Task-IL) — the task identity is provided at test time, so the model uses the correct task-specific head; (2) Domain-Incremental Learning (Domain-IL) — the task identity is not provided but the output space is fixed (same classes, shifting input distributions, e.g. different lighting conditions for image classification); (3) Class-Incremental Learning (Class-IL) — the model must classify all classes seen so far without knowing the task ID, the hardest scenario because the model must both learn new classes and discriminate them from all old classes. Most CL benchmarks (Split-CIFAR10, Permuted-MNIST, Split-CORe50) define variants of these three scenarios.
  • Regularisation-based CL methods add penalty terms to the loss that protect important weights for old tasks from large updates during new task training. Elastic Weight Consolidation (EWC, Kirkpatrick et al., 2017) uses the Fisher information matrix to identify weights that were important for old tasks (high Fisher information = high sensitivity of old-task performance to weight changes) and penalises changes to those weights: L_EWC = L_{new} + (λ/2)·Σ_i F_i·(θ_i − θ*_i)^2, where F_i is the Fisher information for weight i, θ*_i is the weight value after the old task, and λ controls the strength of the consolidation. Online EWC and Synaptic Intelligence (SI) extend EWC with cheaper online computation of importance weights. The connection to our regularisation guide is direct: EWC is a parameter-space regularisation method, using the Fisher information as a data-dependent regularisation mask.
  • Replay-based CL methods combat forgetting by mixing old-task examples into the current training batch, ensuring the network continuously practices on old tasks alongside new ones. Experience Replay stores a small episodic memory of old examples and replays them during training. Gradient Episodic Memory (GEM) uses the stored examples as constraints: the new-task gradient update is projected onto a direction that does not increase the loss on stored old-task examples. Dark Experience Replay (DER++) stores the softmax logits (not just labels) of old examples at training time, then uses a knowledge distillation loss to match the current model’s logits to the stored logits — preserving the full output distribution, not just the class label. The connection to our model compression guide is direct: DER++ uses knowledge distillation (a model compression technique) as a replay mechanism, exploiting the rich information in soft probability distributions over all classes.
  • Architecture-based CL methods avoid forgetting by dynamically expanding the network or by allocating separate sub-networks (masks) for each task. Progressive Neural Networks (Rusu et al., 2016) add a new network column for each task while freezing all previous columns, allowing the new column to access representations from previous columns via lateral connections — zero forgetting by construction, but memory grows linearly with number of tasks. PackNet (Mallya & Lazebnik, 2018) uses iterative pruning to free up network capacity: after training task k, prune the least important weights (using Taylor expansion importance scores), retrain the remaining weights, then lock the pruned weights for future tasks — the network accumulates tasks by filling in freed-up weight slots. GDumb (Prabhu et al., 2020) is a surprisingly strong baseline that simply stores all seen examples in a fixed-size memory buffer (using reservoir sampling) and trains a fresh model from scratch on the buffer before each evaluation — beating many sophisticated CL methods on Class-IL benchmarks.
  • Evaluation metrics for CL: Average Accuracy (AA) = mean accuracy across all tasks after training the full sequence; Backward Transfer (BWT) = how much accuracy on old tasks changes after learning new tasks (negative BWT = forgetting, positive BWT = improvement via generalisation); Forward Transfer (FWT) = how much learning old tasks improves zero-shot accuracy on new tasks (measuring positive transfer). The joint upper bound trains on all tasks simultaneously (the ideal but impractical scenario with unlimited access to all historical data). The connection to our transfer learning guide is direct: forward transfer in CL is the same phenomenon as transfer learning — representations learned on old tasks generalise to new tasks, reducing the data needed to achieve good performance on them.

Neural networks are conventionally trained on a fixed dataset until convergence, then deployed — a static model for a static world. Real-world ML systems, however, face non-stationary environments: new products arrive (requiring new recommendation classes), new medical conditions are identified (requiring new diagnostic categories), and new types of cyber attacks emerge (requiring updated intrusion detection). Retraining from scratch on all historical plus new data is expensive and often impractical when historical data is unavailable due to privacy regulations (GDPR), storage costs, or proprietary data agreements. Continual learning — training neural networks to accumulate knowledge across a sequence of tasks without catastrophic forgetting — is the technical foundation for ML systems that evolve over time without periodic full retraining. This guide covers the full CL toolkit: catastrophic forgetting and its causes, the three canonical CL scenarios, regularisation-based methods (EWC, SI, MAS), replay-based methods (ER, GEM, DER++), architecture-based methods (Progressive Nets, PackNet), and GDumb as a surprisingly effective baseline, with implementation guidance using the Avalanche CL framework.

Catastrophic Forgetting — Causes and Regularisation-Based Solutions

Catastrophic forgetting occurs because stochastic gradient descent has no memory: the loss landscape for task T_k is defined solely by task T_k data, and the gradient of L_{T_k} is orthogonal to the gradient of L_{T_{k-1}} for most weight configurations. After SGD converges on T_k, the weights may have moved far from the region of weight space that was optimal for T_{k-1}. The severity of forgetting scales with the overlap between the representations used for different tasks — if tasks T_1 and T_2 use the same features (high task interference), learning T_2 necessarily disrupts the representations for T_1. EWC addresses this by identifying, for each weight, how sensitive old-task performance is to that weight’s value (the Fisher information), then penalising gradient steps that move high-importance weights away from their old-task optimal values. The Fisher information F_i = E_{x~p(x|T_{old})}[(∂ log p(y|x,θ)/∂θ_i)^2] measures the expected squared gradient of the log-likelihood with respect to weight i, computed over the old task data distribution. High F_i means that weight i strongly determines the model’s predictions on old-task data — small perturbations to it cause large changes in old-task accuracy. EWC’s quadratic penalty with Fisher-weighted coefficients is equivalent to assuming a Gaussian posterior over weights, with the Fisher as the precision (inverse variance) — a Laplace approximation to Bayesian continual learning. The connection to our Bayesian statistics guide is direct: Bayesian CL frames the problem as online Bayesian inference, where the posterior over weights after task k becomes the prior for task k+1, and EWC approximates this posterior with a Gaussian whose precision is the Fisher information. Online EWC uses a running average of the Fisher across all previous tasks, avoiding the need to store a separate Fisher matrix per task. Synaptic Intelligence (SI) computes importance online during training (tracking how much each weight contributed to reducing the loss) rather than via a Fisher matrix computation at the end of each task, making it computationally cheaper and applicable to online learning scenarios.

MethodCategoryMemoryForgetting PreventionClass-IL Strength
EWC / Online EWCRegularisationO(|θ|) for FisherWeight importance penaltyModerate
SI (Synaptic Intelligence)RegularisationO(|θ|) running sumOnline importance trackingModerate
Experience Replay (ER)ReplayFixed memory bufferOld example rehearsalGood
DER++Replay + DistillationBuffer + logit storageLogit matching via KDVery good
Progressive NetsArchitectureGrows with tasksColumn freezingPerfect (by design)
PackNetArchitecture (pruning)Fixed networkWeight locking after pruningGood
GDumbReplay (from scratch)Fixed memory bufferRetrain on bufferSurprisingly strong

Replay-Based Methods, Architecture Approaches and the Avalanche Framework

high-angle photography of mountain covered with snow
Photo by Lucian on Unsplash

Replay-based CL methods are empirically the most effective category for Class-IL, the hardest scenario where task identity is unknown at test time. The core challenge for replay: how to maintain a small memory buffer (typically 200–2000 examples total across all old tasks) that maximally preserves old-task performance. Reservoir sampling maintains a uniform random sample from the stream of all seen examples: for each new example, include it in the buffer with probability m/(t+1) where m is the buffer size and t is the total number of examples seen — ensuring that after n examples the buffer holds a uniform random sample of all n. Ring buffer maintains a fixed number of examples per class (exemplar set), selected to maximally cover the class distribution (herding selection: greedily selecting examples whose mean feature embedding most closely matches the class centroid). iCaRL (Rebuffi et al., 2017) combines herding exemplar selection with knowledge distillation on the stored exemplars during new task training, performing new-class classification via nearest-mean-of-exemplars — a non-parametric approach that naturally avoids the bias toward new classes that parametric softmax classifiers develop. The connection to our active learning guide is direct: exemplar selection in CL is structurally equivalent to coreset selection in active learning — both seek a small subset of labelled examples that maximally represents the full distribution. DER++ (Buzzega et al., NeurIPS 2020) achieves a qualitative improvement over standard ER by storing not just (x, y) but (x, y, z) where z = f_θ(x) is the model’s full softmax logit vector at the time of storing the example. The DER++ loss adds a knowledge distillation term: ||f_θ(x_buf) − z_buf||^2, forcing the current model to reproduce its own past predictions on buffered examples. This soft-label matching is richer than hard-label replay because z encodes the full uncertainty structure — a soft prediction of 0.7/0.2/0.1 over three classes tells the network more about the decision boundary geometry than the hard label 0 alone. The connection to our model compression guide is direct: DER++ applies knowledge distillation (normally used to compress a large teacher into a small student) temporally — the past self is the teacher, the current self is the student. The Avalanche library (Lomonaco et al., 2021) is the standard Python CL framework: it provides benchmark datasets (Split-CIFAR100, CORe50, CUB-200), continual learning strategies as composable PyTorch training plugins, and evaluation protocols computing AA, BWT, and FWT automatically. The connection to our MLflow guide is direct: Avalanche integrates with MLflow for experiment tracking of CL training runs, logging per-task accuracy matrices and forgetting metrics alongside standard model metrics.

✦ SUMMARIZE THIS ARTICLE WITH AI

Regularisation techniques (EWC as parameter-space regularisation) are in our Regularisation guide. Knowledge distillation (used in DER++ and iCaRL) is in our Model Compression guide. Transfer learning (forward transfer in CL) is in our Transfer Learning guide. Active learning (coreset selection = exemplar selection) is in our Active Learning guide. Bayesian statistics (Bayesian CL and EWC’s Laplace approximation) are in our Bayesian Statistics guide. MLflow (experiment tracking for CL runs) is in our MLflow guide. Self-supervised learning (pre-training for stronger CL representations) is in our Self-Supervised Learning guide.

Leave feedback about this

  • Rating

Durgesh Kekare
Durgesh Kekarehttps://www.dataexpertise.in
Durgesh Kekare is a data science educator and founder of DataExpertise.in. With expertise in Python, machine learning, and analytics, he helps 10,000+ learners break into data careers.

Latest Posts

List of Categories