Progressive Prompts: Continual Learning for Language Models

We introduce Progressive Prompts - a simple and efficient approach for continual learning in language models. Our method allows forward transfer and resists catastrophic forgetting, without relying on data replay or a large number of task-specific parameters. Progressive Prompts learns a new soft prompt for each task and sequentially concatenates it with the previously learned prompts, while keeping the base model frozen. Experiments on standard continual learning benchmarks show that our approach outperforms state-of-the-art methods, with an improvement >20% in average test accuracy over the previous best-preforming method on T5 model. We also explore a more challenging continual learning setup with longer sequences of tasks and show that Progressive Prompts significantly outperforms prior methods.

Summary Notes

Blog Post: Simplifying Continual Learning with Progressive Prompts for Language Models

Challenges in Continual Learning

Limitations of Traditional Finetuning

• Finetuning is commonly used to adapt language models for new tasks by updating all the model's parameters with new data. This method works well for learning single tasks but struggles in a continual learning setup where a model needs to learn multiple tasks sequentially without forgetting the old ones.

Advantages of Prompt Tuning

• Prompt tuning advances this by training a set of soft prompts (additional input tokens) while keeping the base model's parameters unchanged. It significantly reduces the number of trainable parameters, making the model more adaptable with less effort.

The Continual Learning Challenge

• Achieving effective continual learning, where a model sequentially learns tasks without forgetting previous ones, remains difficult. Existing solutions often involve complex structures or many task-specific parameters, causing inefficiencies and scalability problems.

Progressive Prompts: A Novel Solution

• Sequential Learning of New Prompts: Learning a new prompt for each task without changing the previously learned prompts or the base model parameters.

• Concatenating Learned Prompts: Combining new prompts with those from past tasks prevents interference with previous learning (avoiding catastrophic forgetting) and allows for knowledge reuse (aiding forward transfer).

• Prompt Embedding Reparameterization: A technique using a residual Multi-Layer Perceptron (MLP) that stabilizes the learning process and ensures the addition of new prompts doesn’t negatively impact performance on earlier tasks.

Insights from Experiments

Experiment Setup

• The effectiveness of Progressive Prompts was tested using a variety of text classification tasks (e.g., AG News, Amazon Reviews, Yelp Reviews) and compared against traditional finetuning and other continual learning approaches.

Results

• Progressive Prompts showed a remarkable improvement, with over 20% better average test accuracy on continual learning benchmarks using the T5 model. It was especially effective for long sequences of tasks and tasks with limited training data, showcasing its efficiency and versatility.

Looking Ahead

How Athina AI can help