On Exploring Stability and Plasticity in Continual Learning

Abstract

Continual Learning (CL) is a learning paradigm that leans on a sequence of tasks and has garnered significant attention across various domains, including computer vision, natural language processing, and reinforcement learning. However, when faced with sequential tasks, achieving both high stability to retain learned knowledge from previous tasks and high plasticity to learn new task well becomes challenging, leading to stability and plasticity dilemma. The dilemma stands as a key challenge in continual learning. Therefore, the objective of this thesis is to contribute to the advancement of continual learning by exploring a better balance between stability and plasticity. To accomplish this goal, the thesis explores effective learning approaches from three perspectives: regularization-based, algorithm-based, and replay-based methods. In the context of the regularization-based method, this thesis incorporates a regularization term that considers parameter interactions and explores the eigenvalues of the Hessian matrix. Furthermore, in terms of the algorithm-based method, the thesis modifies the parameter updating rule of algorithm by projecting gradients onto the null space of previous tasks and proposes a distillation technique to enhance performance. Lastly, for the replay-based method, the thesis stores and replays representative historical data, focusing on online continual learning scenarios where data arrives in a single pass.