Adapting Deep Neutral Networks in Non-stationary Learning Environments

Abstract

Systems that interact with environments for extended periods of time, e.g. autonomous agents, can often observe non-stationary streams of data. This thesis proposes several methods that adapt, i.e. change the structure, of a single deep network in response to changes in the distribution of data in non-stationary settings, while alleviating catastrophic forgetting. Current approaches for non-stationary learning rely on retraining the model frequently on past data or using an ensemble of models which scale poorly for long streams of data. In this thesis, first, a fully connected network that adapts its structure (RA-DAE) is presented. RA-DAE incrementally adapts a subset of its layers over time based on the changes in the training data distribution. These adaptations involve adding or removing a small number of neurons. The adaptations are chosen by a reinforcement learning agent trying to maximise the accuracy while preferring a low model complexity. Second, this approach is extended to convolution networks (AdaCNN) in a more scalable manner. Experiments show that adaptive models show better performance with less parameters, in non-stationary environments. Next, RA-DAE is used to navigate a robot in a self supervised manner. More precisely, the robot is put in an environment without any prior knowledge of the environment. Then data accumulated via various interactions of the robot (e.g. collisions) are used to train the RA-DAE algorithm in real-time. The experiments indicate that RA-DAE is trained faster and reduces the number of collisions over time quicker compared to its fixed counterparts. Finally, AdaCNN is used to navigate a robot in changing environments, which is common for robots operating for longer periods of time. AdaCNN enables the robot to operate without needing to frequently retrain the model with past data. The experiments indicate that AdaCNN shows a stable increase in the accuracy while navigating when new environments are introduced over time.