Parametrising Neural Feedback Policies with Stability and Robustness Guarantees

Abstract

Learning-based control is a powerful tool for nonlinear control in complex dynamical systems. Driven by the rise of deep Reinforcement Learning (RL), most approaches parametrise control policies with black-box neural networks, which are universal approximators for nonlinear systems. These can easily be trained in simulation with simple, gradient-based optimisation schemes. However, black-box approaches such as deep RL suffer a fundamental limitation: they lack certifiable guarantees of closed-loop stability, robustness to disturbances, and sensitivity to model error.

This thesis introduces novel parametrisations of neural feedback policies with built-in stability and robustness guarantees. Instead of relying on black-box networks, we parametrise policies with state-of-the-art robust neural networks that automatically satisfy stability and robustness constraints of their own.

Our main contribution combines robust neural networks with a nonlinear version of the Youla-Kucera parametrisation. We propose a theoretically-motivated framework that fuses classical and learning-based control with model information under a single policy architecture. The resulting policy parametrisation: (1) automatically guarantees closed-loop stability and robustness; (2) allows for plug-and-play optimisation with standard gradient-based training pipelines; and (3) is not restrictive in the controllers it covers (for certain classes of systems). We derive rigorous theoretical certificates for our parametrisation in partially-observed, nonlinear systems with incremental stability requirements (contraction and Lipschitzness), and demonstrate its capability for stability-certified RL in numerical experiments.

We extend our study of robust learning-based control with two further contributions: an empirical study of robustness in deep RL with Lipschitz-bounded policy networks; and a new parametrisation of contracting and Lipschitz networks which are scalable to high-dimensional models.