A Sequential Optimisation Framework for Adaptive Model Predictive Control in Robotics

Abstract

State-of-the-art control and robotics challenges have long been tackled using model-based control methods like model predictive control (MPC) and reinforcement learning (RL). These methods excel in complex dynamic domains, such as manipulation tasks, but struggle with real-world issues like wear-and-tear, uncalibrated sensors, and misspecifications. These factors often perturb system dynamics, leading to the 'reality gap' problem when robots transition from simulations to real-world environments. This work aims to bridge this gap by combining RL and control in a learning framework that adapts MPC to robot decisions, optimizing performance despite uncertainties in dynamics model parameters.

This thesis presents three key contributions to robotics control. The first is a novel reward-based framework for refining stochastic Model Predictive Control (MPC). It utilizes Bayesian Optimization (BO) for efficient data handling and heteroscedastic noise, linking controller hyperparameters with expected rewards through a Gaussian Process (GP). This approach demonstrates success in simulated control environments and robotic tasks.

The second contribution addresses the 'reality gap' in robotics, enhancing controller performance in real-world dynamics. It builds on the first by developing an adaptive stochastic MPC that optimizes hyperparameters while estimating physical parameter distributions, employing a randomized dynamics model. This method is validated in both simulations and with robotic manipulators.

Finally, the thesis proposes an innovative alternative to BO, merging it with supervised classification for a surrogate-based optimization technique. This method adeptly adjusts control hyperparameters in the face of model uncertainty and noise, optimizing complex functions through a binary classifier. Tested on simulated control problems and manipulators, it offers a promising solution to complex robotics and control challenges.