Towards Viable Imitation Learning

Abstract

Imitation learning (IL), a fundamental machine learning paradigm, has achieved remarkable success in various domains, including autonomous driving, games, and robot locomotion. However, current IL relies heavily on an adversarial training scheme, which results in training instability and high computational burdens. These limitations pose challenges, especially in practical applications. Therefore, the thesis focuses on addressing three crucial research problems: improving data flexibility, enhancing safety, and reducing computation. By tackling these challenges, the goal is to advance IL and overcome the obstacles that hinder its practicality in real-world applications. First, the thesis analyzes the difference between conducting IL with expert observations and demonstrations and establishes the almost equivalence between these two methods in deterministic robot environments or robot environments with bounded randomness, promoting the applicability of learning from observation (LfO) in solving real-world problems. Furthermore, the thesis addresses the challenge of handling expert data in the form of visual inputs and proposes an IL framework that can effectively and efficiently learn from visual inputs by extracting meaningful features with data augmentation and maximizing sample reuse with off-policy learning. Then, the thesis presents a two-stage optimization framework, which employs a Lagrange multiplier to model application-oriented safety constraints and can generate policies that satisfy the prescribed safety constraint with a theoretical guarantee. Finally, the thesis conducts pilot studies on how to empower IL algorithms with quantum computing and presents two quantum IL (QIL) algorithms that can be run on quantum computers to reap the quantum advantage, paving the way for the quantum era.