Image-based 3D Object Detection for Autonomous Driving

Abstract

Autonomous driving has the potential to radically change people’s lives, improving mobility and reducing travel time, energy consumption, and emissions. As one of the key enabling technologies for autonomous driving, image-based 3D object detection has received a lot of attention and gradually becomes a hot research topic. In this thesis, we review existing image-based 3D object detection models and propose novel taxonomies to help readers understand common pipelines in this area. We also create a simple baseline model to identify and address the key challenge: 'localization error.' In addition to the ‘result-lifting’ method, we introduce a successful 'pseudo-LiDAR' approach that outperforms other methods. We show that its effectiveness lies in coordinate transformation rather than data representation. We also show how to use LiDAR signals to guide the image-based models. Particularly, we propose a simple and effective scheme to introduce the spatial information from LiDAR signals to the monocular 3D detectors, without introducing any extra cost in the inference phase. We first transform the LiDAR signals into the image representation and train a LiDAR model with the same architecture as the baseline model. This LiDAR model can serve as the teacher to transfer the learned knowledge to the image model, and the experiments show the effectiveness of our scheme. Moreover, to leverage the massive unlabeled data, we also investigate how to apply image-based 3D detection in the semi-supervised setting with the help of LiDAR signals. In summary, in this thesis, we thoroughly review existing image-based 3D detection models and propose new image-based 3D detection paradigms with promising performances. Besides, we also show how to use auxiliary LiDAR signals to guide the image-based model learning spatial features and achieve semi-supervised learning. Finally, we discuss open questions in this research field and point out several promising research directions.