Semantics Augmented Point Cloud Sampling for 3D Object Detection

Abstract

3D object detection is an emerging topic among both industries and research communities. It aims at discovering objects of interest from 3D scenes and has a strong connection with many real-world scenarios, such as autonomous driving. Currently, many models have been proposed to detect potential objects from point clouds. Some methods attempt to model point clouds in the unit of point, and then perform detection with acquired point-wise features. These methods are classified as point-based methods. However, we argue that the prevalent sampling algorithm for point-based models is sub-optimal for involving too much potentially unimportant data and may also lose some important information for detecting objects. Hence, it may lead to a significant performance drop. This thesis manages to improve the current sampling strategy for point-based models in the context of 3D detection. We propose recasting the sampling algorithm by incorporating semantic information to help identify more beneficial data for detection, thus obtaining a semantics augmented sampling strategy. In particular, we introduce a 2-phase augmentation for sampling. In the point feature learning phase, we propose a semantics-guided farthest point sampling (S-FPS) to keep more informative foreground points. In addition, in the box prediction phase, we devise a semantic balance sampling (SBS) to avoid redundant training on easily recognized instances. We evaluate our proposed strategy on the popular KITTI dataset and the large-scale nuScenes dataset. Extensive experiments show that our method lifts the point-based single-stage detector to surpass all existing point-based models and even achieve comparable performance to state-of-the-art two-stage methods.