3D Neural Human Avatar Reconstruction from RGB Images

Abstract

Human avatars play a crucial role in a wide array of real-life applications, including gaming, augmented reality/virtual reality (AR/VR), film production, telepresence, and especially in the emerging metaverse, where individualized avatar representation is in high demand. Traditional methods for constructing human avatars typically rely on 3D scans and motion capture, which involve complex and costly multi-sensor setups that are mostly limited to indoor and controlled environments, restricting their practical use. In recent years, researchers have been exploring the reconstruction of human avatars solely from RGB images, offering a more accessible and user-friendly alternative, as RGB cameras are widely available. Generally, visual features from multiple views or frames are aggregated spatially or temporally to predict the geometry and appearance of the avatar. The human body can be represented as a point cloud, polygon mesh, or voxel grid. Recently, the neural radiance field, as a popular implicit 3D representation, has been introduced and achieved considerable success in human avatar reconstruction. Nevertheless, despite significant progress has been made in this field, 3D neural human avatar reconstruction from 2D RGB images still poses several challenges. our research aims to reconstruct a generalizable and editable neural human avatar rapidly and efficiently from RGB images, while also adapting the reconstruction to various types of cameras. Our approach seeks to contribute to the advancement of human avatar reconstruction technology, enhancing its applicability in a wide range of real-world scenarios.