Kinematics-Driven Motion Analysis for 3D Hand Trajectory Prediction in Virtual Reality

Abstract

Hand motion plays a fundamental role in how people interact with their environment, making hand trajectory prediction and analysis vital across numerous domains, including Human-Computer Interaction (HCI) and Human-Robot Interaction (HRI). Accurate hand motion modeling and prediction are particularly beneficial in Virtual Reality (VR) applications, where they can reduce system latency and enable the development of novel and immersive experiences.

However, despite prior work on various statistical and deep learning approaches, challenges persist in developing accurate, efficient, and generalizable models for deployment in real-time applications on devices with limited resources.

This thesis addresses these challenges by introducing novel techniques for analyzing and predicting hand motion, integrating empirical data and kinematics-based approaches to enhance existing mathematical models and statistical methods.First, we conducted a user study with 20 participants performing hand movements in VR to address the limitations of existing datasets. Using the collected data, we developed user-specific predictive models tailored to individual motion patterns, achieving high accuracy in hand motion prediction. Building on these personalized models, we developed generalized models that maintain comparable performance levels. These generalized models are designed to adapt across a broader user base without requiring individual customization, balancing accuracy and generalizability.

This thesis advances the fields of HCI and HRI by deepening the understanding of human hand motion and presenting techniques for accurate and efficient motion prediction. It offers a scalable approach that balances customization with broader applicability by introducing user-specific and generalized models. The future of this research lies in expanding it to encompass the entire human body, with the potential to significantly impact areas such as VR and collaborative human-robot environments.