Simultaneous Multi-Object Tracking and Classification via Approximate Variational Inference

Abstract

In modern applications, robots are expected to work in complex dynamic environments and extract meaningful information from low-level, noisy data. In particular, they must build a description of the objects they interact with. This description should be both qualitative and quantitative. The former can be expressed in terms of object classes, while the latter is expressed by the object dynamics. Qualitative descriptors can be thought of as discrete assignments of object trajectories to category labels that represent different motion patterns in the environment. Obtaining these descriptors along with the kinematic states of the objects, from data, is a challenging task due to the noisy nature of sensor measurements, sensor failure, object occlusions and the presence of objects with infrequent dynamics. Quantitative descriptors such as locations and velocities are usually obtained using widely known filtering techniques derived from the Kalman filter. Nevertheless, when dealing with measurements originated by multiple objects, associating these measurements with individual objects generates a number of hypotheses that grows combinatorially with the number of measurements, and exponentially with time. Generating these assignments, while also estimating the kinematic state and classes of the objects is a computationally intractable problem. This thesis proposes a probabilistic model that exploits the correlations between object trajectories and classes and an inference procedure that renders the problem tractable through a structured variational approximation. The framework presented is very generic, and can be used for various tracking applications. It can handle objects with different and/or infrequent dynamics, such as cars and pedestrians, and it can seamlessly integrate multi-modal features, for example object dynamics and appearance