Multi-Modal Active Perception for Robotic Information Gathering in Science Missions

Abstract

Information gathering using mobile robots in dangerous and remote environments such as deep sea, underground, and outer space has significantly improved humanity's ability to understand the world. Typically in such science missions, a robot's role is limited to that of a sensing platform which passively gathers data along prescribed waypoints. Higher level decision making such as planning where to go in the context of mission objectives, making inferences from observations, and deciding which sensing modalities to deploy, is largely handled by a supervisory science team working remotely. However, communication constraints can hinder these processes and hence the rate of scientific progress. This thesis presents a novel active perception approach to science missions that aims to reduce robots' reliance on human supervision and improve science productivity by encoding an approximation of scientists' domain knowledge, inference, and decision making processes on-board the robot. Science missions can involve two complexities that are not addressed by existing approaches to robotic information gathering. Firstly, the variables of scientific interest may not be directly observable from on-board sensors, and instead require combining multi-modal proxy measurements with non-trivial amounts of scientific domain knowledge to infer. Secondly, robots may be equipped with multiple sensing modalities which measure different environmental variables and have some sensing cost associated with their usage. The first contribution of the thesis is the formulation of these requirements into a new information gain maximization sensor planning problem called scientific information gathering. An initial solution is presented which addresses scientific information gathering as two separate subproblems- estimating the variables of interest from sensor data by encoding an approximation of scientific domain knowledge, and planning paths and sensing actions that maximize the information gained on the variable of interest. Efficient solutions to these subproblems form the next two contributions. To tackle the first subproblem, a Bayesian network (BN) architecture that creates a probabilistic mapping between observations and the variables of scientific interest is proposed. The BN intuitively and jointly models critical aspects of scientific knowledge, as well as other prior knowledge such as sensor and classifier models, while the proposed network structure allows the robot to robustly handle noisy observations, has fast inference times, allows recursive estimation of key variables, and can utilize online parameter tuning to overcome initial modeling uncertainties. To address the second subproblem, a sampling-based forward simulation approach based on Monte Carlo tree search (MCTS) is proposed, which exploits the domain knowledge encoded in the BN and efficiently plans informative sensing actions with multiple sensing modalities in partially observable environments. The computational complexity of the BN-MCTS active perception framework does not grow with the number of observations taken, and allows long horizon planning in an anytime manner, making it highly applicable to field robots operating in large scale environments. The final contribution of the thesis is the application and evaluation of the proposed approach to three instances of scientific information gathering with unique mission constraints. The first is a Mars exploration mission where the robot is required to infer the geological type of locations in the environment by autonomously planning informative paths and observing geological features. The second scenario is modelled on the NASA Mojave Volatiles Prospector (MVP) project, where the robot attempts to autonomously refine errors in prior scientific knowledge during the mission. The final scenario explores how the active perception approach can be applied to next generation autonomous entry descent and landing (EDL) missions, which requires reasoning about temporal constraints. Extensive simulation results with synthetic and real data show that the proposed active perception approach significantly outperforms both myopic and passive approaches. Algorithms were also implemented on a prototype planetary rover along with supporting subsystems such as image processing modules, localization, and control. Autonomous end to end planning and execution of a science mission is demonstrated on an analog Mars environment, and key practical challenges in deploying the approach on real systems are discussed.