Indoor Autonomous Flight Using Deep Learning-Based Image Understanding Techniques

Abstract

Indoor autonomous flight using artificial intelligence (AI) and machine learning techniques is presented. Flying inside a building without a positioning system requires a particular framework to connect computer vision, machine learning, control theory, and AI. The framework consists of six modules/disciplines presented to support indoor autonomous flight: optimization, state estimation, control, object detection, deep learning, and guidance. In this regard, the mathematical model of a quadcopter/drone is derived from an accurate model with a high level of fidelity by considering the non-linearity, uncertainties, and coupling. For the optimization module, a new heuristic optimization algorithm is designed to solve nonlinear optimization problems. The proposed algorithm utilizes a stochastic method to reach the optimal point based on simplex techniques. Swarm simplexes are distributed stochastically in the search space to locate the best optimal point. The designed algorithm is applied to 25 well-known benchmarks, and its performance is compared with Particle Swarm Optimization (PSO), the Nelder-Mead simplex algorithm, and the Grey Wolf Optimizer (GWO), both on its own and in hybrid forms where it is combined with either pattern search (hGWO-PS) or random exploratory search algorithms (hGWO-RES). The numerical results show that the presented algorithm, called Stochastic Dual Simplex Algorithm (SDSA), exhibits competitive performance in terms of accuracy and complexity. This feature makes SDSA efficient for tuning hyper-parameters and achieving the optimal weights of the reconstructed layer in deep learning modules. For the second filter module, a novel filter for nonlinear system state estimation is represented. This new filter formulates the state estimation problem as a stochastic dynamic optimization problem and utilizes a new stochastic method based on a genetic algorithm to find and track the best estimation. The experimental results show that the performance of the proposed filter, named Genetic Filter (GF), is competitive in comparison to that of classical and heuristic filters. GF is implemented to estimate unknown parameters required for the control. For the third control module, a new Proportional-Integral-Derivative-Accelerated (PIDA) control with a derivative filter was designed to improve quadcopter flight stability in a noisy environment. SDSA tunes the proposed PIDA controller associated with the objective of controlling. The simulation results show that the proposed control scheme is able to track the desired point in the presence of disturbances. Thus, the desired point is generated by extracting contextual information from images. For the fourth feature selection module, a novel multi-region feature-selection method is proposed to define histogram values of basic areas and random areas, from which it combines with continuous ant colony filter detection to represent the original target. The presented approach also achieves smooth tracking on different video sequences, especially with the motion blur problem. Both target recognition and tracking of the dynamic target are critical features for the autonomous drone. The experiment result demonstrates better and faster tracking abilities regarding traditional methods. The quality of the image is the crucial requirement to support high performance. Finally, the deep learning and guidance module issue commands to the system for actions. Improving the image resolution can enhance the performance of the image processing module’s tasks, such as object tracking, object detection, and depth detection. A new method, called a post-trained convolutional neural network (CNN), is proposed to increase the accuracy of current state-of-the-art single image super-resolution (SISR) methods. This method utilizes contextual information to update the last reconstruction layer of CNN using SDSA. The drone utilizes high-quality images to identify the target and estimate the relative distance. The estimated distance passes through the guidance low (i.e., pure proportional navigation (PPN)) to generate acceleration commands. The simulation results show that adapting the deep learning-based image understanding techniques (i.e., RetinaNet ant colony detection and Pyramid Stereo Matching Network (PSMNet)) into the proposed controller enables the drone to generate and track the desired point in the presence of disturbances in the complex environment.