Joint AI and System Solutions for Intelligent Video Processing on the Edge

Abstract

Multimedia data, ubiquitous in today's camera-equipped devices like drones, security systems, and IoT devices, presents a landscape filled with both opportunities and challenges. The existing network and computational infrastructures, however, find themselves struggling to adequately manage this deluge of data. This shortfall becomes particularly pronounced in scenarios that demand swift video analytics, such as surveillance operations during search and rescue missions, where even minor delays can have significant consequences. This thesis introduces innovative joint AI and system solutions aimed at enhancing edge intelligence, leveraging edge computing's potential to process video data locally. This strategy is crucial in scenarios where privacy and bandwidth limitations are significant concerns. The first part of the thesis proposes a multi-machine with restart scheduling algorithm optimized for processing stored video sequences in an edge-cloud system, aiming to minimize the total processing time of the entire system. The second part introduces a Deep Neural Network (DNN) model for enhancing the quality of stored videos through joint denoising and superresolution, leveraging advanced DNN architectures for high fidelity and usability. The third part presents another DNN model on joint video denoising and super-resolution, designed for real-time video streams. The fourth part proposes an innovative redesign of the adaptive bitrate (ABR) algorithm to cope with the DNN together with a system implementation. This comprehensive approach not only enhances data processing efficiency but also ensures adherence to prevailing ethical and technical constraints, marking a significant advancement in digital multimedia. Each chapter of the thesis methodically unveils the design and provides empirical results from real-world experiments, convincingly demonstrating the synergy of combining artificial intelligence and system solutions to enhance edge intelligence.