Dynamic Adaptive Model Selection Based on Content and Resolution

Abstract

Machine learning-based video analytics applications have been adopted for a wide range of services in recent years. To efficiently utilize computing resources and reduce the pressure and congestion risks on central servers, there is a great demand to deploy these services in edge devices. A major challenge in deploying these video analytics applications in edge devices is the limited power supply of the edge devices compared to the high power consumption of the machine learning models. In a scenario where video stream is collected on an edge device and different machine learning models are chosen to perform video analytics on each frame, a key challenge we face is the balance between limited energy resources of the edge device and the quality of video analytics results. To address this challenge, we are motivated to develop a model selection framework that adapts the workload of the edge devices according to the complexity and resolution of the incoming video frames. We propose the Dynamic Adaptive Model Selection Based on Content and Resolution (DAMSBCR) system, a Markov Decision Process (MDP)-based model selection scheme for energy-constrained video analysis framework for edge computing. DAMSBCR enforces the long-term energy constraint through the virtual energy queue, filters out the inferior models through model pre-selection before implementation to reduce the computational load for finding optimal decisions, and finds the best model selection strategy through our MDP-based model selection algorithm to achieve high inference accuracy without compromising the energy consumption goal. We evaluate the performance of our DAMSBCR system through trace-driven simulations and show that DAMSBCR significantly outperforms all benchmarks on all parameters, with the model selection algorithm making a significant contribution to balancing short- and long-term inference accuracy.