QoS-Aware Resource Management for Multi-Tenant Distributed Systems

Abstract

Distributed systems, such as data processing systems and systems for web services, generally have been designed for multiple users with various quality of service (QoS) requirements. In this thesis, we study QoS-aware resource management for multi-tenant distributed systems explicitly taking into account sudden workload surges. To this end, we design a set of resource management solutions for recent multi-tenant distributed systems, key-value data stores and serverless computing platforms, with the overarching QoS metric being end-to-end response time. First, we develop a control theory based decentralised admission controller that dynamically regulates incoming requests based on their corresponding QoS classes for key-value data stores. The controller consists of a set of resource management modules and algorithms. It operates in a decentralised manner, running one instance per user. In particular, each of these controllers deals with service requests with only local performance metrics, response time and queue waiting time. Despite the use of such local information, these controllers are capable of coping with workload surges respecting QoS requirements. Second, we address the problem of ensuring QoS for serverless computing platforms in the presence of workload fluctuations and sudden surges. We develop two closed-loop (feedback-based) CPU cap controllers that dynamically adjust CPU usage limit/cap to fulfil QoS requirements. The first controller adjusts the number of worker threads and the second one throttles CPU usage cap per worker thread. Experimental results of the first controller show they reduce QoS violations in many folds compared to static resource allocation techniques used in commercial serverless computing platforms. And the second controller also significantly decreases the skewness of response time up to two folds without overusing CPU resources.