Multi-Objective Management of Multiprocessor Systems: From Heuristics to Reinforcement Learning

In my thesis, I reveal several already-existing and emerging challenges in multi-objective management of multiprocessor systems, and address them through novel solutions, from heuristics to RL, depending on the complexity of the problem. Conventional multi-objective management of multiprocessor systems mostly focuses on hot spots as the main factor of lifetime reliability. For modern multiprocessor systems and workloads, thermal stress has become the dominant factor in determining the Mean Time-To-Failure (MTTF). Together with the advances in multiprocessor systems, cooling technologies have been also progressively improving. As a result, existing Dynamic Thermal Management (DTM) policies should adapt to these emerging challenges and technologies to further improve the lifetime reliability. Finally, Therefore, I first propose a holistic, yet fast thermal stress-aware heuristic approach. The results demonstrate that the lifetime reliability can increase by up to 47%. Then, I show how emerging cooling technologies, such as two-phase liquid-cooling thermosyphon, necessitates adapting conventional heuristics to gain the greatest possible advantage from all its potential. My proposed approach decreases thermal hot spots and thermal stress by up to 10 oC and 45%, respectively, with 45% less cooling power consumption. Input-dependent workload variation in emerging applications and services, such as multimedia streaming makes power and performance management more challenging. Thus, I propose a machine learning-based framework for workload prediction and throughput estimation using hardware events available on modern multiprocessor systems. The proposed machine learning framework achieves 3.4x higher throughput with 15% less power consumption for High Efficiency Video Coding (HEVC), as a test-case application. I address runtime management and design space search of large and dynamic environments through RL. In particular, I first propose an RL-based framework to enable proactive fan speed control along with DVFS and workload allocation, providing up to 40% cooling power savings without any thermal constraint violations. Second, I address multimedia workload allocation of HEVC encoder on heterogeneous Systems-on-Chip (SoCs) through RL, achieving 20% higher compared to state of the arts. Third, I propose an RL-based approach that enables joint optimization of application- and system-level parameters, improving power consumption, performance, and average temperature of multiprocessor systems by 13%, 15%, and 10%, respectively, while improving the video quality and video compression of HEVC encoders, as a use-case application, by up to 1.19 dB and 24%. Then, to speed up design space search, I propose a Multi-Agent Reinforcement Learning (MARL) approach for multi-objective runtime management of multiprocessor systems. I use HEVC encoder as a test-case application, where MARL can enhance QoS violations by 5x, while speeding up the learning phase 15x. Finally, I address hyperparameter optimization of Convolutional Neural Network through a novel MARL approach. My proposed solution can reduce the model size, training time, and inference time by up to, respectively, 83x, 52%, and 54% without any degradation in accuracy.