Co-optimising Design and Operation of Energy Systems using Reinforcement Learning

The energy transition is ongoing, in Switzerland and abroad, leading to an ever-increasing development of renewable energy sources. These sources are decentralised, heterogeneous, come with a high dependance on weather conditions, and are therefore hard to integrate into so-called renewable energy systems. In this evolving landscape, digital transformation, especially Artificial Intelligence (AI), emerges as a key driver for this integration. The thesis leverages theoretical advances in Reinforcement Learning (RL), a subfield of AI, to tackle this challenge of the transition. Overall, the thesis provides a detailed data-driven analysis for renewable energy systems and introduces a new RL method to co-optimise design and control in such systems. The first part of the thesis is interested in the design problem, i.e., determining the optimal size of components, of renewable energy systems. This begins by pointing up the complementarity of renewable sources through a data-driven approach, using statistical indicators and energy flow simulations. Linear optimisation, a common method in the energy field, is then applied and evaluated to address the design problem in an islanded case study. The second part explores the potential of AI as an optimisation tool for energy systems. This exploration highlights RL's capability to address both the design problem and operation challenges, i.e., how to optimise the control of components, paving the way for the concept of co-optimisation. The final part of the research focuses on the co-optimisation of design and operation in energy systems using RL. In the energy sector, co-optimisation typically relies on various optimisation methods that require detailed mathematical modelling of the systems. Additionally the application of RL is limited to systems' operations. Outside the energy sector, RL has built upon recent advances in policy gradient methods to perform co-optimisation, but it has limited applications. Capitalising on these theoretical developments, the thesis introduces a new RL framework, tailored to tackle the co-optimisation challenge within energy systems. This framework optimises both control and design policies, thereby enhancing the integration of renewable sources and improving the system's efficiency. Its key advantages are its model-free nature, which eliminates the need for a predefined mathematical model of systems, and its ability to handle infinite time horizons. We believe this contribution opens new pathways for RL applications in the energy sector, ultimately contributing to more efficient energy management and better integration of renewable energy sources.