Reinforcement Learning for proactive operation of residential energy systems by learning stochastic occupant behavior and fluctuating solar energy: Balancing comfort, hygiene and energy use

When it comes to residential buildings, there are several stochastic parameters, such as renewable energy production, outdoor air conditions, and occupants’ behavior, that are hard to model and predict accurately, with some being unique in each specific building. This increases the complexity of developing a generalizable optimal control method that can be transferred to different buildings. Rather than hard-programming human knowledge into the controller (in terms of rules or models), a learning ability can be provided to the controller such that over the time it can learn by itself how to maintain an optimal operation in each specific building. This research proposes a model-free control framework based on Reinforcement Learning that takes into account the stochastic hot water use behavior of occupants, solar power generation, and weather conditions, and learns how to make a balance between the energy use, occupant comfort and water hygiene in a solar-assisted space heating and hot water production system. A stochastic-based offline training procedure is proposed to give a prior experience to the agent in a safe simulation environment, and further ensure occupants comfort and health when the algorithm starts online learning on the real house. To make a realistic assessment without interrupting the occupants, weather conditions and hot water use behavior are experimentally monitored in three case studies in different regions of Switzerland, and the collected data are used in simulations to evaluate the proposed control framework against two rule-based methods. Results indicate that the proposed framework could achieve an energy saving from 7% to 60%, mainly by adapting to solar power generation, without violating comfort or compromising the health of occupants.

The digitalization of energy systems: towards higher energy efficiency

Marina Dorokhova

The energy industry is going through challenging times of disruptive changes caused by decarbonization, decentralization, and digitalization. As the energy value chain is restructuring itself to accommodate the growing penetration of renewables, increasing number of independent power producers, and augmented self-consumption, new energy management approaches are required to accomplish energy transition. With the Fourth Industrial Revolution underway, it becomes evident that digitalization is the key to increase energy efficiency and ensure stable, reliable, and secure operations of the electric grid. Due to the energy industry's massive inertia, most energy utilities are missing the real momentum for unleashing large-scale digitalization enabled by Information and Communication Technology (ICT). This thesis proposes a set of ICT-based software applications, models, and tools aiming to bridge the gap between strategic roadmaps focused on the energy industry's digitalization and their actual implementation in real-world scenarios through digital energy services. The research is conducted within the designed ICT-based smart building and smart community frameworks, the modular structure and scalability of which can serve as a backbone for future digital energy management solutions. The introduction of a novel unsupervised load disaggregation approach helps raise awareness of one's energy-related behavior and understand what drives the energy usage in residential households without compromising privacy and security. Showcasing algorithm's performance on real-world datasets from Norway and Germany highlights compliance with state-of-the-art disaggregation accuracy and reduced computational costs. The development of machine learning-based supervised and unsupervised building occupancy forecasting algorithms with prediction accuracies beyond 97% helps identify best-suited windows for energy-saving opportunities and deliver insights into one's presence and absence patterns. Built on top of that occupancy-centric rule-based heating and air conditioning automation algorithm strives to unlock the buildings' massive potential for energy savings without compromising the occupants' thermal comfort. Simulations on real-world datasets collected in Portugal demonstrate more than 15% potential energy savings. Zooming out from smart buildings towards smart communities, we focus on the important role of intelligent green mobility in supporting further digitalization of the electric power sector. To overcome the inconveniences posed by the sparsity of charging infrastructure and facilitate the adoption of Electric Vehicles (EVs), we present a reinforcement learning (RL)-based EV-specific routing method that guarantees paths' energy feasibility in a graph-theoretical context. Consequently, we propose several deep RL algorithms to control EV charging with the aim to increase renewables' self-consumption and EV drivers' satisfaction. Benchmarking against rule-based and model predictive control demonstrates RL's superior computational performance and better fitness for future mobility systems. Finally, we introduce an innovative decentralized blockchain-supported framework that enables secure and reliable accounting of energy exchanges within the smart community. Implementing it in a demonstration site in Switzerland shows blockchain's potential to reduce EV charging costs, transform the market's business model, and facilitate the large-scale deployment of EVs.

EPFL2021

Optimisation of Urban Form by the Evaluation of the Solar Potential

Marylène Montavon

What will the cities of tomorrow look like? This is a crucial question because in the four and a half billion years of its existence, the Earth has never been so threatened, as it is today. Since 2008 half of the world's population has lived in towns – more than 70% in rich nations. Three quarters of CO2 emissions are produced in towns, 75% of the world's energy is consumed in towns. The battle for our climate, for the continuation of our species and all other life, will be largely waged in cities. Will it be possible to create the ideal ecological town? The objective of this study is to compare actual and theoretical urban forms in order to explore the diverse effects of daylighting and solar potential on densely built-up sites. Indeed, the search for ideal solar orientations has been a persistent problem from the 19th century until the beginning of the 20th. Even today, for all our scientific knowledge and empirical experience of solar energy, making the correct choice has still not crossed the boundary from an enigmatic, tricky art into the field of an exact science. As a first step, six existing urban sites in Switzerland and Brazil, as well as two projects in the United Kingdom, a few generic models and a utopian city, were modelled and analyzed. The goal was not merely the construction of new, exemplary districts but also the transformation of existing towns. The findings have been used to assess the potential of façades and roofs located in urban areas for active and passive solar heating, photovoltaic electricity generation and daylighting. The results obtained from these different case studies revealed large variations of the potential for solar energy collection on the buildings' façades and roofs. Ideally, these investigations are expected to help architects and town planners understand how to optimise solar collection on buildings design. In addition, a study was made of the Contemporary City of Three Million Inhabitants created by Le Corbusier. According to the latter, the Contemporary City could increase the urban capacity and at the same time improve the urban environment and the efficiency of the city. Therefore the present study extracted the solar potential of this utopian city in order to assess Le Corbusier's propositions. Since 1922, Le Corbusier's studies have influenced architects and town-planners in their choice of orientations and urban forms: it was therefore crucial to discover if this reliance on Le Corbusier was justified. Solar irradiation and illuminance values obtained through numerical simulations form the core part of the method. The method adopted for the entire set of computer simulations consisted in modelling — via a computer-aided drawing piece of software, namely AutoCAD — the digital model of each building or group of buildings prior to exporting them to an urban performance analysis software (PPF/RADIANCE software package). Spatial distributions of solar irradiation and daylight fluxes over the overall building façades and roofs were calculated using ray-tracing simulation techniques to determine the appropriate placement of different solar technologies (passive and active solar, photovoltaic and daylighting). Performance indicators were used to assess the solar utilisation potential of the urban sites (e.g. statistics of sky view factors and daylight factors) in order to determine the optimal solar strategies for a given urban context. In addition, various works and sun-related recommendations that have appeared mainly since the end of the nineteenth century, together with the various scientific models and software tools pertaining to this topic, are presented in this study.

EPFL2010

Evaluation of a Combined Cycle Setup for Solar Tower Power Plants

James Spelling

As our awareness of the dangers of global climate change grows, the development of new renewable energy technologies is of primary importance in the effort to reduce emissions of carbon dioxide and other greenhouse gases. Rapid increases in the price of oil and worries about the stability and security of the extraction of fossil fuels have lead to renewed interest in the development of local energy sources, thereby reducing dependence on foreign sources. Amid the plethora of option available, one of the more promising solutions is the increased use of concentrating solar thermal power. Solar thermal power generation shows great potential for supplying the base electricity needs of numerous countries in the sun-belt regions, stretching from the tropics to the Mediterranean. This has lead to a recent renewal in the development of such systems, most notably with the construction of the PS10 and PS20 central receiver power plants in Spain. However, all currently existing solar thermal power plants have been based around the use of Rankine or steam turbine cycles, which are limited in the efficiencies they can achieve. In order to make better use of the Sun’s energy, higher efficiency cycles should be used. Recent developments in the field of high temperature solar receivers [13] have made it possible to imagine the use of solar thermal energy to drive gas turbine units, potentially allowing the use of combined cycle setups in solar thermal power plants. In order to evaluate the potential improvement in efficiency that can be achieved by using combined cycles in solar thermal power plants, a detailed simulation of such a setup will be performed. By taking into account the requirements of thermal energy storage and the design of the necessary heat exchangers, an objective evaluation of the performance of the combined cycle can be obtained. As a key element in the energy conversion process, particular attention will be given to the modelling and simulation of the high temperature receiver. Due to the highly variable nature of the solar flux, the performance of the receiver will be evaluated over a range of operating points. In order to obtain a detailed breakdown of the sources of losses and irreversibilities within the system, a complete exergy balance will be performed on the power plant. Combined with the standard energy balance, this analysis will allow identification of the key areas for improvement in the design of solar thermal power plants. Finally, in order to determine whether such a setup is economically viable, the construction, operation and maintenance costs will be evaluated in order to predict the levelised energy cost associated with electricity production in a combined cycle plant.

2008