Multi-agent modelling of electricity markets

This thesis describes the development of three conceptual models built to serve as decision support tools in liberalised electricity markets. The introduction of competition, higher uncertainty and decentralised planning requires new planning and analysis tools on the medium to long term to support decision making at the level of the industry as well as at the level of the market authorities. We have mainly developed models of market simulation and investment decision by investigating how multi-agent systems can contribute to the understanding of the decentralised nature of competitive decision making in a liberalised electricity market. The thesis focused particularly on bilateral transactions arguing that such transaction processes are specially relevant from the point of view of risk management when considering medium to long term decisions (for both portfolio management and investment decision making). We have used decision analysis under uncertainty and dynamic modeling approaches under a multi-agent framework by considering that such approaches are more adequate in a competitive environment due to the market actors' exposure to risks and need for adaptation in a changing environment. Monthly spot market prices are taken into account through scenarios that reproduce volatility by lognormal processes. Monte Carlo simulation is used to quantify the risks among which the spot prices are taken as the most significant source of uncertainty. The main objective of this thesis is to contribute to the problem of modeling investment decisions in new generation capacities using electricity market simulation from a planning point of view. This global objective is driven according to the: Modeling of the bilateral transaction processes between generation companies and demand companies at the level of wholesale market by taking into account the possible tradeoffs between spot and bilateral markets. The transaction processes models represent the market simulation modules by: Distinguishing the different forms the transaction processes can take (negotiation vs. auctions). Modeling of the market actors' portfolio management from a planning perspective by taking into account the market uncertainties (sales portfolio for generation companies and supply portfolios for demand companies). Modeling of the investment decisions in new electricity generation power plants. "Negotiation based portfolio management in a liberalised electricity market: a multiagent approach" The first model of the thesis formalizes the transactions of demand companies that would have to face a single generation company. The transaction process is here modeled as a bilateral negotiation algorithm between buyers and a seller. The negotiation is driven on two issues: prices and quantities of monthly contracts. It is simulated as a succession of offers and counteroffers during a limited period of time. Each market actor is searching for efficient transactions that satisfy its objectives, preferences and constraints. The reasoning process of the agents is based on the multi-attribute tradeoffs under uncertainty seeking each month the optimal configuration of portfolios composed of bilateral and spot contracts. The simulated output of the negotiation does not necessarily optimise the social welfare as both parties have opposing interests and interact under incomplete information. The efficiency of the algorithm is characterized by a ratio relatively to the Pareto optimal contracts. It was found that a supplier with nuanced risk aversion have high efficiency beyond the Pareto optimal points (between 100 and 120%). It indicates that the single generation company has a higher bargaining power than the individual demand companies independently of the preferences and negotiation behaviour of its counterparts. The negotiation efficiency of the different demand companies ranges from 80 to 100% and is related to their attitude toward risks. Risk averse demand companies accept higher risk premiums in order to avoid spot market risk exposure unlike other agents that may have more nuanced risk preferences. The bilateral contracts concluded covers the electricity demand partially or totally depending on the risk attitude. Demand companies with nuanced risk aversion negotiate contracts in order to manage some flexibility and take benefit from potential spot market opportunities. "Bilateral transactions in a competitive electricity market: multiple auctions using an agent based approach" The second transaction process models bilateral transactions between demand companies and several generation companies through multiple auctions. In this context, the generation companies must compete between each other to satisfy the call-for-bids (CfBs) launched simultaneously by each demand company, i.e. each buyer in the system opens an auction by expressing its needs and preferences by product (base and peak). After receiving the CfBs, the suppliers send offers responding to the multiple auctions opened by the demand companies. The offers timing is sequenced by a priority list that ranks the demand companies according to their value. The construction of the offers is modeled with a function of the evaluated utility of the customer based on its CfB. The demand companies, at the closure of their individual auction, compare and select the offers with maximum value by including the evaluation of the possible tradeoffs between bilateral and spot markets. The dynamic of competition requires the suppliers to adapt their monthly pricing strategies to satisfy their profit and market share objectives. The adaptive behaviour of the generation companies is here modeled using a directed learning approach. Learning is performed on the basis of two variables (risk premiums and market shares) which previous month instances permit causal inferences on eight rules of pricing parameters adaptation. The results of a case study show that the agents in the system build flexible portfolios (partial or full supply contracts, supply portfolio with several producers, spot contracts ...). The agents learning process tend to satisfy their objectives of profit and market shares. The bilateral market transaction volume represents between 70 and 90% of the total transaction volume (10 to 30% of spot contracts) as a result of the tradeoffs between the two markets. "Electricity generation capacity expansion under competition: A multi-agent dynamic programming model" The last model proposed in this thesis intends to model the decentralized investment decision making in a competitive electricity market. The capacity expansion model proposed aims at evaluating the medium to long term investment decisions according to criteria of risks and profits maximisation. We use the agent-based multiple-auction module to simulate he electricity market. The investment decisions of each supplier agent in the system are assessed according to a dynamic programming algorithm that explores different potential expansion plans in predefined periods. We consider that capital rationing, preservation of capital under risk and market share targeted ranges are three constraints that bound the expansion plans. Two major sources of uncertainties are considered: market price risk and strategic risk. Strategic risks are evaluated through a customer and competitors focus by putting into perspective the investment decision relatively to the competitors' potential expansion plans. Hence, the evaluation of an expansion plan is built as the convolution of the realizable incomes through the potential configurations of competitors strategic investment decisions. In our model, a supplier agent is seeking to take market positions relatively to its competitor through the market share that he can expect to secure its investment decisions. It appears that risk reduction highlighted here through bilateral transactions is a more realistic approach when assessing investment decisions in a competitive and uncertain electricity market.

Investment decisions with flexibility under uncertainty in the competitive electricity supply industry

Jun Dong

The restructuring of the electricity supply industry (ESI) has introduced new actors and market mechanisms. Electricity prices are more market oriented and fluctuate greatly. These changes bring about more uncertainties and risks to the market participants. When there are high risks in the future, the actors tend towards short-term decisions in which the risks are relatively easier to be coped with than in the long-term decisions. Much research has been carried out on short and medium-term risk management. The contracts of financial derivatives are managed for hedging risks. However, the development of long-term planning and risk management methods in the liberalised ESI has so far not attracted much attention. Long-term anticipations and planning have always been a significant issue given the specific characteristics of the ESI and the importance of security of supply. In liberalised electricity markets, the traditional centralised least-cost generation capacity planning is less relevant. The planning activities have become decentralised, i.e., individual actor makes planning for achieving its goals under certain risks. The ESI is in a transition from monopoly to competitive market and the liberalised electricity market has a short history. It is dominated by structural changes and actors' game behaviour, which are usually not of a random nature. Therefore, it is not always able to model the long-term uncertainties with the widely used probability methods in risk management. Under the large uncertainties and with the difficulty of uncertainty modelling, how would the actors make long-term investment decisions and manage risks? This thesis is an attempt to answer this question. The problem under study in this research is generalised as "investment planning with flexibility in the competitive electricity market". We argue that to select a flexible alternative strategy is one of the ways for managing long-term risks. The flexibility means that the actor is capable of adapting the investment decisions under different market situations in the future. For example, a project can be deferred to wait for more information, be expanded if the market situation becomes favourable or be abandoned if the market situation becomes unfavourable. For an electricity generator, the problem is capacity planning, for example, to construct a new power plant. For a consumer, it is energy portfolio planning. The alternatives may include different purchasing contracts and investment for self-generation. The purpose of this research is to develop a new approach for dealing with these kinds of problems. The requirements of the new approach are proposed in this thesis as: The flexibility of the investment projects or strategies should be accounted in the decision-making; The uncertainty modelling should reflect the long-term trends and characteristics of the liberalised electricity markets; Electricity price forecasting should reflect the interactions among the market players; The approach is an integration of scenario building, market simulation, strategy evaluation and decision-making. The new approach is developed within a conceptual framework of an intelligent decision support system proposed by Dr E. Gnansounou. The framework consists of four modules: Scenario builder, Problem formulator and attributes evaluator, Electricity market multi-agent system and Decision making aid. The principal researches focus on the methods of attributes evaluation taking into account flexibility actions and agent-based market simulation. The attributes evaluation of investment alternatives emphasizes the incorporation of uncertainty modelling and flexibility actions. Two types of attributes evaluation methods are developed using real options theory and fuzzy set theory respectively. Real options theory, in which uncertainty is modelled with probability methods, is introduced for computing and incorporating the flexibility value. In addition, a new method is developed for attributes evaluation taking into account the flexibility actions, using fuzzy set to represent imprecise or vague knowledge in electricity markets. For a long-term perspective, the main factors influencing investment decisions, such as electricity prices, are affected mainly by the structural changes and strategic behaviours of the actors in the market. It is not pertinent to forecast the values of these variables from the historical data using statistic models. The agent-based market simulation system is developed to simulate the interactions among the market participants and to provide electricity price scenarios. Three applications are dealt with by applying the proposed approach. Firstly, agent-based market simulation is applied to an illustrative power system. Secondly, a generation capacity planning and project evaluation for an electricity generator is performed. Thirdly the energy portfolios of a consumer is evaluated and compared. The research found that the new method to take into account flexibility actions is practical. It incorporates the flexibility actions into the cash flows. The flexibility cost is given exogenously. The value of a base strategy with flexibility actions is calculated. The alternatives are compared with these values. It is different from the real options method. The real options method assumes that the future value of the strategy is known with a predefined probability distribution and the value of the option is calculated. It deals with the pricing of a contract or a project. Differently from that, the new method deals with the comparison of portfolio given the prices and the costs are assumed as known and included into the cash flows, and how to consider the flexibility in the portfolio comparison. The real options method is operative for incorporating flexibilities in investment decisions if the uncertain variables follow the Wiener stochastic process. The advantages of this method are that it provides flexibility values and optimal timing of a project; it models the future value of the project with the stochastic process therefore it does not require the estimations on the values of driving variables for the lifetime of the project; it uses risk-free rate that is easier to estimate than the discount rate used in the DCF method. The limitations are that the assumptions of probabilistic distributions of the future value of the project are not always satisfied. The determination of the stochastic process and the estimation of the volatility of the value of the project are not straightforward. As a complement to the real options method and dealing with a different problem, the fuzzy method is proved in this thesis to give a "softer" approach when modelling the imprecision with fuzzy numbers. This method is suitable to electricity markets with incomplete information, lack of sufficient data and non-random movement. It is found that the agent-based simulation model is effective for giving "what if" scenarios on electricity market states in the future. The model is flexible in simulating imperfect electricity markets where game play exists. The main contributions of the research are summarised as follows: A new approach of investment decision support is developed fitting to electricity markets with regard to uncertainty and flexibility modelling. Incorporation of flexibility into the investment decisions is developed as a way for managing long-term risks. Uncertainty modelling with the real options method and imprecision data modelling with the fuzzy method can be applied according to the pertinence to the situation of electricity markets. The attributes evaluation of the investment strategies is based on agent-based market simulation. Instead of traditional least-cost centralised planning, the generation expansion planning is considered as decentralised planning where the individual agent makes expansion planning, taking into account other agents' plans.

EPFL2004

Scalable Multi-agent Coordination and Resource Sharing

Panayiotis Danassis

A plethora of real world problems consist of a number of agents that interact, learn, cooperate, coordinate, and compete with others in ever more complex environments. Examples include autonomous vehicles, robotic agents, intelligent infrastructure, IoT devices, and so on. As more and more autonomous agents are deployed in the real-world, it will bring forth the need for novel algorithms, theory, and tools to enable coordination on a massive scale. In this thesis, we develop such tools to tackle two central challenges in multi-agent coordination research: solving allocation problems, and resource sharing, focusing on solutions that are scalable, practical, and applicable to real-world problems.In the first part of the thesis we tackle the problem of allocating resources to agents, i.e., solving a weighted matching problem. Real-world matching problems may occur in massively large systems, they are distributed and information-restrictive, and individuals have to reveal their preferences over the possible matches in order to get a high quality match, which brings forth significant privacy risks. As such, there are three main challenges: complexity, communication, and privacy.Our proposed approach, ALMA, is a practical heuristic designed for real-world, large-scale (

10^6

agents) applications. It is based on a simple altruistic behavioral convention: agents have a higher probability to back-off from contesting a resource if they have good alternatives, potentially freeing the resource for some agent that does not. ALMA tackles all of the aforementioned challenges: it is decentralized, runs on-device, requires no inter-agent communication, converges in constant time -- under reasonable assumptions --, and provides strong, worst-case, privacy guarantees. Moreover, by incorporating learning we can mitigate the loss in social welfare and increase fairness. Finally, rational agents can use such simple conventions, along with an arbitrary signal from the environment, to learn a correlated equilibrium for accessing a set resources, under high congestion.In the second part of the thesis we focus on a critical open problem: the question of cooperation in socio-ecological and socio-economical systems, and sustainability in the use of common-pool resources. In recent years, learning agents, especially deep reinforcement learning agents, have become ubiquitous in such systems. Yet, scaling to environments with a large number of agents and low observability continues to be a challenge. In our work, we focus on common-pool resources. Individuals face strong incentives to appropriate, which results in overuse and even the depletion of the resources. Our goal is to apply simple interventions to steer the population to desirable states.We propose a simple, yet powerful, and robust technique: allow agents to observe an arbitrary common signal from the environment. The agents learn to couple their policies, and avoid depletion in a wider range of settings, while achieving higher social welfare and convergence speed.Finally, we propose a practical approach to computing market prices and allocations via a deep reinforcement learning policymaker agent. Compared to the idealized market equilibrium outcome -- which can not always be efficiently computed -- our policymaker is much more flexible, allowing us to tune the prices with regard to diverse objectives such as sustainability and resource wastefulness, fairness, buyers' and sellers' welfare, etc.

EPFL2022

A multi-agent approach for planning activities in decentralized electricity markets

Jun Dong, Edgard Gnansounou, Abdelaziz Lahlou

Despite the rigour and ability of game theory to cope with oligopolistic electric markets, it fails to model many existing behaviours in the real-world circumstances. The traditional models such as statistical extrapolation or econometrics are not capable to anticipate the changes in the pattern of the market prices due to the future structural changes. Furthermore, in such free and open markets, there is a more intense need for each participant to benefit from a certain level of autonomy, while keeping some capabilities to interact, communicate, collaborate and negotiate with other participants in an efficient way. As a result, there is a need for a novel framework of modelling that could include game theoretical assumptions as well as other more complex assumptions. Agent technologies in general and agent-based simulation in particular offer this possibility. This paper proposes, in a decision-making perspective, a new multi-agent architecture specifically designed to support planning activities in decentralized electricity markets, with a certain level of flexibility. In this model, synthetic agents are created allowing flexible representations of the multi-functional market players and possible mergers and coalitions in the electricity market.

2007