Reliability Mechanisms for Controllers in Real-Time Cyber-Physical Systems

Cyber-physical systems (CPSs) are real-world processes that are controlled by computer algorithms. We consider CPSs where a centralized, software-based controller maintains the process in a desired state by exchanging measurements and setpoints with process agents (PAs). As CPSs control processes with low-inertia, e.g., electric grids and autonomous cars, the controller needs to satisfy stringent real-time constraints.

However, the controllers are susceptible to delay and crash faults, and the communication network might drop, delay or reorder messages. This degrades the quality of control of the physical process, failure of which can result in damage to life or property. Existing reliability solutions are either not well-suited for real-time CPSs or impose serious restrictions on the controllers. In this thesis, we design, implement and evaluate reliability mechanisms for real-time CPS controllers that require minimal modifications to the controller itself.

We begin by abstracting the execution of a CPS using events in the CPS, and the two inherent relations among those events, namely network and computation relations. We use these relations to introduce the intentionality relation that uses these events to capture the state of the physical process. Based on the intentionality relation, we define three correctness properties namely, state safety, optimal selection and consistency, that together provide linearizability (one-copy equivalence) for CPS controllers.

We propose intentionality clocks and Quarts, and prove that they provide linearizability. To provide consistency, Quarts ensures agreement among controller replicas, which is typically achieved using consensus. Consensus can add an unbounded-latency overhead. Quarts leverages the properties specific to CPSs to perform agreement using pre-computed priorities among sets of received measurements, resulting in a bounded-latency overhead with high availability. Using simulation, we show that availability of Quarts, with two replicas, is more than an order of magnitude higher than consensus.

We also propose Axo, a fault-tolerance protocol that uses active replication to detect and recover faulty replicas, and provide timeliness that requires delayed setpoints be masked from the PAs. We study the effect of delay faults and the impact of fault-tolerance with Axo, by deploying Axo in two real-world CPSs.

Then, we realize that the proposed reliability mechanisms also apply to unconventional CPSs such as software defined networking (SDN), where the controlled process is the routing fabric of the network. We show that, in SDN, violating consistency can cause implementation of incorrect routing policies. Thus, we use Quarts and intentionality clocks, to design and implement QCL, a coordination layer for SDN controllers that guarantees control-plane consistency. QCL also drastically reduces the response time of SDN controllers when compared to consensus-based techniques.

In the last part of the thesis, we address the problem of reliable communication between the software agents, in a wide-area network that can drop, delay or reorder messages. For this, we propose iPRP, an IP-friendly parallel redundancy protocol for 0 ms repair of packet losses. iPRP requires fail-independent paths for high-reliability. So, we study the fail-independence of Wi-Fi links using real-life measurements, as a first step towards using Wi-Fi for real-time communication in CPSs.

Reliable and Robust Cyber-Physical Systems for Real-Time Control of Electric Grids

Wajeb Saab

Real-time control of electric grids is a novel approach to handling the increasing penetration of distributed and volatile energy generation brought about by renewables. Such control occurs in cyber-physical systems (CPSs), in which software agents maintain safe and optimal grid operation by exchanging messages over a communication network. We focus on CPSs with a centralized controller that receives measurements from the various resources in the grid, performs real-time computations, and issues setpoints. Long-term deployment of such CPSs makes them susceptible to software agent faults, such as crashes and delays of controllers and unresponsiveness of resources, and to communication network faults, such as packet losses, delays, and reordering. CPS controllers must provide correct control in the presence of external non-idealities, i.e., be robust, and in the presence of controller faults, i.e., be reliable. In this thesis, we design, test, and deploy solutions that achieve these goals for real-time CPSs. We begin by abstracting a CPS for electric grids into four layers: the control layer, the network layer, the sensing and actuation layer, and the physical layer. Then, we provide a model for the components in each layer, and for the interactions among them. This enables us to formally define the properties required for reliable and robust CPSs. We propose two mechanisms, Robuster and intentionality clocks, for making a single controller robust to unresponsive resources and non-ideal network conditions. These mechanisms enable the controller to compute and issue setpoints even when some measurements are missing, rather than to have to wait for measurements from all resources. We show that our proposed mechanisms guarantee grid safety and outperform state-of-the-art alternatives. Then, we propose Axo: a framework for crash- and delay-fault tolerance via active replication of the controller. Axo ensures that faults in the controller replicas are masked from the resources, and it provides a mechanism for detecting and recovering faulty replicas. We prove the reliable validity and availability guarantees of Axo and derive the bounds on its detection and recovery time. We showcase the benefits of Axo via a stability analysis of an inverted pendulum system. Solutions based on active replication must guarantee that the replicas issue consistent setpoints. Traditional consensus-based schemes for achieving this are not suitable for real-time CPSs, as they incur high latency and low availability. We propose Quarts, an agreement mechanism that guarantees consistency and a low bounded latency- overhead. We show, via extensive simulations, that Quarts provides an availability at least an order of magnitude higher than state-of-the-art solutions. In order to test the effect of our proposed solutions on electric grids, we developed T-RECS, a virtual commissioning tool for software-based control of electric grids. T-RECS enables us to test the proper functioning of the software agents both in ideal and faulty conditions. This provides insight into the effect of faults on the grid and helps us to evaluate the impact of our reliability solutions. We show how our proposed solutions fit together, and that they can be used to design a reliable and robust CPS for real-time control of electric grids. To this end, we study a CPS with COMMELEC, a real-time control framework for electric grids via explicit power setpoints. We analyze the reliability issues...

EPFL2019

Modeling and Measuring Performance of Data Dissemination in Opportunistic Networks

Nikodin Ristanovic

In this thesis we focus on understanding, measuring and describing the performance of Opportunistic Networks (ONs) and their applications. An “opportunistic network” is a term introduced to describe a sparse, wireless, ad hoc network with highly mobile nodes. The opportunistic networking paradigm deviates from the traditional end-to-end connectivity concept: Forwarding is based on intermittent connectivity between mobile nodes (typically, users with wireless devices); complete routes between sources and destinations rarely exist. Due to this unique property of spontaneous link establishment, the challenges that exist in ONs are specific. The unstructured nature of these networks makes it difficult to give any performance guarantees on data dissemination. For this reason, in Part I of this thesis we explore the dynamics that affect the performance of opportunistic networks. We choose a number of meaningful scenarios where our models and algorithms can be validated using large and credible data sets. We show that a drift and jump model that takes a spatial approach succeeds in capturing the impact of infrastructure and mobile-to-mobile exchanges on an opportunistic content update system. We describe the effects of these dynamics by using the age distribution of a dynamic piece of data (i.e., information updates) as the performance measure. The model also succeeds in capturing a strong bias in user mobility and reveals the existence of regions, whose statistics play a critical role in the performance perceived in the network. We exploit these findings to design an application for greedy infrastructure placement, which relies on the model approximation for a large number of nodes. Another great challenge of opportunistic networking lies in the fact that the bandwidth available on wireless links, coupled with ad hoc networking, failed to rival the capacity of backbones and to establish opportunistic networks as an alternative to infrastructure-based networks. For this reason, we never study ONs in an isolated context. Instead, we consider the applications that leverage interconnection between opportunistic networks and legacy networks and we study the benefits this synergy brings to both. Following this approach, we use a large operator-provided data set to show that opportunistic networks (based on Wi-Fi) are capable of offloading a significant amount of traffic from 3G networks. At the same time, the offloading algorithms we propose reduce the amount of energy consumed by mobiles, while requiring Wi-Fi coverage that is several times smaller than in the case of real-time offloading. Again we confirm and reuse the fact that user mobility is biased towards certain regions of the network. In Part II of this thesis, we treat another issue that is essential for the acceptance and evolution of opportunistic networks and their applications. Namely, we address the absence of experimental results that would support the findings of simulation based studies. Although the techniques such as contact-based simulations should intuitively be able to capture the performance of opportunistic applications, this intuition has little evidence in practice. For this reason, we design and deploy an experiment with real users who use an opportunistic Twitter application, in a way that allows them to maintain communication with legacy networks (i.e., cellular networks, the Internet). The experiment gives us a unique insight into certain performance aspects that are typically hidden or misinterpreted when the usual evaluation techniques (such as simulation) are used. We show that, due to the commonly ignored factors (such as the limited transmission bandwidth), contact-based simulations significantly overestimate delivery ratio and obtain delays that are several times lower than those experimentally acquired. In addition to this, our results unanimously show that the common practice of assuming infinite cache sizes in simulation studies, leads to a misinterpretation of the effects of a backbone on an opportunistic network. Such simulations typically overestimate the performance of the opportunistic component, while underestimating the utility of the backbone. Given the discovered deficiencies of the contact-based simulations, we consider an alternative statistical treatment of contact traces that uses the weighted contact graph. We show that this approach offers a better interpretation of the impact of a backbone on an opportunistic network and results in a closer match when it comes to modeling certain aspects of performance (namely, delivery ratio). Finally, the security requirements for the opportunistic applications that involve an interconnection with legacy networks are also highly specific. They cannot be fully addressed by the solutions proposed in the context of autonomous opportunistic (or ad hoc) networks, nor by the security frameworks used for securing the applications with continuous connectivity. Thus, in Part III of this thesis, we put together a security framework that fits the networks and applications that we target (i.e., the opportunistic networks and applications with occasional Internet connectivity). We then focus on the impact of security print on network performance and design a scheme for the protection of optimal relaying capacity in an opportunistic multihop network. We fine-tune the parameters of our scheme by using a game-theoretic approach and we demonstrate the substantial performance gains provided by the scheme.