iPRP: Parallel Redundancy Protocol for IP Networks

Reliable packet delivery within stringent delay constraints is of primal importance to industrial processes with hard real-time constraints, such as electrical grid monitoring. Because retransmission and coding techniques counteract the delay requirements, reliability is achieved through replication over multiple fail-independent paths. Existing solutions such as parallel redundancy protocol (PRP) replicate all packets at the MAC layer over parallel paths. PRP works best in local area networks, e.g., sub-station networks. However, it is not viable for IP layer wide area networks, a key element of emerging smart grids. Such a limitation on scalability, coupled with lack of security, and diagnostic inability, renders it unsuitable for reliable data delivery in smart grids. To address this issue, we present a transport-layer design: IP parallel redundancy protocol (iPRP). Designing iPRP poses non-trivial challenges in the form of selective packet replication, soft-state and multicast support. Besides unicast, iPRP supports multicast, which is widely using in smart grid networks. It duplicates only time-critical UDP traffic. iPRP only requires a simple software installation on the end-devices. No other modification to the existing monitoring application, end-device operating system or intermediate network devices is needed. iPRP has a set of diagnostic tools for network debugging. With our implementation of iPRP in Linux, we show that iPRP supports multiple flows with minimal processing and delay overhead. It is being installed in our campus smart grid network and is publicly available.

iPRP - the Parallel Redundancy Protocol for IP Networks: Protocol Design and Operation

Jean-Yves Le Boudec, Miroslav Popovic, Dan-Cristian Tomozei

Reliable packet delivery within stringent delay-constraints is of paramount importance to mission-critical computer applications with hard real-time constraints. Because retransmission and coding techniques counteract the delay requirements, reliability may be achieved through replication over multiple fail-independent paths. Existing solutions, such as the parallel redundancy protocol (PRP), replicate all packets at the MAC layer over parallel paths. PRP works best in local area networks. However, it is not viable for IP networks that are a key element of emerging mission-critical systems. This limitation, coupled with diagnostic inability and lack of security, renders PRP unsuitable for reliable data-delivery in these IP networks. To address this issue, we present a transport-layer solution: the IP parallel redundancy protocol (iPRP). Designing iPRP poses non-trivial challenges in the form of selective packet replication, soft-state and multicast support. iPRP replicates only time-critical unicast or multicast UDP traffic. iPRP requires no modifications to the existing monitoring application, end-device operating system or to the intermediate network devices. It only requires a simple software installation on the end-devices. iPRP has a set of diagnostic tools for network debugging. With our implementation of iPRP in Linux, we show that iPRP supports multiple flows with minimal processing-and-delay overhead. It is being installed in our campus smart-grid network and is publicly available.

Institute of Electrical and Electronics Engineers2016

iPRP: Parallel Redundancy Protocol for IP Networks

Jean-Yves Le Boudec, Miroslav Popovic, Dan-Cristian Tomozei

Reliable packet delivery within stringent time limits is a key requirement of smart-grid and other industrial communication networks. The Parallel Redundancy Protocol (PRP), used in this context, duplicates packets at the MAC layer over parallel networks and is thus claimed to repair packet loss in 0 ms. However, PRP has several drawbacks: it works only for bridged networks; it requires special hardware; as a layer 2 protocol, it does not have diagnostic tools; it imposes the same MAC addresses to different interfaces, which exacerbates the management difficulty; multicast packet delivery is not natively supported. To address these problems we propose the IP Parallel Redundancy Protocol (IPRP). Unlike PRP, IPRP works in routed networks. It supports UDP applications, unicast or multicast. It is transparent to applications and requires no changes in application software. It is also transparent to the network as IPRP-related packets are regular IP packets; it requires no specific hardware. Operation at the IP layer, possibly with more than two networks makes the design of IPRP very different from PRP. We implemented IPRP in Linux 3.6.11-rt29 using iptables and deployed it on a campus smart grid. We describe our implementation and some performance results.

2014

Reliability Mechanisms for Controllers in Real-Time Cyber-Physical Systems

Maaz Mashood Mohiuddin

Cyber-physical systems (CPSs) are real-world processes that are controlled by computer algorithms. We consider CPSs where a centralized, software-based controller maintains the process in a desired state by exchanging measurements and setpoints with process agents (PAs). As CPSs control processes with low-inertia, e.g., electric grids and autonomous cars, the controller needs to satisfy stringent real-time constraints. However, the controllers are susceptible to delay and crash faults, and the communication network might drop, delay or reorder messages. This degrades the quality of control of the physical process, failure of which can result in damage to life or property. Existing reliability solutions are either not well-suited for real-time CPSs or impose serious restrictions on the controllers. In this thesis, we design, implement and evaluate reliability mechanisms for real-time CPS controllers that require minimal modifications to the controller itself. We begin by abstracting the execution of a CPS using events in the CPS, and the two inherent relations among those events, namely network and computation relations. We use these relations to introduce the intentionality relation that uses these events to capture the state of the physical process. Based on the intentionality relation, we define three correctness properties namely, state safety, optimal selection and consistency, that together provide linearizability (one-copy equivalence) for CPS controllers. We propose intentionality clocks and Quarts, and prove that they provide linearizability. To provide consistency, Quarts ensures agreement among controller replicas, which is typically achieved using consensus. Consensus can add an unbounded-latency overhead. Quarts leverages the properties specific to CPSs to perform agreement using pre-computed priorities among sets of received measurements, resulting in a bounded-latency overhead with high availability. Using simulation, we show that availability of Quarts, with two replicas, is more than an order of magnitude higher than consensus. We also propose Axo, a fault-tolerance protocol that uses active replication to detect and recover faulty replicas, and provide timeliness that requires delayed setpoints be masked from the PAs. We study the effect of delay faults and the impact of fault-tolerance with Axo, by deploying Axo in two real-world CPSs. Then, we realize that the proposed reliability mechanisms also apply to unconventional CPSs such as software defined networking (SDN), where the controlled process is the routing fabric of the network. We show that, in SDN, violating consistency can cause implementation of incorrect routing policies. Thus, we use Quarts and intentionality clocks, to design and implement QCL, a coordination layer for SDN controllers that guarantees control-plane consistency. QCL also drastically reduces the response time of SDN controllers when compared to consensus-based techniques. In the last part of the thesis, we address the problem of reliable communication between the software agents, in a wide-area network that can drop, delay or reorder messages. For this, we propose iPRP, an IP-friendly parallel redundancy protocol for 0 ms repair of packet losses. iPRP requires fail-independent paths for high-reliability. So, we study the fail-independence of Wi-Fi links using real-life measurements, as a first step towards using Wi-Fi for real-time communication in CPSs.

EPFL2018

Reliability Mechanisms for Controllers in Real-Time Cyber-Physical Systems

Maaz Mashood Mohiuddin

EPFL2018