Operating System and Network Co-Design for Latency-Critical Datacenter Applications

Datacenters are the heart of our digital lives. Online applications, such as social-networking and e-commerce, run inside datacenters under strict Service Level Objectives for their tail latency. Tight latency SLOs are necessary for such services to remain interactive and keep users engaged. At the same time, datacenters operate under a single administrative domain which enables the deployment of customized network and hardware solutions based on specific application requirements. Customization enables the design of datacenter-tailored and SLO-aware mechanisms that are more efficient and have better performance. In this thesis we focus on three main datacenter challenges. First, latency-critical, in-memory datacenter applications have ÎŒs-scale services times and run on top of hardware infrastructure which is also capable of ÎŒs-scale inter-node round-trip times. Existing operating systems, though, were designed under completely different assumptions and are not ready for ÎŒs-scale computing. Second, the base of datacenter communications is Remote Procedure Calls (RPCs) that depend on a message-oriented paradigm, while TCP still remains widely-used for intra datacenter communications. The mismatch between TCPâs bytestream-oriented abstraction and RPCs causes several inefficiencies and deteriorates tail latency. Finally, datacenter applications follow a scale-out paradigm based on large fan-out communication schemes. In such a scenario, tail latency becomes a critical metric due to the tail at scale problem. The two main factors that affect tail latency is interference and scheduling/load balancing decisions. To deal with the above challenges we advocate for a co-design of network and operating system mechanisms targeting ÎŒs-scale tail optimisations for latency-critical datacenter applications. Our approach investigates the potential of pushing functionality to the network leveraging emerging in-network programmability features. Whenever existing abstractions fail to meet the ÎŒs-scale requirements or restrict our design space, we propose new ones given the design and deployment freedom the datacenter offers. This thesis contributions can be split in three main parts. We, first, design and build tools and methodologies for ÎŒs-scale latency measurements and system evaluation. Our approach depends on a robust theoretical background in statistics and queueing theory. We, then, revisit existing operating system and networking mechanisms for TCP-based datacenter applications. We design an OS scheduler for ÎŒs-scale tasks, while we modify TCP to improve L4 load balancing, and provide an SLO-aware flow control mechanism. Finally, after identifying the problems related to TCP-based RPC services, we introduce a new transport protocol for datacenter RPCs and in-network policy enforcement that enables us to push functionality to the network. We showcase how the new protocol improves the performance and simplifies the implementation of in-network RPC load balancing, SLO-aware RPC flow control, and application-agnostic fault-tolerant RPCs.

Cybersecurity Solutions for Active Power Distribution Networks

Teklemariam Tsegay Tesfay

An active distribution network (ADN) is an electrical-power distribution network that implements a real-time monitoring and control of the electrical resources and the grid. Effective monitoring and control is realised by deploying a large number of sensing and actuating devices and a communication network to facilitates the two-way transfer of data. The reliance of ADN operations on a large number of electronic devices and on communication networks poses a challenge in protecting the system against cyber-attacks. Identifying these challenges and commissioning appropriate solutions is of utmost importance to realize the full potential of a smart grid that seamlessly integrates distributed generation, such as renewable energy sources. As a first step, we perform a thorough threat analysis of a typical ADN. We identify potential threats against field devices, the communication infrastructure and servers at control centers. We also propose a check-list of security solutions and best practices that guarantee a distribution network's resilient operation in the presence of malicious attackers, natural disasters, and other unintended failures that could potentially lead to islanded communication zone. For the next step, we investigate the security of MPLS-TP, a technology that is mainly used for long-distance inter-domain communication in smart grid. We find that an MPLS-TP implementation in Cisco IOS has serious security vulnerabilities in two of its protocols, BFD and PSC. These two protocols control protection-switching features in MPLS-TP. In our test-bed, we demonstrate that an attacker who has physical access to the network can exploit the vulnerabilities in order to inject forged BFD or PSC messages that affect the network's availability. Third, we consider multicast source authentication for synchrophasor data communication in grid monitoring systems (GMS). Ensuring source authentication without violating the stringent real-time requirement of GMS is challenging. Through an extensive review of existing schemes, we identified a set of schemes that satisfy some desirable requirements for GMS. The identified schemes are ECDSA, TV-HORS and Incomplete- key-set. We experimentally compared these schemes using computation, communication and key management overheads as performance metrics. A tweak in ECDSA's implementation to make it use pre-generated tokens to generate signatures significantly improves the computation overhead of ECDSA, making it the preferred scheme for GMS. This finding is contrary to the generally accepted view that asymmetric cryptography is inapplicable for real-time systems. Finally, we studied a planning problem that arises when a utility wants to roll out a software patch that requires rebooting to all PMUs while maintaining system observability. The problem we address is how to find a partitioning of the set of the deployed PMUs into as few subsets as possible such that all the PMUs in one subset can be patched in one round while all the PMUs in the other subsets provide full observability. We show that the problem is NP-complete in the general case and and formulated it as binary integer linear programming (BILP) problem. We have also provided an heuristic algorithm to find an approximate solution. Furthermore, we have identified a special case of the problem where the grid is a tree and provided a polynomial-time algorithm that finds an optimal patching plan that requires only two rounds to patch the PMUs.

EPFL2017

X.400 MHS: First Steps Towards an EDI Communication Standard (2nd publication)

Electronic Data Interchange (EDI) is an increasingly important application of computer communications. Until recently, EDI has developed independently of the communications technology and related standards. EDI users coped with this situation by setting up bilateral communication agreements or by using the services of Value Added Data Services centres, which provided protocol conversion. The introduction of X.400 MHS based systems and services is slowly modifying the general attitude towards EDI communications. For the first time, a CCITT rapporteur group is drafting a recommendation for EDI communications within the context of X.400 MHS. More importantly, there already exist official guidelines, both in the U.S.A. and in Europe, which explain how to take advantage of X.400 MHS systems today. The purpose of this paper is to comment upon these first approaches to an EDI communication standard. For this reason, and because we expect only a few readers to be familiar with both EDI and X.400 MHS, we begin with introductions to both subjects. Originally published in Computer Communication Review (ACM), vol 20, n. 2, April 1990.

IEEE Computer Society Press1992

R2P2: Making RPCs first-class datacenter citizens

Edouard Bugnion, Jonas Fietz, Adrien Ghosn, Evangelos Marios Kogias, Georgios Prekas

Remote Procedure Calls are widely used to connect datacenter applications with strict tail-latency service level objectives in the scale of µs. Existing solutions utilize streaming or datagram-based transport protocols for RPCs that impose overheads and limit the design flexibility. Our work exposes the RPC abstraction to the endpoints and the network, making RPCs first-class datacenter citizens and allowing for in-network RPC scheduling. We propose R2P2, a UDP-based transport protocol specifically designed for RPCs inside a datacenter. R2P2 exposes pairs of requests and responses and allows efficient and scalable RPC routing by separating the RPC target selection from request and reply streaming. Leveraging R2P2, we implement a novel join-bounded-shortest-queue (JBSQ) RPC load balancing policy, which lowers tail latency by centralizing pending RPCs in the router and ensures that requests are only routed to servers with a bounded number of outstanding requests. The R2P2 router logic can be implemented either in a software middlebox or within a P4 switch ASIC pipeline. Our evaluation, using a range of microbenchmarks, shows that the protocol is suitable for µs-scale RPCs and that its tail latency outperforms both random selection and classic HTTP reverse proxies. The P4-based implementation of R2P2 on a Tofino ASIC adds less than 1µs of latency whereas the software middlebox implementation adds 5µs latency and requires only two CPU cores to route RPCs at 10 Gbps linerate. R2P2 improves the tail latency of web index searching on a cluster of 16 workers operating at 50% of capacity by 5.7× over NGINX. R2P2 improves the throughput of the Redis key-value store on a 4-node cluster with master/slave replication for a tail-latency service-level objective of 200µs by more than 4.8× vs. vanilla Redis.

USENIX ASSOC2019

Cybersecurity Solutions for Active Power Distribution Networks

Teklemariam Tsegay Tesfay

EPFL2017