Solving consensus | EPFL Graph Search

Distributed systems are the basis of widespread computing facilities enabling many of our daily life activities. Telebanking, electronic commerce, online booking-reservation, and telecommunication are examples of common services that rely on distributed systems in order to achieve their desirable ubiquity. The success of these services increasingly depends on fault-tolerant systems to preserve their availability and reliability despite partial failures. Fault tolerance can be achieved by introducing redundancy into the system. This leads to the need of coordinating replicated components which has been one of the biggest trends in software technologies. This thesis is about process coordination in fault-tolerant distributed systems. In particular, it focuses on the problem of reaching agreement among processes considering the well-known Consensus problem. Consensus is regarded as an abstract problem whose solutions embody the complexity inherent to most agreement problems. This thesis studies the solvability of consensus in asynchronous network systems augmented with a failure detector oracle. Both, the processes and the network, are subject to benign failures. The thesis' overall contribution is a consensus algorithm that tolerates process crash and network omission failures, and exploits the recovery of crashed process. A contribution of this thesis is the definition of a novel type of weakly reliable communication channels, called Stubborn. The semantics of Stubborn channels reflect the reliability on communication required to solve consensus. Furthermore, it is shown that an algorithm designed with Stubborn in-stead of reliable channels can offer the same number of messages exchanged in favorable runs and avoid communication steps in disadvantageous runs. Stubborn channels can be easily implemented on top of existing network layers and make reasonable requirements on message buffering space. To handle the crash and recovery of processes it is proposed a model which encompasses the different patterns of crashes and recoveries that can be exhibited by the processes. The approach considers two different problems concerning crash and recovery: (1) processes that can be disruptive because they are always crashing and recovering, and (2) amnesia failures caused by crashes. The solution for the first problem consists in the use of a new class of failure detectors distinguished by its recurrent strong completeness property. To cope with the crash and recovery of any process involved, process amnesia cannot be arbitrary and thus processes are equipped with a stable storage device. The proposed algorithm judiciously specifies accesses to stable storage, and by exploiting the recovery of processes it can solve consensus in scenarios where other algorithms would otherwise block.

Signaling and Reciprocity

Wojciech Galuba

Complex networks exist for a number of purposes. The neural, metabolic and food networks ensure our survival, while the social, economic, transportation and communication networks allow us to prosper. Independently of the purposes and particularities of the physical embodiment of the networks, one of their fundamental functions is the delivery of information from one part of the network to another. Gossip and diseases diffuse in the social networks, electrochemical signals propagate in the neural networks and data packets travel in the Internet. Engineering networks for robust information flows is a challenging task. First, the mechanism through which the network forms and changes its topology needs to be defined. Second, within a given topology, the information must be routed to the appropriate recipients. Third, both the network formation and the routing mechanisms need to be robust against a wide spectrum of failures and adversaries. Fourth, the network formation, routing and failure recovery must operate under the resource constraints, either intrinsic or extrinsic to the network. Finally, the autonomously operating parts of the network must be incentivized to contribute their resources to facilitate the information flows. This thesis tackles the above challenges within the context of several types of networks: 1) peer-to-peer overlays – computers interconnected over the Internet to form an overlay in which participants provide various services to one another, 2) mobile ad-hoc networks – mobile nodes distributed in physical space communicating wirelessly with the goal of delivering data from one part of the network to another, 3) file-sharing networks – networks whose participants interconnect over the Internet to exchange files, 4) social networks – humans disseminating and consuming information through the network of social relationships. The thesis makes several contributions. Firstly, we propose a general algorithm, which given a set of nodes embedded in an arbitrary metric space, interconnects them into a network that efficiently routes information. We apply the algorithm to the peer-to-peer overlays and experimentally demonstrate its high performance, scalability as well as resilience to continuous peer arrivals and departures. We then shift our focus to the problem of the reliability of routing in the peer-to-peer overlays. Each overlay peer has limited resources and when they are exhausted this ultimately leads to delayed or lost overlay messages. All the solutions addressing this problem rely on message redundancy, which significantly increases the resource costs of fault-tolerance. We propose a bandwidth-efficient single-path Forward Feedback Protocol (FFP) for overlay message routing in which successfully delivered messages are followed by a feedback signal to reinforce the routing paths. Internet testbed evaluation shows that FFP uses 2-5 times less network bandwidth than the existing protocols relying on message redundancy, while achieving comparable fault-tolerance levels under a variety of failure scenarios. While the Forward Feedback Protocol is robust to message loss and delays, it is vulnerable to malicious message injection. We address this and other security problems by proposing Castor, a variant of FFP for mobile ad-hoc networks (MANETs). In Castor, we use the same general mechanism as in FFP; each time a message is routed, the routing path is either enforced or weakened by the feedback signal depending on whether the routing succeeded or not. However, unlike FFP, Castor employs cryptographic mechanisms for ensuring the integrity and authenticity of the messages. We compare Castor to four other MANET routing protocols. Despite Castor's simplicity, it achieves up to 40% higher packet delivery rates than the other protocols and recovers at least twice as fast as the other protocols in a wide range of attacks and failure scenarios. Both of our protocols, FFP and Castor, rely on simple signaling to improve the routing robustness in peer-to-peer and mobile ad-hoc networks. Given the success of the signaling mechanism in shaping the information flows in these two types of networks, we examine if signaling plays a similar crucial role in the on-line social networks. We characterize the propagation of URLs in the social network of Twitter. The data analysis uncovers several statistical regularities in the user activity, the social graph, the structure of the URL cascades as well as the communication and signaling dynamics. Based on these results, we propose a propagation model that accurately predicts which users are likely to mention which URLs. We outline a number of applications where the social network information flow modelling would be crucial: content ranking and filtering, viral marketing and spam detection. Finally, we consider the problem of freeriding in peer-to-peer file-sharing applications, when users can download data from others, but never reciprocate by uploading. To address the problem, we propose a variant of the BitTorrent system in which two peers are only allowed to connect if their owners know one another in the real world. When the users know which other users their BitTorrent client connects to, they are more likely to cooperate. The social network becomes the content distribution network and the freeriding problem is solved by leveraging the social norms and reciprocity to stabilize cooperation rather than relying on technological means. Our extensive simulation shows that the social network topology is an efficient and scalable content distribution medium, while at the same time provides robustness to freeriding.

EPFL2011

Artificial immune system for the Internet

Slavisa Sarafijanovic

We investigate the usability of the Artificial Immune Systems (AIS) approach for solving selected problems in computer networks. Artificial immune systems are created by using the concepts and algorithms inspired by the theory of how the Human Immune System (HIS) works. We consider two applications: detection of routing misbehavior in mobile ad hoc networks, and email spam filtering. In mobile ad hoc networks the multi-hop connectivity is provided by the collaboration of independent nodes. The nodes follow a common protocol in order to build their routing tables and forward the packets of other nodes. As there is no central control, some nodes may defect to follow the common protocol, which would have a negative impact on the overall connectivity in the network. We build an AIS for the detection of routing misbehavior by directly mapping the standard concepts and algorithms used for explaining how the HIS works. The implementation and evaluation in a simulator shows that the AIS mimics well most of the effects observed in the HIS, e.g. the faster secondary reaction to the already encountered misbehavior. However, its effectiveness and practical usability are very constrained, because some particularities of the problem cannot be accounted for by the approach, and because of the computational constrains (reported also in AIS literature) of the used negative selection algorithm. For the spam filtering problem, we apply the AIS concepts and algorithms much more selectively and in a less standard way, and we obtain much better results. We build the AIS for antispam on top of a standard technique for digest-based collaborative email spam filtering. We notice un advantageous and underemphasized technological difference between AISs and the HIS, and we exploit this difference to incorporate the negative selection in an innovative and computationally efficient way. We also improve the representation of the email digests used by the standard collaborative spam filtering scheme. We show that this new representation and the negative selection, when used together, improve significantly the filtering performance of the standard scheme on top of which we build our AIS. Our complete AIS for antispam integrates various innate and adaptive AIS mechanisms, including the mentioned specific use of the negative selection and the use of innate signalling mechanisms (PAMP and danger signals). In this way the AIS takes into account users' profiles, implicit or explicit feedback from the users, and the bulkiness of spam. We show by simulations that the overall AIS is very good both in detecting spam and in avoiding misdetection of good emails. Interestingly, both the innate and adaptive mechanisms prove to be crucial for achieving the good overall performance. We develop and test (within a simulator) our AIS for collaborative spam filtering in the case of email communications. The solution however seems to be well applicable to other types of Internet communications: Internet telephony, chat/sms, forum, news, blog, or web. In all these cases, the aim is to allow the wanted communications (content) and prevent those unwanted from reaching the end users and occupying their time and communication resources. The filtering problems, faced or likely to be faced in the near future by these applications, have or are likely to have the settings similar to those that we have in the email case: need for openness to unknown senders (creators of content, initiators of the communication), bulkiness in receiving spam (many recipients are usually affected by the same spam content), tolerance of the system to a small damage (to small amounts of unfiltered spam), possibility to implicitly or explicitly and in a cheap way obtain a feedback from the recipients about the damage (about spam that they receive), need for strong tolerance to wanted (non-spam) content. Our experiments with the email spam filtering show that our AIS, i.e. the way how we build it, is well fitted to such problem settings.

EPFL2008

Cybersecurity Solutions for Active Power Distribution Networks

Teklemariam Tsegay Tesfay

An active distribution network (ADN) is an electrical-power distribution network that implements a real-time monitoring and control of the electrical resources and the grid. Effective monitoring and control is realised by deploying a large number of sensing and actuating devices and a communication network to facilitates the two-way transfer of data. The reliance of ADN operations on a large number of electronic devices and on communication networks poses a challenge in protecting the system against cyber-attacks. Identifying these challenges and commissioning appropriate solutions is of utmost importance to realize the full potential of a smart grid that seamlessly integrates distributed generation, such as renewable energy sources. As a first step, we perform a thorough threat analysis of a typical ADN. We identify potential threats against field devices, the communication infrastructure and servers at control centers. We also propose a check-list of security solutions and best practices that guarantee a distribution network's resilient operation in the presence of malicious attackers, natural disasters, and other unintended failures that could potentially lead to islanded communication zone. For the next step, we investigate the security of MPLS-TP, a technology that is mainly used for long-distance inter-domain communication in smart grid. We find that an MPLS-TP implementation in Cisco IOS has serious security vulnerabilities in two of its protocols, BFD and PSC. These two protocols control protection-switching features in MPLS-TP. In our test-bed, we demonstrate that an attacker who has physical access to the network can exploit the vulnerabilities in order to inject forged BFD or PSC messages that affect the network's availability. Third, we consider multicast source authentication for synchrophasor data communication in grid monitoring systems (GMS). Ensuring source authentication without violating the stringent real-time requirement of GMS is challenging. Through an extensive review of existing schemes, we identified a set of schemes that satisfy some desirable requirements for GMS. The identified schemes are ECDSA, TV-HORS and Incomplete- key-set. We experimentally compared these schemes using computation, communication and key management overheads as performance metrics. A tweak in ECDSA's implementation to make it use pre-generated tokens to generate signatures significantly improves the computation overhead of ECDSA, making it the preferred scheme for GMS. This finding is contrary to the generally accepted view that asymmetric cryptography is inapplicable for real-time systems. Finally, we studied a planning problem that arises when a utility wants to roll out a software patch that requires rebooting to all PMUs while maintaining system observability. The problem we address is how to find a partitioning of the set of the deployed PMUs into as few subsets as possible such that all the PMUs in one subset can be patched in one round while all the PMUs in the other subsets provide full observability. We show that the problem is NP-complete in the general case and and formulated it as binary integer linear programming (BILP) problem. We have also provided an heuristic algorithm to find an approximate solution. Furthermore, we have identified a special case of the problem where the grid is a tree and provided a polynomial-time algorithm that finds an optimal patching plan that requires only two rounds to patch the PMUs.

EPFL2017