Scalable Multi-agent Coordination and Resource Sharing

A plethora of real world problems consist of a number of agents that interact, learn, cooperate, coordinate, and compete with others in ever more complex environments. Examples include autonomous vehicles, robotic agents, intelligent infrastructure, IoT devices, and so on. As more and more autonomous agents are deployed in the real-world, it will bring forth the need for novel algorithms, theory, and tools to enable coordination on a massive scale. In this thesis, we develop such tools to tackle two central challenges in multi-agent coordination research: solving allocation problems, and resource sharing, focusing on solutions that are scalable, practical, and applicable to real-world problems.In the first part of the thesis we tackle the problem of allocating resources to agents, i.e., solving a weighted matching problem. Real-world matching problems may occur in massively large systems, they are distributed and information-restrictive, and individuals have to reveal their preferences over the possible matches in order to get a high quality match, which brings forth significant privacy risks. As such, there are three main challenges: complexity, communication, and privacy.Our proposed approach, ALMA, is a practical heuristic designed for real-world, large-scale ( $10^6$ agents) applications. It is based on a simple altruistic behavioral convention: agents have a higher probability to back-off from contesting a resource if they have good alternatives, potentially freeing the resource for some agent that does not. ALMA tackles all of the aforementioned challenges: it is decentralized, runs on-device, requires no inter-agent communication, converges in constant time -- under reasonable assumptions --, and provides strong, worst-case, privacy guarantees. Moreover, by incorporating learning we can mitigate the loss in social welfare and increase fairness. Finally, rational agents can use such simple conventions, along with an arbitrary signal from the environment, to learn a correlated equilibrium for accessing a set resources, under high congestion.In the second part of the thesis we focus on a critical open problem: the question of cooperation in socio-ecological and socio-economical systems, and sustainability in the use of common-pool resources. In recent years, learning agents, especially deep reinforcement learning agents, have become ubiquitous in such systems. Yet, scaling to environments with a large number of agents and low observability continues to be a challenge. In our work, we focus on common-pool resources. Individuals face strong incentives to appropriate, which results in overuse and even the depletion of the resources. Our goal is to apply simple interventions to steer the population to desirable states.We propose a simple, yet powerful, and robust technique: allow agents to observe an arbitrary common signal from the environment. The agents learn to couple their policies, and avoid depletion in a wider range of settings, while achieving higher social welfare and convergence speed.Finally, we propose a practical approach to computing market prices and allocations via a deep reinforcement learning policymaker agent. Compared to the idealized market equilibrium outcome -- which can not always be efficiently computed -- our policymaker is much more flexible, allowing us to tune the prices with regard to diverse objectives such as sustainability and resource wastefulness, fairness, buyers' and sellers' welfare, etc.

Communication in a Swarm of Miniature Robots: The e-Puck as an Educational Tool for Swarm Robotics

Christopher Cianci, Alcherio Martinoli, James Pugh, Xavier Raemy

Swarm intelligence, and swarm robotics in particular, are reaching a point where leveraging the potential of communication within an artificial systempromises to uncover newand varied directions for interesting research without compromising the key properties of swarmintelligent systems such as self-organization, scalability, and robustness. However, the physical constraints of using radios in a robotic swarm are hardly obvious, and the intuitive models often used for describing such systems do not always capture them with adequate accuracy. In order to demonstrate this effectively in the classroom, certain tools can be used, including simulation and real robots. Most instructors currently focus on simulation, as it requires significantly less investment of time, money, and maintenance—but to really understand the differences between simulation and reality, it is also necessary to work with the real platforms from time to time. To our knowledge, our coursemay be the only one in the world where individual students are consistently afforded the opportunity to work with a networked multi-robot system on a tabletop. The e-Puck,1 a low-cost small-scale mobile robotic platform designed for educational use, allows us bringing real robotic hardware into the classroom in numbers sufficient to demonstrate and teach swarm-robotic concepts.We present here a custom module for local radio communication as a stackable extension board for the e-Puck, enabling information exchange between robots and also with any other IEEE 802.15.4-compatible devices. Transmission power can be modified in software to yield effective communication ranges as small as fifteen centimeters. This intentionally small range allows us to demonstrate interesting collective behavior based on local information and control in a limited amount of physical space, where ordinary radios would typically result in a completely connected network. Here we show the use of this module facilitating a collective decision among a group of 10 robots.

2007

Protecting Privacy in Multi-agent Optimization

Eric Zbinden

Many real-life optimization problems involve multiple entities, or agents (individuals, companies...), with their own private constraints and preferences, communicating with each other in order to find a solution that maximizes the overall public satisfaction. In Artificial Intelligence, the field of Distributed Constraint Optimization (DCOP) has been addressing such multi-agent optimization problems, through distributed message-passing algorithms such as the DPOP algorithm. However, the research in DCOP has been neglecting the privacy of the information exchanged by the agents during the computing of the solution, which is critical to many real-life problems. While agents are willing to cooperate with each other to produce an optimal solution, they are most often reluctant to reveal their private constraints and preferences to other, which hinders this cooperation. The goal of this project was to implement, test, and evaluate a secured version of DPOP, P-DPOP. P-DPOP provides strong agent privacy and topology privacy by randomization, constraints privacy and limited decision privacy by obfuscation. The algorithm was implemented in Java, as part of the open-source FRODO platform for DCOP.

2010

Decentralized multi-robot coordination in crowded workspaces

Laleh Makarem

The coordination of multi-robot systems is becoming one of the most important areas of research in robotics, mostly because it is required by numerous complex applications. These applications range from intelligent transportation systems, search and rescue robots, and medical robots, to cosmology and astrophysics. The coordination of multi-robot systems is based upon cooperation. The actions performed by each robot take into account the actions executed by the others in such a way that the whole system can operate coherently and efficiently. Regardless of the application, coordination is the key to the successful design and implementation of multi-robot systems. The number of robots involved in the aforementioned applications is increasing along with advances in miniaturization and automation. Consequently, a large number of robots need to share a workspace. This crowded workspace introduces new challenges into the coordination problem by increasing the risk of collision. To take into account communication constraints and sensor ranges, robots rely on local information. Therefore, efficient but simple coordination algorithms are required. This thesis investigates decentralized approaches based on navigation functions for the coordination of multi-robot systems in crowded workspaces. Decentralization allows robots to rely on local information, guarantees scalability and enables real-time deployment. Navigation functions are a special category of potential functions. Their negated gradient vector-field is attractive towards the goal and repulsive with respect to fixed or moving obstacles to avoid collision. In the first part of the thesis, we present the multi-robot coordination problem using navigation functions in a game-theory based framework. We propose a motion model along with a control law that leads the robots to a Nash equilibrium. The existence of the Nash equilibrium enables navigation functions to be exploited for studying, building, and running coordination frameworks for multi-robot systems. In the second part, we address the coordination of autonomous vehicles at intersections. A novel decentralized navigation function is proposed. It guarantees collision-free crossing of autonomous vehicles modeled as first order dynamic systems. The inertia of the vehicles is also introduced in the navigation functions to ensure deadlock-free coordination. The proposed approach does not require adaptation of the road infrastructure and relies upon onboard vehicles sensor data. Compared with traffic lights and roundabouts, the proposed method significantly reduces the travel time and the number of stops, thus decreasing energy consumption and pollution emission. This provides a strong motivation to pursue efforts towards the deployment of autonomous vehicles on roads. In the third part of the thesis, we investigate a coordination framework for a large number of miniaturized fiber positioner robots. The fiber positioner robots are designed and built as parts of the next generation of telescopes enabling large spectroscopic surveys. The proposed decentralized framework ensures the collision-free coordination of the fiber positioners sharing a crowed workspace at the focal plate of the telescope. The dynamical (max speed) and the mechanical (limited actuation range) constraints of the positioners are taken into account in the proposed coordination approach, which significantly reduces the time to reach a new robot configuration.