Multi-agent Reinforcement Learning for Assembly of a Spanning Structure

In this master thesis, multi-agent reinforcement learning is used to teach robots to build a self-supporting structure connecting two points. To accomplish this task, a physics simulator is first designed using linear programming. Then, the task of building a self-supporting structure is modeled as a Markov game, where the robot arms correspond to the agents of the game. This formalism is then used to design learning agents and train them using deep reinforcement learning. Two different types of deep neural network models, based on image analysis and graph theory, respectively, are used to develop their policy. The agents are then trained either centrally or distributively to compare their learning processes and weaknesses. In a final experiment, the efficiency of the learning algorithm Soft Actor-Critic, is compared to Advantage Actor-Critic, highlighting the effectiveness of using Shannon entropy to search through the policy space. Finally, the training procedure allows agents to successfully build a structure that spans ten times the width of the building blocks without the need to use any binding between them or a removable scaffold during assembly.