A Reinforcement Learning Approach to Train Routing and Scheduling

Good train scheduling for a big network with many trains is very hard to achieve. As the trains are competing for the tracks with one another, the number of constraints grows rapidly. Trying to take advantage of emerging technologies in the areas of optimization and machine learning, the Swiss Federal Railways have created a challenge to find the best algorithm to solve this problem. Two algorithms to solve this task were implemented in this project. First, a greedy strategy trying to schedule all trains at their earliest possible and then solving conflict after conflict was reproduced. It is shown that this algorithm is able to keep up with the best methods existing. With the objective of generalizing and improving the result, a policy gradient method is then added to take on the most critical decisions that have to be made in every iteration of the first algorithm. Even though this second enhanced algorithm does not obtain optimal solutions, it is able to outperform the first one in a generalized task. Hence, it can be shown that reinforcement learning is applicable to the train routing and scheduling task and, more generally, in environments with an open structure.

Representation Learning for Multi-relational Data

Eda Bayram

Recent years have witnessed a rise in real-world data captured with rich structural information that can be better depicted by multi-relational or heterogeneous graphs.However, research on relational representation learning has so far mostly focused on the problems arising in simple, homogeneous graphs. Integrating the structural priors provided by multi-relational data may further empower the generalization capacity of representation learning models, yet it still remains an open challenge.Although there is a strong line of works on relational machine learning on knowledge graphs, it is quite concentrated on the task ofcompleting missing edges, which is known as link prediction. In this thesis, we shift the focus away from the well-addressed node and graph classification problems on simple graphs or the link prediction problem on knowledge graphs, and prompt new research questions targeting the representation learning problems that are overlooked in multi-relational data.First, we focus on the problem of node regression on multi-relational graphs, noting that inference of continuous node features across a graph is rather under-studied in the current relational learning research.We propose a novel propagation method which aims to complete missing features at the nodes of a multi-relational and directed graph.Our multi-relational propagation algorithm is composed of iterative neighborhood aggregations which originate from a relational local generative model. Our findings show the benefit of exploiting the inductive bias led by the multi-relational structure of the data.Next, we consider the node attribute completion problem in knowledge graphs, which is relatively unexplored by the knowledge graph reasoning literature.We propose a novel multi-relational attribute propagation method where we harness not only the relational structure of the knowledge graph, but also the dependencies between various types of numerical node attributes relying on a heterogeneous feature space.Our algorithm is framed within a message-passing scheme where the propagation parameters are estimated in advance. We also propose an alternative semi-supervised learning framework where the parameters and the missing node attributes are inferred in an end-to-end fashion.Experimental results on well-known knowledge graph datasets relay the effectiveness of our message-passing approach, which specifies the computational graph by the heterogeneity of the data.Finally, we study graph learning in multi-relational data domain.Unlike the existing structure inference methods, we aim at exploiting and combining each source of relational information provided by the data domain to learn the underlying graph of a set of observations.For this purpose, we employ a multi-layer graph representation which encodes multiple types of relationships between data entities. Then, we propose a mask learning method to infer a specific combination of the layers which reveals the structure of observations.Experiments conducted both on simulated and real-world data suggest that incorporating multi-relational domain knowledge enhances structure inference by boosting its adaptability to a variety of input data conditions.

EPFL2021

Mean-Field methods for Structured Deep-Learning in Computer Vision

Pierre Bruno Baqué

In recent years, Machine Learning based Computer Vision techniques made impressive progress. These algorithms proved particularly efficient for image classification or detection of isolated objects. From a probabilistic perspective, these methods can predict marginals, over single or multiple variables, independently, with high accuracy. However, in many tasks of practical interest, we need to predict jointly several correlated variables. Practical applications include people detection in crowded scenes, image segmentation, surface reconstruction, 3D pose estimation and others. A large part of the research effort in today's computer-vision community aims at finding task-specific solutions to these problems, while leveraging the power of Deep-Learning based classifiers. In this thesis, we present our journey towards a generic and practical solution based on mean-field (MF) inference. Mean-field is a Statistical Physics-inspired method which has long been used in Computer-Vision as a variational approximation to posterior distributions over complex Conditional Random Fields. Standard mean-field optimization is based on coordinate descent and in many situations can be impractical. We therefore propose a novel proximal gradient-based approach to optimizing the variational objective. It is naturally parallelizable and easy to implement. We prove its convergence, and then demonstrate that, in practice, it yields faster convergence and often finds better optima than more traditional mean-field optimization techniques. Then, we show that we can replace the fully factorized distribution of mean-field by a weighted mixture of such distributions, that similarly minimizes the KL-Divergence to the true posterior. Our extension of the clamping method proposed in previous works allows us to both produce a more descriptive approximation of the true posterior and, inspired by the diverse MAP paradigms, fit a mixture of mean-field approximations. We demonstrate that this positively impacts real-world algorithms that initially relied on mean-fields. One of the important properties of the mean-field inference algorithms is that the closed-form updates are fully differentiable operations. This naturally allows to do parameter learning by simply unrolling multiple iterations of the updates, the so-called back-mean-field algorithm. We derive a novel and efficient structured learning method for multi-modal posterior distribution based on the Multi-Modal Mean-Field approximation, which can be seamlessly combined to modern gradient-based learning methods such as CNNs. Finally, we explore in more details the specific problem of structured learning and prediction for multiple-people detection in crowded scenes. We then present a mean-field based structured deep-learning detection algorithm that provides state of the art results on this dataset.

EPFL2018

Automated Design: A Journey Across Modelling, Optimization, and Education

Cyril Picard

Machine intelligence greatly impacts almost all domains of our societies. It is profoundly changing the field of mechanical engineering with new technical possibilities and processes. The education of future engineers also needs to adapt in terms of techniques and even skills. Using the design of electro-mechanical actuators as a common thread, this work explores the many-facets of automated design: modeling, optimization, and education, and looks for the prerequisites essential to its successful application. The journey starts by building a modular and integrated model. It focuses on the prediction of system-level specifications that yield high added-value for decision-makers and shorten the path from the model to the final product. Combined with multi-objective evolutionary algorithms (MOEAs) and visualization tools, the model forms an automated design tool that helps engineers and decision-makers to rapidly get important insights into their design task. Its potential and benefits are validated through two specific applications. The results, however, also highlight a gap between the reported performance of optimizers on common benchmark problems and the actual performance on these problems. To further develop optimizers, appropriate and realistic benchmark problems are needed. A subset of the integrated design model is used to formulate a new test suite called MODAct, composed of 20 constrained multi-objective optimization problems (CMOPs) with variable levels of complexity. In addition, numerical approaches to evaluate the constraint landscape of CMOPs are introduced and applied to identify the differences in features of MODAct against 45 benchmark problems from literature. Further, the convergence performance of three algorithms on the same problems highlights the key role of constraints and, in particular, the number of simultaneously violated constraints in MODAct problems. In a next step, existing constraint handling strategies suitable for MOEAs along with a newly proposed technique for many-constraint problems are evaluated. Their parameters are tuned for different problems. The performance of the various configurations further highlights the difference between MODAct and other benchmark problems and show the highly competitive results of the proposed constraint handling technique on realistic design problems. As the technical limits are removed, the impact of automated design on the work of future engineers should be considered. On the one hand, the development of professional skills by students working on team project in different settings has been evaluated thanks to 205 students from three classes. Explicitly addressing these skills within the project seems key to support stronger and broader learning, suggesting changes that do not require a full curriculum redesign. On the other hand, nine groups (33 students) have been asked to design an actuator using a conventional approach followed by an automated design approach. The actuators suggested by students using the automated tool outperform the designs obtained through the traditional approach. Six groups even suggest solutions cheaper, three of which are also smaller, than the product of experienced industry engineers. Students proved thus capable of leveraging the tool within a short time. The analysis of their mistakes suggests possible improvements for future tools. As these students leave university, they carry the hope to see such methods spread in industry.

EPFL2021

Automated Design: A Journey Across Modelling, Optimization, and Education

Cyril Picard

EPFL2021

Mean-Field methods for Structured Deep-Learning in Computer Vision

Pierre Bruno Baqué

EPFL2018

Representation Learning for Multi-relational Data

Eda Bayram

EPFL2021