Definitive Consensus for Distributed Data Inference

Leonidas Georgopoulos
EPFL, 2011
Thèse EPFL

Résumé

Inference from data is of key importance in many applications of informatics. The current trend in performing such a task of inference from data is to utilise machine learning algorithms. Moreover, in many applications that it is either required or is preferable to infer from the data in a distributed manner. Many practical difficulties arise from the fact that in many distributed applications we avert from transferring data or parts of it due to costs, privacy and computation considerations. Admittedly, it would be advantageous if the final knowledge, attained through distributed data inference, is common to every participating computing node. The key in achieving the aforementioned task is the distributed average consensus algorithm or simply the consensus algorithm herein. The latter has been used in many applications. Initially the main purpose has been for the estimation of the expectation of scalar valued data distributed over a network of machines without a central node. Notably, the algorithm allows the final outcome to be the same for every participating node. Utilising the consensus algorithm as the centre piece makes the task of distributed data inference feasible. However, there are many difficulties that hinder its direct applicability. Thus, we concentrate on the consensus algorithm with the purpose of addressing these difficulties. There are two main concerns. First, the consensus algorithm has asymptotic convergence. Thus, we may only achieve maximum accuracy if the algorithm is left to run for a large number of iterations. Second, the accuracy attained at any iteration during the consensus algorithm is correlated with the standard deviation of the initial value distribution. The consensus algorithm is inherently imprecise at finite time and this hardens the learning process. We solve this problem by introducing the definitive consensus algorithm. This algorithm attains maximum precision in a finite number of iterations, namely in a number of iterations equal to the diameter of the graph in a distributed and decentralised manner. Additionally, we introduce the nonlinear consensus algorithm and the adaptive consensus algorithm. These are modifications of the original consensus algorithm that allow improved precision with fewer iterations in cases of unknown, partially known and stochastically time-varying network topologies. The definitive consensus algorithm can be incorporated in a distributed data inference framework. We approach the problem of data inference from the perspective of machine learning. Specifically, we tailor this distributed inference framework for machine learning on a communication network with data partitioned on the participating computing nodes. Particularly, the distributed data inference framework is detailed and applied to the case of a multilayer feed forward neural network with error back-propagation. A substantial examination of its performance and its comparison with the non-distributed case, is provided. Theoretical foundation for the definitive consensus algorithm is provided. Moreover, its superior performance is validated by numerical experiments. A brief theoretical examination of the nonlinear and the adaptive consensus algorithms is performed to justify their improved performance with respect to the original consensus algorithm. Moreover, extensive numerical simulations are given to compare the nonlinear and the adaptive algorithm with the original consensus algorithm. The most important contributions of this research are principally the definitive consensus algorithm and the distributed data inference framework. Their combination yields a decentralised distributed process over a communication network capable for inference in agreement over the entire network.

Official source

https://infoscience.epfl.ch/record/164020?ln=fr

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Concepts associés

Chargement

Publications associées

Chargement

Leonidas Georgopoulos
EPFL, 2011
Thèse EPFL

Résumé

Official source

https://infoscience.epfl.ch/record/164020?ln=fr

À propos de ce résultat

Concepts associés (9)

Concepts associés

Chargement

Publications associées (3)

Publications associées

Chargement

Prior knowledge in Kernel methods

Alexei Pozdnoukhov

Machine Learning is a modern and actively developing field of computer science, devoted to extracting and estimating dependencies from empirical data. It combines such fields as statistics, optimization theory and artificial intelligence. In practical tasks, the general aim of Machine Learning is to construct algorithms able to generalize and predict in previously unseen situations based on some set of examples. Given some finite information, Machine Learning provides ways to exract knowledge, describe, explain and predict from data. Kernel Methods are one of the most successful branches of Machine Learning. They allow applying linear algorithms with well-founded properties such as generalization ability, to non-linear real-life problems. Support Vector Machine is a well-known example of a kernel method, which has found a wide range of applications in data analysis nowadays. In many practical applications, some additional prior knowledge is often available. This can be the knowledge about the data domain, invariant transformations, inner geometrical structures in data, some properties of the underlying process, etc. If used smartly, this information can provide significant improvement to any data processing algorithm. Thus, it is important to develop methods for incorporating prior knowledge into data-dependent models. The main objective of this thesis is to investigate approaches towards learning with kernel methods using prior knowledge. Invariant learning with kernel methods is considered in more details. In the first part of the thesis, kernels are developed which incorporate prior knowledge on invariant transformations. They apply when the desired transformation produce an object around every example, assuming that all points in the given object share the same class. Different types of objects, including hard geometrical objects and distributions are considered. These kernels were then applied for images classification with Support Vector Machines. Next, algorithms which specifically include prior knowledge are considered. An algorithm which linearly classifies distributions by their domain was developed. It is constructed such that it allows to apply kernels to solve non-linear tasks. Thus, it combines the discriminative power of support vector machines and the well-developed framework of generative models. It can be applied to a number of real-life tasks which include data represented as distributions. In the last part of the thesis, the use of unlabelled data as a source of prior knowledge is considered. The technique of modelling the unlabelled data with a graph is taken as a baseline from semi-supervised manifold learning. For classification problems, we use this apporach for building graph models of invariant manifolds. For regression problems, we use unlabelled data to take into account the inner geometry of the input space. To conclude, in this thesis we developed a number of approaches for incorporating some prior knowledge into kernel methods. We proposed invariant kernels for existing algorithms, developed new algorithms and adapted a technique taken from semi-supervised learning for invariant learning. In all these cases, links with related state-of-the-art approaches were investigated. Several illustrative experiments were carried out on real data on optical character recognition, face image classification, brain-computer interfaces, and a number of benchmark and synthetic datasets.

EPFL2006

Chargement

École Polytechnique Fédérale de Lausanne2006

Chargement

Advancing computational and data-driven methods for the design and discovery of nanoporous materials

Seyedmohamad Moosavi

Metal-organic framework (MOFs) and related nanoporous materials have emerged as promising candidates for a variety of applications, such as gas separation and storage, catalysis, sensing, etc. Their building block structure allows us to generate a huge number of distinct materials only by changing the metal nodes and organic linkers. This, in principle, allows the design and discovery of materials that perform optimally for a given application. However, the conventional process of material development, from discovery to synthesis and performance evaluation, is too slow and expensive for exploring this enormous pool of materials. Complimentary methods are therefore needed to accelerate this process. The aim of this thesis is to investigate and expand the computational and data-driven methods for the development of nanoporous materials for gas separation and storage. The prevailing use of these methods is for high-throughput screening of the materials for a target application. However, the capability and success of such a screening approach depend on fast, reliable, and accurate prediction of material properties, as well as on the effective exploration of the chemical space. Therefore, in this thesis, we develop material descriptors for the chemistry and pore geometry of MOFs, which allow us to use machine learning to rapidly evaluate their adsorption properties with high accuracy. We next introduce a methodology to quantify the diversity of material databases to assess how well the chemical space is explored when a given material database is screened. We illustrate the importance of this diversity analysis by showing how the lack of diversity in MOF databases hinders material discovery, leads to chemical insights that are not generalisable and makes machine learning models not transferable. The promising materials discovered in a screening study are only of interest if they can be synthesised and are sufficiently stable to withstand the operating conditions of the corresponding application. Therefore, we investigate the applicability and capability of computational and data-driven methods to address some of the challenges in the material synthesis and the mechanical stability of MOFs. Material synthesis still mainly rests on the chemical intuition of synthetic chemists. Here, we introduce a method using machine learning and a genetic algorithm to capture this chemical intuition for MOF synthesis. We demonstrate how this simple approach can be powerful for guiding the synthesis of new materials. Lastly, we study how the mechanical stability of MOFs, which is fundamentally important for most of their practical applications, is affected by its underlying structure, i.e., the framework bonding topology and ligand structure. We show how this understanding can be used to develop strategies to design MOFs with enhanced mechanical stability.

EPFL2020

Chargement

Advancing computational and data-driven methods for the design and discovery of nanoporous materials

Seyedmohamad Moosavi

EPFL2020

Prior knowledge in Kernel methods

Alexei Pozdnoukhov

EPFL2006

École Polytechnique Fédérale de Lausanne2006