Mutual information measures for subclass error-correcting output codes classification

Error-Correcting Output Codes (ECOCs) reveal a common way to model multi-class classification problems. According to this state of the art technique, a multi-class problem is decomposed into several binary ones. Additionally, on the ECOC framework we can apply the subclasses technique (sub-ECOC), where by splitting the initial classes of the problem we aim to the creation of larger but easier to solve ECOC configurations. The multi-class problem's decomposition is achieved via a searching procedure known as sequential forward floating search (SFFS). The SFFS algorithm in each step searches for the optimum binary separation of the classes that compose the multi-class problem. The separation decision is based on the maximization or minimization of a criterion function. The standard criterion used is the maximization of the mutual information (MI) between the bi-partitions created in each step of the SFFS. The materialization of the MI measure is achieved by a method called fast quadratic Mutual Information (FQMI). Although FQMI is quite accurate in modelling the MI, its computation is of high algorithmic complexity, which as a consequence makes the ECOC and sub-ECOC techniques applicable only on small datasets. In this paper we present some alternative separation criteria of reduced computational complexity that can be used in the SFFS algorithm. Furthermore, we compare the performance of these criteria over several multi-class classification problems. © Springer-Verlag Berlin Heidelberg 2010.

Subclass error correcting output codes using fisher's linear discriminant ratio

Nikolaos Arvanitopoulos Darginis

Error-Correcting Output Codes (ECOC) with subclasses reveal a common way to solve multi-class classification problems. According to this approach, a multiclass problem is decomposed into several binary ones based on the maximization of the mutual information (MI) between the classes and their respective labels. The MI is modelled through the fast quadratic mutual information (FQMI) procedure. However, FQMI is not applicable on large datasets due to its high algorithmic complexity. In this paper we propose Fisher's Linear Discriminant Ratio (FLDR) as an alternative decomposition criterion which is of much less computational complexity and achieves in most experiments conducted better classification performance. Furthermore, we compare FLDR against FQMI for facial expression recognition over the Cohn-Kanade database. © 2010 IEEE.

2010

Towards a theoretical foundation for morphological computation with compliant bodies

Auke Ijspeert, Rolf Pfeifer

The control of compliant robots is, due to their often nonlinear and complex dynamics, inherently difficult. The vision of morphological computation proposes to view these aspects not only as problems, but rather also as parts of the solution. Non-rigid body parts are not seen anymore as imperfect realizations of rigid body parts, but rather as potential computational resources. The applicability of this vision has already been demonstrated for a variety of complex robot control problems. Nevertheless, a theoretical basis for understanding the capabilities and limitations of morphological computation has been missing so far. We present a model for morphological computation with compliant bodies, where a precise mathematical characterization of the potential computational contribution of a complex physical body is feasible. The theory suggests that complexity and nonlinearity, typically unwanted properties of robots, are desired features in order to provide computational power. We demonstrate that simple generic models of physical bodies, based on mass-spring systems, can be used to implement complex nonlinear operators. By adding a simple readout (which is static and linear) to the morphology, such devices are able to emulate complex mappings of input to output streams in continuous time. Hence, by outsourcing parts of the computation to the physical body, the difficult problem of learning to control a complex body, could be reduced to a simple and perspicuous learning task, which can not get stuck in local minima of an error function.

2011

Re-proving Channel Polarization Theorems

Mine Alsan

The general subject considered in this thesis is a recently discovered coding technique, polar coding, which is used to construct a class of error correction codes with unique properties. In his ground-breaking work, Arikan proved that this class of codes, called polar codes, achieve the symmetric capacity --- the mutual information evaluated at the uniform input distribution ---of any stationary binary discrete memoryless channel with low complexity encoders and decoders requiring in the order of

O(N\log N)

operations in the block-length

N

. This discovery settled the long standing open problem left by Shannon of finding low complexity codes achieving the channel capacity. Polar codes are not only appealing for being the first to 'close the deal'. In contrast to most of the existing coding schemes, polar codes admit an explicit low complexity construction. In addition, for symmetric channels, the polar code construction is deterministic; the theoretically beautiful but practically limited "average performance of an ensemble of codes is good, so there must exist one particular code in the ensemble at least as good as the average'' formalism of information theory is bypassed. Simulations are thus not necessary in principle for evaluating the error probability which is shown in a study by Telatar and Arikan to scale exponentially in the square root of the block-length. As such, at the time of this writing, polar codes are appealing for being the only class of codes proved, and proved with mathematical elegance, to possess all of these properties. Polar coding settled an open problem in information theory, yet opened plenty of challenging problems that need to be addressed. This novel coding scheme is a promising method from which, in addition to data transmission, problems such as data compression or compressed sensing, which includes all types of measurement processes like the MRI or ultrasound, could benefit in terms of efficiency. To make this technique fulfill its promise, the original theory has been, and should still be, extended in multiple directions. A significant part of this thesis is dedicated to advancing the knowledge about this technique in two directions. The first one provides a better understanding of polar coding by generalizing some of the existing results and discussing their implications, and the second one studies the robustness of the theory over communication models introducing various forms of uncertainty or variations into the probabilistic model of the channel. See the fulltext of the thesis for the complete abstract.

EPFL2015

Re-proving Channel Polarization Theorems

Mine Alsan

O(N\log N)

operations in the block-length

N

EPFL2015