Learning of physical systems: from inference to control

To characterize a physical system to behave as desired, either its underlying governing rulesmust be known a priori or the system itself be accurately measured. The complexity of fullmeasurements of the system scales with its size. When exposed to real-world conditions, suchas perturbations or time-varying settings, the system calibrated for a fixed working conditionmight require non-trivial re-calibration, a process that could be prohibitively expensive, inefficientand impractical for real-world use cases.In this thesis, a learning procedure for solving highly ill-posed problems of modeling a system'sforward and backward response functions is proposed. In particular, deep neural networksare used to infer the input of a system from partial measurements of its outputs or to obtain adesired target output from a physical system.I showcase the applicability of the proposed methods for inference and control in opticalmultimode fibers. Amplitude/phase-encoded input of a multimode fiber is reconstructedfrom intensity-only measurements of the outputs. Conversely, the required input of the fiberfor projecting a desired output is obtained using intensity-only measurements of the output.Next, the stochastic neural network of the retina in Salamander is modeled by a probabilisticneural network. The model is used to optimize the input stimuli so as to find the simplestspatiotemporal patterns that elicit the same neuronal spike responses as those elicited byhigh-dimensional stimuli.As demonstrated in this thesis, application of data-driven methods for characterization ofcomplex large-scale real-world systems has proved useful in simplifying the measurement apparatus,end-to-end optimization of the system and automatic compensation of perturbation.

Actor neural networks for the robust control of partially measured nonlinear systems showcased for image propagation through diffuse media

Navid Borhani, Eirini Kakkava, Damien Claude-Marie Loterie, Christophe Moser, Demetri Psaltis, Babak Rahmani, Ugur Tegin

The output of physical systems, such as the scrambled pattern formed by shining the spot of a laser pointer through fog, is often easily accessible by direct measurements. However, selection of the input of such a system to obtain a desired output is difficult, because it is an ill-posed problem; that is, there are multiple inputs yielding the same output. Information transmission through scattering media is an example of this problem. Machine learning approaches for imaging have been implemented very successfully in photonics to recover the original input phase and amplitude objects of the scattering system from the distorted intensity diffraction pattern outputs. However, controlling the output of such a system, without having examples of inputs that can produce outputs in the class of the output objects the user wants to produce, is a challenging problem. Here, we propose an online learning approach for the projection of arbitrary shapes through a multimode fibre when a sample of intensity-only measurements is taken at the output. This projection system is nonlinear, because the intensity, not the complex amplitude, is detected. We show an image projection fidelity as high as similar to 90%, which is on par with the gold-standard methods that characterize the system fully by phase and amplitude measurements. The generality and simplicity of the proposed approach could potentially provide a new way of target-oriented control in real-world applications when only partial measurements are available.

SPRINGERNATURE2020

Cognitive navigation based on non-uniform Gabor space sampling, unsupervised growing networks, and reinforcement learning

Wulfram Gerstner

We study spatial learning and navigation for autonomous agents. A state space representation is constructed by unsupervised Hebbian learning during exploration. As a result of learning, a representation of the continuous two-dimensional (2-D) manifold in the high-dimensional input space is found. The representation consists of a population of localized overlapping place fields covering the 2-D space densely and uniformly. This space coding is comparable to the representation provided by hippocampal place cells in rats. Place fields are learned by extracting spatio-temporal properties of the environment from sensory inputs. The visual scene is modeled using the responses of modified Gabor filters placed at the nodes of a sparse Log-polar graph. Visual sensory aliasing is eliminated by taking into account self-motion signals via path integration. This solves the hidden state problem and provides a suitable representation for applying reinforcement learning in continuous space for action selection. A temporal-difference prediction scheme is used to learn sensorimotor mappings to perform goal-oriented navigation. Population vector coding is employed to interpret ensemble neural activity. The model is validated on a mobile Khepera miniature robot.

Institute of Electrical and Electronics Engineers2004

Learning to move in modular robots using central pattern generators and online optimization

Auke Ijspeert, Jérôme Maye

This article addresses the problem of how modular robotics systems, i.e. systems composed of multiple modules that can be configured into different robotic structures, can learn to locomote. In particular, we tackle the problems of online learning, that is, learning while moving, and the problem of dealing with unknown arbitrary robotic structures. We propose a framework for learning locomotion controllers based on two components: a central pattern generator (CPG) and a gradient-free optimization algorithm: Powell’s method. The CPG is implemented as a system of coupled nonlinear oscillators in our YaMoR modular robotic system, with one oscillator per module. The nonlinear oscillators are coupled together across modules using Bluetooth communication to obtain specific gaits, i.e. synchronized patterns of oscillations among modules. Online learning is done by running the Powell optimization algorithm in parallel to the CPG model, with the speed of locomotion being the criterion to be optimized. Interesting aspects of the optimization are: it is carried out online, it does not require stopping or resetting the robots, and it is fast. We present results showing the interesting properties of this framework for a modular robotic system. In particular, our CPG model can readily be implemented in a distributed system, it is cheap computationally, it exhibits limit cycle behavior (temporary perturbations are rapidly forgotten), it produces smooth trajectories even when control parameters are abruptly changed, and it is robust against imperfect communication among modules. We also present results of learning to move with three different robot structures. Interesting locomotion modes are obtained after running the optimization for less than 60 minutes.

2008