Trial and error | EPFL Graph Search

Trial and error is a fundamental method of problem-solving characterized by repeated, varied attempts which are continued until success, or until the practicer stops trying. According to W.H. Thorpe, the term was devised by C. Lloyd Morgan (1852–1936) after trying out similar phrases "trial and failure" and "trial and practice". Under Morgan's Canon, animal behaviour should be explained in the simplest possible way. Where behavior seems to imply higher mental processes, it might be explained by trial-and-error learning. An example is a skillful way in which his terrier Tony opened the garden gate, easily misunderstood as an insightful act by someone seeing the final behavior. Lloyd Morgan, however, had watched and recorded the series of approximations by which the dog had gradually learned the response, and could demonstrate that no insight was required to explain it. Edward Lee Thorndike was the initiator of the theory of trial and error learning based on the findings he showed how

Using reinforcement learning for optimizing the reproduction of tasks in robot programming by demonstration

Florent Günter

As robots start pervading human environments, the need for new interfaces that would simplify human-robot interaction has become more pressing. Robot Programming by Demonstration (RbD) develops intuitive ways of programming robots, taking inspiration in strategies used by humans to transmit knowledge to apprentices. The user-friendliness of RbD is meant to allow lay users with no prior knowledge in computer science, electronics or mechanics to train robots to accomplish tasks the same way as they would with a co-worker. When a trainer teaches a task to a robot, he/she shows a particular way of fulfilling the task. For a robot to be able to learn from observing the trainer, it must be able to learn what the task entails (i.e. answer the so-called "What-to-imitate?" question), by inferring the user's intentions. But most importantly, the robot must be able to adapt its own controller to fit at best the demonstration (the so-called "How-to-imitate?" question) despite different setups and embodiments. The latter is the question that interested us in this thesis. It relates to the problem of optimizing the reproduction of the task under environmental constraints. The "How-to-imitate?" question is subdivided into two problems. The first problem, also known as the "correspondence problem", relates to resolving the discrepancy between the human demonstrator and robot's body that prevent the robot from doing an identical reproduction of the task. Even though we helped ourselves by considering solely humanoid platforms, that is platforms that have a joint configuration similar to that of the human, discrepancies in the number of degrees of freedom and range of motion remained. We resolved these by exploiting the redundant information conveyed through the demonstrations by collecting data through different frames of reference. By exploiting these redundancies in an algorithm comparable to the damped least square algorithm, we are able to reproduce a trajectory that minimizes the error between the desired trajectory and the reproduced trajectory across each frame of reference. The second problem consists in reproducing a trajectory in an unknown setup while respecting the task constraints learned during training. When the information learned from the demonstration no longer suffice to generalize the task constraints to a new set-up, the robot must re-learn the task; this time through trial-and-error. Here we considered the combination of trial-and-error learning to complement RbD. By adding a trial-and-error module to the original Imitation Learning algorithm, the robot can find a solution that is more adapted to the context and to its embodiment than the solution found using RbD. Specifically, we compared Reinforcement Learning (RL) – to other classical optimization techniques. We show that the system is advantageous in that: a) learning is more robust to unexpected events that have not been encountered during the demonstrations and b) the robot is able to optimize its own model of the task according to its own embodiment.

EPFL2009

Benchmark for Bimanual Robotic Manipulation of Semi-deformable Objects

Aude Billard, Fanjun Bu, Konstantinos Chatzilygeroudis, Bernardo Fichera, Farshad Khadivar, Ilaria Lauzana, Kunpeng Yao

We propose a new benchmarking protocol to evaluate algorithms for bimanual robotic manipulation semi-deformable objects. The benchmark is inspired from two real-world applications: (a) watchmaking craftsmanship, and (b) belt assembly in automobile engines. We provide two setups that try to highlight the following challenges: (a) manipulating objects via a tool, (b) placing irregularly shaped objects in the correct groove, (c) handling semi-deformable objects, and (d) bimanual coordination. We provide CAD drawings of the task pieces that can be easily 3D printed to ensure ease of reproduction,and detailed description of tasks and protocol for successful reproduction, as well as meaningful metrics for comparison. We propose four categories of submission in an attempt to make the benchmark accessible to a wide range of related fields spanning from adaptive control, motion planning to learning the tasks through trial-and-error learning.

2020

A virtual vehicle approach to distributed control for formation keeping of underactuated vehicles

Andrea Alessandretti, Colin Neil Jones, Yann Roth

This paper addresses the control problem of formation keeping of a fleet of underactuated vehicles under communication constraints, where each agent is allowed to communicate with only a subset of the agents of the fleet. We adopt a virtual vehicle approach where every underactuated agent tracks a virtual vehicle, described by a single integrator model, driven by a consensus-based distributed controller. This approach results in a distributed dynamic controller for formation keeping of underactuated vehicles with exponential convergence guarantee of the formation error to zero. Simulation results are presented for both wheeled-like vehicles (2-D case) and UAV-like vehicles (3-D case).

2015

Using reinforcement learning for optimizing the reproduction of tasks in robot programming by demonstration

Florent Günter

EPFL2009