Laboratory of algorithms and learning systems

Laboratory

Related publications (90)

drozBot: Using Ergodic Control to Draw Portraits

Sylvain Calinon

We present drozBot: le robot portraitiste, a robotic system that draws artistic portraits of people. The input images for the portrait are taken interactively by the robot itself. We formulate the problem of drawing portraits as a problem of coverage which is then solved by an ergodic control algorithm to compute the strokes. The ergodic computation of the strokes for the portrait gives an artistic look to them. The specific ergodic control algorithm that we chose is inspired by the heat equation. We employed a 7-axis Franka Emika robot for the physical drawings and used an optimal control strategy to generate joint angle commands. We explain the influence of the different hyperparameters and show the importance of the image processing steps. The attractiveness of the results was evaluated by conducting a survey where we asked the participants to rank the portraits produced by different algorithms.

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

, ,

We study the Acceleration Obstacle (AO) as a concept to enable a robot's navigation in human crowds. The AO's geometric properties are analyzed and a direct sampling-free algorithm is proposed to approximate its boundary by linear constraints. The resulting controller is formulated as a quadratic program and evaluated in interaction with simulated bi-directional crowd flow in a corridor. We compare it to alternative robotic controllers, considering the robot's and the crowd's performance and the robot's behavior with respect to emergent lanes. Our results indicate that the robot can achieve higher efficiency when being less integrated in lanes.

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

GANs for All: Supporting Fun and Intuitive Exploration of GAN Latent Spaces

Richard Lee Davis, Pierre Dillenbourg, Wei Jiang, Kevin Gonyop Kim

We have developed a new tool that makes it possible for people with zero programming experience to intentionally and meaningfully explore the latent space of a GAN. We combine a number of methods from the literature into a single system that includes multiple functionalities: uploading and locating images in the latent space, image generation with text, visual style mixing, and intentional and intuitive latent space exploration. This tool was developed to provide a means for designers to explore the "design space" of their domains. Our goal was to create a system to support novices in gaining a more complete, expert understanding of their domain's design space by lowering the barrier of entry to using deep generative models in creative practice.

PMLR2022

Mechanical intelligence for learning embodied sensor-object relationships

Ahalya Prabhakar

Intelligence involves processing sensory experiences into representations useful for prediction. Understanding sensory experiences and building these contextual representations without prior knowledge of sensor models and environment is a challenging unsupervised learning problem. Current machine learning methods process new sensory data using prior knowledge defined by either domain knowledge or datasets. When datasets are not available, data acquisition is needed, though automating exploration in support of learning is still an unsolved problem. Here we develop a method that enables agents to efficiently collect data for learning a predictive sensor model-without requiring domain knowledge, human input, or previously existing data-using ergodicity to specify the data acquisition process. This approach is based entirely on data-driven sensor characteristics rather than predefined knowledge of the sensor model and its physical characteristics. We learn higher quality models with lower energy expenditure during exploration for data acquisition compared to competing approaches, including both random sampling and information maximization. In addition to applications in autonomy, our approach provides a potential model of how animals use their motor control to develop high quality models of their sensors (sight, sound, touch) before having knowledge of their sensor capabilities or their surrounding environment. Information-based search strategies are relevant for the learning of interacting agents dynamics and usually need predefined data. The authors propose a method to collect data for learning a predictive sensor model, without requiring domain knowledge, human input, or previously existing data.

NATURE PORTFOLIO2022

Picking up objects to toss them on a conveyor belt are activities generated on a daily basis in the industry. These tasks are still done largely by humans. This paper proposes a unified motion generator for a bimanual robotic systems that enables two 7 degrees of freedom robotic arms to grab and toss an object in one swipe. Unlike classical approaches that grab the object with quasi-zero contact velocity, the proposed approach is able to grasp the object while in motion. We control the contact forces prior to and following impact so as to stabilize the robots’ grip on the object. We show that such swift grasping speeds up the pick-and-place process and reduces energy expenditure for tossing. Continuous control of reach, grab and toss motion is achieved by combining a sequence of time-invariant dynamical system in a single control framework. We introduce a state-dependent modulation function to control the generated velocity in different directions. The framework is validated in simulation and on a real dual-arm system. We show that we can precisely toss objects within a workspace of 0.2 by 0.4 square meter. Moreover, we show that the algorithm can adapt on the fly to changes in object location.

2022

, ,

This paper describes a novel approach to learn an implicit, differentiable distance function for arbitrary configurations of a robotic manipulator used for reactive control. By exploiting GPU processing, we efficiently query the learned collision representation and obtain an implicit distance between the robot and the environment. The differentiable nature of the learned function allows for calculating valid gradients wrt. any robot configuration, providing a repulsive vector field in joint space that can be injected in various control methods to improve collision avoidance. We present preliminary results on solving collision avoidance for a 7DoF robot with a reactive inverse kinematics solution, as well as improving performance of a sampling-based model-predictive controller.

2022

Circuit optimization and PCB design for suspended microchannel resonators

Sarah Bonaly

2021

Design and evaluation of digital tools to expand experience in vocational education

Kevin Gonyop Kim

Vocational education and training (VET) is the most popular form of upper-secondary education in Switzerland, with more than two-thirds of the teenagers enrolling in a VET program after finishing their compulsory education. The vast majority of the Swiss VET systems have a dual-track structure where students learn in schools for one or two days per week while they do an apprenticeship in a company for the remaining days. Their professional competencies are developed through a combination of workplace experiences reinforced by the theoretical knowledge learned in school.While the apprenticeship at workplaces provides valuable experience to the learners, one concern is that there is a lack of diversity in the experiences of an individual apprentice. Motivated by this limited experience of VET learners in the workplace, the goal of our research has been to explore the possibilities of expanding their experiences. Focusing on design-related professions in VET, we consider exploring digital variations of a design as a way to expand the original experience related to the design. The research objectives of this thesis are (1) exploring possibilities to expand the experience of learners in design-related VET and (2) investigating the potential benefits of the expanded experience. To accomplish the research objectives, we formalize the idea of expanding experience for VET learners by defining the three dimensions of expansion: parametric, temporal, and social. We explore these three dimensions in three studies.For the parametric dimension, we investigate how learners explore a multidimensional space of possible designs using a digital tool and how we can support the process with an interface that enables disentangled exploration of design parameters. We present a tool called BloomGraph that has been developed for florist apprentices to explore flower bouquet designs, and the results from an experimental study that investigates the effectiveness of a graph interface for design exploration.For the temporal dimension, we explore how we can support the VET learners with envisioning the designs that change over time. We present a virtual-reality tool called GardenVR developed for gardener apprentices that allows creating a garden and exploring it in an immersive environment. In an experimental study with gardener apprentices, we investigate how they explore the time dimension using the tool, compare the tool with the current method of practice, paper-sketching, and study how the two interfaces can be combined in an instructional design to enhance the design outcome.Lastly, for the social dimension, we present a tool called Mixplorer that has been designed for a classroom scenario of gardener apprentices. The tool provides a way to socially explore design spaces by allowing the apprentices to create an initial design and recombine it with the designs of other learners. We present the results of two studies conducted using the tool that investigate its feasibility in the VET setting, potential benefits for the learners, and the effectiveness of the design-mixing process in creative practices.

EPFL2022

Optimization of robot configurations for motion planning in industrial riveting

Sylvain Calinon, Hakan Girgin, Teguh Santoso Lembono

Collaborative robots (cobots) have been increasingly used in the industry in recent years. The cobots are often 7-axis kinematically redundant manipulators. Exploiting these redundancies in industrial tasks is still a challenge as the change in the robot joint configuration can be unnecessarily large from one task to another depending on the initialization of the numerical inverse kinematics. In this article, we address this challenge in a particularly important under-explored industrial task: percussive riveting. The load exerted on the rivet for its plastic deformation can be dangerous for the robot if the natural frequency of the robot coincides with the frequency of the impact forces. In this work, we propose to take into account the vibrational characteristics of the manipulator to determine the robot joint configurations that minimize the overall end-effector's displacement. We also propose to exploit these safe joint configurations for each rivet hole in a task and motion planning algorithm called RoboTSP to generate the optimal trajectory visiting all target holes. We demonstrate our method with a Franka Emika Panda robot in a simulated environment.

IEEE2021

, ,

paper studies exploiting action-level learning (imitation) in the optimal control problem context. Cost functions defined by the optimal control methods are similar to the goal-level learning (emulation) in animals. However, imitating the robot's or others' (e.g. human's) previous experiences (demonstrations) could help the system to improve its performances. We propose to use demonstrations more efficiently by predicting an initialization for the optimal control problems (OCPs) and adding an imitation term to the cost functions. While the predicted initial guess initializes the OCPs close to their local optima, the imitation term guides the optimization, resulting in a faster convergence rate. We test our algorithm in a physical assistive task where a robot should help a human perform a sit-to-stand (STS) task. We define this task as two optimal control problems. The first OCP predicts the human's desired assistance and the other one controls the robot. We have tested our method on different experiments with different conditions for the human in which the robot should quickly solve the two optimization problems exploiting some demonstrations of how it can perform the task. Our proposed method reduced the number of iterations by more than 90% and 70% for the human assistance prediction and the robot controller, respectively, compared to the standard problem which does not take the demonstrations into account.

IEEE2021