Publication

Identifying important sensory feedback for learning locomotion skills

Abstract

Robot motor skills can be acquired by deep reinforcement learning as neural networks to reflect state-action mapping. The selection of states has been demonstrated to be crucial for successful robot motor learning. However, because of the complexity of neural networks, human insights and engineering efforts are often required to select appropriate states through qualitative approaches, such as ablation studies, without a quantitative analysis of the state importance. Here we present a systematic saliency analysis that quantitatively evaluates the relative importance of different feedback states for motor skills learned through deep reinforcement learning. Our approach provides a guideline to identify the most essential feedback states for robot motor learning. By using only the important states including joint positions, gravity vector and base linear and angular velocities, we demonstrate that a simulated quadruped robot can learn various robust locomotion skills. We find that locomotion skills learned only with important states can achieve task performance comparable to the performance of those with more states. This work provides quantitative insights into the impacts of state observations on specific types of motor skills, enabling the learning of a wide range of motor skills with minimal sensing dependencies. Traditional feedback-state selection in robot learning is empirical and requires substantial engineering efforts. Yu et al. develop a quantitative and systematic state-importance analysis, revealing crucial feedback signals for learning locomotion skills.

About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.
Related concepts (33)
Robotics
Robotics is an interdisciplinary branch of electronics and communication, computer science and engineering. Robotics involves the design, construction, operation, and use of robots. The goal of robotics is to design machines that can help and assist humans. Robotics integrates fields of mechanical engineering, electrical engineering, information engineering, mechatronics engineering, electronics, biomedical engineering, computer engineering, control systems engineering, software engineering, mathematics, etc.
Motor skill
A motor skill is a function that involves specific movements of the body's muscles to perform a certain task. These tasks could include walking, running, or riding a bike. In order to perform this skill, the body's nervous system, muscles, and brain have to all work together. The goal of motor skill is to optimize the ability to perform the skill at the rate of success, precision, and to reduce the energy consumption required for performance. Performance is an act of executing a motor skill or task.
Motor learning
Motor learning refers broadly to changes in an organism's movements that reflect changes in the structure and function of the nervous system. Motor learning occurs over varying timescales and degrees of complexity: humans learn to walk or talk over the course of years, but continue to adjust to changes in height, weight, strength etc. over their lifetimes. Motor learning enables animals to gain new skills, and improves the smoothness and accuracy of movements, in some cases by calibrating simple movements like reflexes.
Show more
Related publications (44)

Exploration-based model learning with self-attention for risk-sensitive robot control

Sudong Lee

Model-based reinforcement learning for robot control offers the advantages of overcoming concerns on data collection and iterative processes for policy improvement in model-free methods. However, both methods use exploration strategy relying on heuristics ...
2023

Mechanisms underlying reinforcement learning of motor skills

Pierre Theopistos Vassiliadis

Motor learning allows animals, including human beings, to acquire skills that are es-sential for efficient interactions with the environment. This ability to learn new motor skills is of great practical relevance for daily-life activities (such as when lea ...
EPFL2023

Neurocognitive and motor-control challenges for the realization of bionic augmentation

Silvestro Micera

Robotic fingers and arms that augment the motor abilities of non-disabled individuals are increasingly feasible yet face neurocognitive barriers and hurdles in efferent motor control. ...
NATURE PORTFOLIO2022
Show more
Related MOOCs (20)
Thymio: un robot pour se former à l'informatique
On propose dans ce MOOC de se former à et avec Thymio : apprendre à programmer le robot Thymio et ce faisant, s’initier à l'informatique et la robotique.
The Thymio robot as a tool for discovering digital science
This MOOC teaches basic understanding of robots’ mechanisms and Thymio’s programming languages, classroom use and pedagogical elements.
The Thymio robot as a tool for discovering digital science
This MOOC teaches basic understanding of robots’ mechanisms and Thymio’s programming languages, classroom use and pedagogical elements.
Show more

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.