The robot masters the terrain through animal-like walking transitions

summary: The researchers leveraged deep reinforcement learning (DRL) to enable the robot to adaptively switch its gait, mimicking animal movements such as trot and gallop, to effectively traverse complex terrain. Their study explores the concept of survivability – or fall prevention – as a primary driver of such gait transitions, challenging previous beliefs that energy efficiency is the main driver.

This new approach not only enhances the robot's ability to handle difficult terrain, but also provides deeper insights into animal movement. The team's findings suggest that prioritizing fall prevention may lead to more flexible and efficient robotic and biological movement across uneven surfaces.

Key facts:

1. Adapting gait to survive: The EPFL DRL robot was used to learn walking transitions primarily for continuity, effectively adapting its movement strategies to avoid falling when navigating terrain with gaps.
2. Re-evaluating energy efficiency: Contrary to previous theories, the study found that improvements in energy efficiency are a consequence, rather than a driver, of walking transitions in challenging environments.
3. Bio-inspired robotic agility: The research demonstrated a bio-inspired learning architecture that allowed learning-driven spontaneous gait transitions, demonstrating advanced robotic agility in navigating across successive gaps in experimental terrain.

source: EPFL

With the help of a form of machine learning called deep reinforcement learning (DRL), the EPFL robot specifically learned to transition from a trot to a walk – an arch-supported hopping gait used by animals such as springbok and deer – to navigate difficult terrain with gaps of 14-30cm.

The study, conducted by the BioRobotics Laboratory at EPFL's Faculty of Engineering, provides new insights into why and how such gait transformations occur in animals.

“Previous research has presented energy efficiency and avoiding musculoskeletal injuries as the main explanations for gait shifts. More recently, biologists have said that stability on flat terrain could be more important.”

“But experiments with animals and robots have shown that these hypotheses are not always correct, especially on uneven terrain,” says doctoral student Milad Shafii, first author of a paper published in Nature Communications.

Chaveille, co-authors Guillaume Bellegarda, and BioRobotics Lab head Auke Eijsbert were interested in a new hypothesis about why gait transitions occur: the ability to stay on, or avoid, falls. To test this hypothesis, they used DRL to train a four-legged robot to cross different terrains.

On flat terrain, they found that different gaits showed different levels of strength in the face of random pushes, and that the robot switched from walking to trotting to maintain survivability, just as four-legged animals do when accelerating.

When the robot encountered successive gaps in the experimental surface, it automatically switched from trotting to running to avoid falling. Furthermore, survivability was the only factor improved by such gait transitions.

“We showed that on flat terrain and difficult discrete terrain, survivability induces shifts in gait, but energy efficiency does not necessarily improve,” explains Shafii.

“It appears that energy efficiency, previously thought to be the driver of such transformations, may be more of an outcome. When an animal navigates difficult terrain, its first priority is likely to not fall, followed by energy efficiency.”

Bio-inspired educational architecture

To model movement control in their robot, the researchers took into account the three interacting elements that drive the animal's movement: the brain, the spinal cord, and sensory feedback from the body.

They used DRL to train a neural network to mimic the transmission of brain signals from the spinal cord to the body as the robot crossed experimental terrain. Next, the team assigned different weights to three potential learning objectives: energy efficiency, force reduction, and survivability.

A series of computer simulations revealed that of these three goals, survivability was the only goal that prompted the robot to change its gait automatically – without instructions from scientists.

The team emphasizes that these observations represent the first learning-based framework of locomotion, in which gait transitions emerge automatically during the learning process, as well as the most dynamic crossing of such large successive gaps for a quadruped robot.

“Our life-inspired learning architecture demonstrated state-of-the-art agility of quadcopter robots on challenging terrain,” says Shafi.

The researchers aim to expand their work with additional experiments that place different types of robots in a wide range of challenging environments.

In addition to further elucidating animal locomotion, they hope their work will eventually enable more widespread use of robotics in biological research, reducing reliance on animal models and the ethical concerns associated with them.

About robotics and artificial intelligence research news

author: Celia Lauterbacher
source: EPFL
communication: Celia Lauterbacher – EPFL
picture: Image credit to BioRob EPFL

Original search: Open access.
“Survivability leads to gait transitions in learning agile quadrupedal locomotion in difficult terrain“By Milad Shafie et al. Nature Communications

a summary

Survivability leads to gait transitions in learning agile quadrupedal locomotion in difficult terrain

Four-legged animals are able to transition smoothly between their different gaits. While energy efficiency appears to be one reason for the change in gait, other determining factors likely play a role as well, including terrain characteristics.

In this article, we propose that survivability, i.e., avoidance of falls, represents an important criterion for gait transitions.

We investigate the emergence of gait transformations through the interaction between the supraspinal motor (brain), the central pattern generator in the spinal cord, the body, and external sensing by leveraging deep reinforcement learning and robotics tools.

Consistent with data on quadruped animals, we show that the trotting gait transmission of quadruped robots on flat terrain improves both vitality and energy efficiency.

Furthermore, we study the effects of discrete terrain (i.e., crossing successive gaps) on enforcing gait transitions, and find the emergence of trotting transitions to avoid unviable situations.

Survivability is the only improving factor after walking transitions on both flat and discrete gap terrain, suggesting that survivability could be a primary and universal goal for walking transitions, while other criteria are secondary goals and/or a consequence of ability. To stay.

Furthermore, our experiments demonstrate the state-of-the-art agility of the four-legged robot in challenging scenarios.