A clip from the simulation in which virtual robots learn to climb stairs.
An army of more than 4,000 marching dog-like robots is a vaguely ominous sight, even in a simulation. But it can lead the way for machines to learn new tricks.
The virtual robot army was developed by researchers from ETH Zurich in Switzerland and chip manufacturer Nvidia. They used the wandering bots to train an algorithm that was then used to control the legs of a real robot.
In the simulation, the machines – called ANYmally – face challenges such as inclines, steps and steep slopes in a virtual landscape. Every time a robot learned to master a challenge, the researchers presented a more difficult challenge and made the control algorithm more sophisticated.
From a distance, the resulting scenes resemble an army of ants writhing over a large area. During the training, the robots mastered walking up and down stairs without any problems; more complex obstacles took longer. It was particularly difficult to negotiate slopes, although some of the virtual robots learned to slide down.
When the resulting algorithm was carried over to a real-world version of ANYmal, a four-legged robot the size of a large dog with sensors on its head and a detachable robotic arm, it was able to navigate stairs and blocks but had problems at higher speeds. The researchers blamed inaccuracies in the sensors’ perception of the real world compared to the simulation.
Similar types of robot learning could help machines learn all sorts of useful things from sorting packages to sewing clothes to harvesting grain. The project also reflects the importance of simulations and custom computer chips for future advances in applied artificial intelligence.
“At a high level, very fast simulation is a really great thing,” says Pieter Abbeel, professor at UC Berkeley and co-founder of Covariant, a company that uses AI and simulations to train robotic arms to select and sort objects for logistics companies . He says the Swiss and Nvidia researchers “got some nice accelerations”.
AI has shown great promise in training robots for real-world tasks that cannot simply be written in software or that require some kind of customization. For example, the ability to capture awkward, slippery, or unfamiliar objects cannot be written in lines of code.
The 4,000 simulated robots were trained using reinforcement learning, an AI method inspired by research on how animals learn through positive and negative feedback. While the robots move their legs, an algorithm assesses how this affects their ability to walk and adjusts the control algorithms accordingly.