What to pay attention to
There are a few things to keep in mind when watching the rise of robot demonstration videos even live streams. First, such robot demonstrations are not indicative of robots operating autonomously without human supervision or control. Deepam PatelPhD candidate in computer science at Purdue University and researcher at the US Army DevCom Army Research Laboratory. Many demonstrations still rely on human operators directly controlling the robots’ movements via teleoperation.
“Unless a research paper or a company clearly mentions that (the robot) is completely autonomous, you have to take it with a very large pinch of salt,” Patel told Ars.
Another question to consider is whether the demonstration shows the robots tackling an entirely new test environment for the first time, or whether the robots are simply repeating a task they’ve already learned in that particular training environment. The new test environment will be significantly more effective in demonstrating robots that can perform tasks autonomously, Patel said.
For any robot demonstration, it’s also worth checking the playback speed of the video, because for safety and other reasons, “usually the robots are very slow,” Patel said. Companies may sometimes disclose that a robot demonstration video runs twice or four times as fast as normal, meaning the robot may take twice or four times as long as a human to perform the same task.
Robot demonstration videos can also vary greatly in terms of their informative value and transparency. Some are clearly meant to be performance entertainment clips that can go viral on social media, or polished promotional videos of companies looking for new customers and investors. Others may offer a more behind-the-scenes look at the robot training process, admitting robot mistakes along the way.
But even if a robot demo video looks incredibly impressive and original when it comes from a more reputable company or research lab, remember that it’s still a small glimpse of the big picture. The real indicators of progress in robotic skills are not so easily packaged for an Internet audience.
