Back to Discover

A robot AI that learned by watching people tries tasks it never practiced on a robot

LiberAI's white robot, in a cap with a smiling face on its screen, holds out a finished cup of boba tea it made

Robot company LiberAI says its new AI, Liber-0, learned to handle objects mostly from more than 2 million hours of recordings of people, some made with a glove that tracks finger movements. On tasks it had never practiced on a robot, it succeeded about a third of the time in LiberAI's own tests, and long chores with several steps were the hardest. Anyone can try a simulated version online, but LiberAI hasn't said whether it will offer the model itself or sell robots that run it.

Teaching a robot to use its hands is expensive. Usually a person steers the robot through each job while it records, on that same robot and for that same task, which LiberAI says costs a lot and is hard to do in large amounts. The robot company has now shown Liber-0, a robot AI that learned by watching people instead. It was trained on more than 2 million hours of recordings of people doing things. LiberAI says that lets a robot try some jobs it never practiced on a robot.

Video alone has a gap. It shows a hand from the outside, but not exactly how each finger moved. So LiberAI made a thin glove that records the wearer's finger movements while cameras on the chest and wrists film the work, from about where the robot's own cameras sit. Recording this way needs no robot in the room, which LiberAI says makes it cheaper to do in many places. LiberAI's robot is built to move more like a person too, with wrists that turn like ours and a waist that bends, which LiberAI says lets movements learned from people carry over more directly. In LiberAI's tests, the glove recordings taught the robot more than a far larger pile of ordinary online video.

To test Liber-0, LiberAI had GPT-6 suggest tabletop tasks, so the list didn't come from its own test set. The robot then tried them with no robot practice on those tasks, though its training did include robot recordings of other jobs. It succeeded about a third of the time overall. Picking something up and putting it somewhere went best. Longer chores with several steps rarely worked, though another LiberAI video shows the robot finishing one: opening a drawer, putting a pen in a pouch, storing the pouch and closing the drawer.

LiberAI also put GPT-6 Astra, a general AI that plans step by step, in sole charge of the robot on the same tasks. LiberAI says it succeeded far less often and struggled with anything that needed careful contact.

The video's boba tea scene shows something else. LiberAI says it adapted the model to that job, so the scene doesn't show a job the robot had never practiced, though the robot does it on its own. It hasn't said what that adaptation took, and the clip runs faster than real time.

All of these results come from LiberAI's own tests, and LiberAI hasn't said whether anyone outside the company has checked them. The glove records movement but not touch or grip strength, which LiberAI says it could add for jobs like fitting parts together. Visitors to a LiberAI website can upload their own objects and watch a simulated version of Liber-0 handle them, but the company hasn't said whether it will release Liber-0 or sell robots that run it.

Sources

About this video

  • Autonomous: On-screen text at 2:06: "3X speed, fully autonomous" Source
  • Partly sped up: On-screen text at 2:06: "3X speed, fully autonomous" Source
  • Real footage: “Adapt pretrained manipulation skills to real-world workflows, demonstrated through autonomous boba tea preparation.” Source
  • The official sources don't say whether you can use it yet.