Kyutai, an AI research lab based in Paris, released REKI on Friday, Oct. 16, built with researchers from Inria and the École nationale des ponts et chaussées. It is a vision-language-action model, a robot model that combines what the robot sees with instructions in words and turns them into movements. Given a camera view and a line such as "put the bowl on the plate", it works out how the arm should move next. Kyutai published the model's trained files, so anyone can download it and run it on their own computers, along with the code for every step from preparing the data to running it on a real robot.
What sets REKI apart is a short memory. Most models of this kind look only at the present moment, the researchers say, and without memory their own model often got stuck moving an arm back and forth between two positions, which they put down to each position looking like a moment in its training examples where the arm went on toward the other. REKI keeps a small summary of the camera images from the past two seconds, one per second. In computer-simulated tests with two-armed robots, arms got stuck in such a loop in 4.1% of attempts with the memory, against 18.6% without it, and the researchers say the memory adds less than 1% to the time the model takes to work out each move, and no extra cost in training.
Kyutai also tried REKI on a real robot arm, a UR5, after giving it about an hour of extra training on 50 recorded runs of each of five tasks, in which a person steered the arm with a virtual-reality controller. On the three longer tasks, which ask the arm to handle two or three objects in a set order, so it must keep track of what it has already done, the version with memory scored 12 to 17.5 percentage points higher than the same model without it, on a score that gives partial credit for each step completed. On the two shorter tasks the memory made no difference. Each task was tried 10 times.
The model is small for its kind, with 2.4 billion parameters, the numbers a model tunes as it learns. Starting from an existing model that reads images and text, it was trained only on public data, including recordings of robots and of people handling objects, so Kyutai says other researchers can repeat its training. The researchers say that among models trained only on public data, REKI did best on two standard tests that change things like lighting, camera angle and clutter between training and testing, even though it is the smallest. Models about twice its size, trained on companies' private data or built on AI video generators, still score higher on one of those two tests and on a third, and the researchers say models like these can be costly to run.
The model's files come under a license that allows non-commercial uses such as research, but not commercial use, while the code itself is free for any use. The memory covers only a few seconds, which the researchers say seems enough for most standard tests, but they say it remains to be tested whether it can handle tasks that depend on something that happened much earlier. REKI has also been tried on only one kind of real arm, 10 times per task, and all the results so far come from the researchers' own tests. Its training examples are mostly tabletop jobs for two-fingered grippers, and the researchers expect it to do worse on other tasks or hands without extra training.