VAST, the company behind Tripo, a service that makes 3D models with AI, released EdenLive on Oct. 18. It is AI that repaints a video while the video is still playing. Give it a rough video, such as a game drawn in plain gray blocks, a simple animation or ordinary camera footage, along with a picture of how the opening frame should look and a few words of description. It keeps where everything is and how it moves, and draws the scene in detail, anywhere from lifelike to hand-drawn; camera footage can be turned into a cartoon, pixel art or an oil painting. VAST says it works fast enough to keep up as the video plays.
Instead of asking one AI to invent both what happens in a game and what it looks like, an ordinary program keeps track of the game, the score and where everything is, and the AI only paints the picture. VAST says AI that dreams up each frame by itself can lose track of an object that drifts out of view, and can't keep a score or a health bar true to a game's rules. VAST counts a string of other projects since August, including one from Nvidia, that split the work the same way. One of them, Matrix-Game 4.0, trains separate painting AIs for different scenes, VAST says, while one EdenLive model handles blocky games, 2D game footage and real camera footage.
VAST also released a helper for AI coding assistants that turns a written description of a game, such as a street fight or a submarine treasure hunt, into a simple blocky game for EdenLive to paint. VAST suggests uses beyond games, too: trying different materials and lighting on a rough 3D scene, and repainting a video of human hands handling objects so the hands become robot arms, which could give robots more examples to learn from.
The AI and the code to run it are free to download, though some of the AI's parts come with their own license terms. VAST says the smaller of its two versions runs live on one top-end home graphics card, Nvidia's RTX 5090, with an outside speed-up add-on; its own setup instructions were tested on Nvidia data-center cards with 80 gigabytes of memory. It is not a casual download: the files come to about 210 gigabytes, setting it up takes technical skill, and the all-in-one workspace that builds games from a description is for noncommercial use only. People who would rather not install it can try it in online demos and in Unbound Loop, a 3D editing app that has added it.
VAST lists limits of its own. The AI decides for itself which character each gray block stands for, and details can change when a character comes back after a long time out of view or the camera returns to a place it has already shown. VAST says a key press currently takes a few tenths of a second to show up in the painted picture. VAST's quality comparisons were scored by another AI model, not by people, and its speed figures count only the AI's own work. No one outside the company has yet shown how it holds up in long play.