Back to Discover

Skywork's free AI repaints a blocky video game in detail as you play

Four game scenes, each split between the AI's detailed version and the plain blocky original: a castle, a city street, an ancient Egyptian town and a fighter jet over red desert hills

Skywork AI has released Matrix-Game 4.0, free AI that splits a video game in two: an ordinary game engine runs the rules and controls and draws plain, blocky shapes, and a separate AI for each game repaints them as a detailed castle, ancient Egyptian town or city street while you play. The download comes with three games, but running them takes a Linux computer with a powerful Nvidia graphics card and a fair amount of technical setup, and Skywork says the picture can drift or go wrong in long sessions.

Skywork AI released Matrix-Game 4.0 on Oct. 15, the latest in its series of AI models for games that people can play. Skywork says AI that dreams up every frame of a game by itself has a weakness: once something leaves the view it is gone, and the game's rules have to be worked out again from each picture. Matrix-Game 4.0 splits the job. An ordinary game engine, the software behind most video games, keeps track of the world, the physics and the player's controls, and draws it all as plain, blocky shapes. An AI then repaints that stream as detailed scenes while the game is being played.

The download comes with three games: two shooting games, set in a medieval castle and in ancient Egypt, and a city driving game with time trials and chases. Skywork's video shows each AI-painted scene beside the blocky version it was painted from, and also shows an air-combat game that is not part of the release. The game's on-screen display and its sound come from the game itself, not the AI, and are added on top of the picture.

The AI and the three games are free to download, along with the games' code. Playing is not a one-click affair: it needs a Linux computer with an Nvidia graphics card, and the game shows up in a desktop web browser, which can run on a second computer with no graphics card of its own. Only one person can play at a time. Skywork has tuned it for Nvidia's RTX 5090, 5080 and 5070 Ti home graphics cards, but the speeds it gives are measured before the video reaches the browser, and it has not said how long the delay is between pressing a key and seeing the result on screen.

It has clear limits. Each game has its own AI, built only for that game's scenes and camera angles, and Skywork says shapes it hasn't seen before, fast camera moves, objects blocking the view and long sessions can make the look of things slowly change or get shapes wrong. What the AI paints does not change the game itself: if it draws a wall, the player can't bump into it. Skywork also says it does not promise any particular speed, or that the AI will fit in a home graphics card's memory.

Skywork's instructions invite people to change the games' rules, layouts and the way computer-controlled characters act, using an AI coding assistant, and its project page describes its own assistant turning a request such as "Godmode," a mode where the player can't be hurt, into new game rules. The release does not say whether that assistant is included. So far, how well it plays has been shown only in Skywork's own videos and measurements, and the company has not said whether it plans full games built this way.

Sources

About this video

  • AI-generated video: “Left of the divider is the generated frame, right of it is the engine's low-poly proxy.” Source
  • Open code or weights: “Matrix-Game is Skywork AI's series of open-source interactive world models.” Source
  • The official sources don't say whether the video is sped up.