Back to Discover

Tencent's GAE helps AI video keep 3D scenes in shape as the camera moves

Official GAE Stage 1 method diagram: frozen geometry foundation features compressed into a compact geometry-native latent jointly supervised for RGB and geometry decode
Image: the researchers

Tencent and university researchers built GAE, which lets an AI video maker keep track of a scene's 3D shape while it draws each frame. AI video can look real while the scene drifts out of shape and the camera strays off course.

Tencent's ARC Lab and the Hong Kong University of Science and Technology unveiled GAE on Sept. 21, 2026, with help from the University of Hong Kong and the University of Texas at Austin. It is a new way for artificial intelligence video tools to keep track of the 3D shape of the scenes they draw. Today's AI video can look photo-real while the world underneath slips: shapes drift from frame to frame, and the camera strays from the path it was asked to follow. The researchers argue that if AI video is to become the basis for simulated worlds, every frame has to describe the same lasting place.

Most AI video tools work from a compressed sketch of how each frame looks. GAE builds that sketch instead from a separate AI that is already good at reading depth and camera position from photos, so shape is baked in from the start rather than bolted on afterward. Each frame the model makes can then be read back two ways: as a picture, and as a 3D map of where everything sits. Think of a painter who keeps a floor plan beside the easel, so the doorway stays where it belongs.

The team's showcase pairs AI-made scenes with the 3D shape the model built for each one. Give the released model one photo and a planned camera path, and it makes a short video plus a matching 3D model of the scene. A written prompt works too, producing a picture with its own 3D shape. When the team swapped GAE into an otherwise identical video system, the videos looked better and, on one set of scenes, the camera's drift from its requested path was roughly cut in half.

Tencent has made the code, the training tools and the model public, but its license allows academic use only and rules out commercial or production work. The researchers say their gains held up in larger runs with more data, sharper video and longer clips. They pitch GAE as a common language linking AI that reads 3D scenes with AI that creates them.

The long clips the team shows off came from a bigger model trained separately, and those clips were not part of the head-to-head tests. The gains come from the team's own tests. Its Camera Studio tool, which lets people steer the camera with a mouse or game-style keys before anything is made, is set up by default for a machine with eight graphics cards.