Back to Discover

Tencent ARC and HKUST GAE learns a geometry-native latent for 3D-consistent generation

Official GAE Stage 1 method diagram: frozen geometry foundation features compressed into a compact geometry-native latent jointly supervised for RGB and geometry decode

On Sept. 21, 2026, Tencent ARC Lab with the Hong Kong University of Science and Technology, the University of Hong Kong, and the University of Texas at Austin submitted GAE (arXiv:2609.24981), a geometry-native autoencoder whose latent jointly decodes RGB, depth, cameras, and point maps. In author-controlled matches versus SD-VAE under identical DiT/flow, GAE-64 FVD is 225.7 / 287.0 (RealEstate10K / DL3DV), down 12.7% / 23.1%; VGGT ATE falls to 0.0034 on RealEstate10K (-52.8%, the abstract's roughly halved claim) and 0.0056 on DL3DV (-23.3%). Hugging Face Papers (Daily Sept. 23), alphaXiv, the lab project page, and TencentARC/GAE-D64-1B weights index the release without metric replication.

Tencent ARC Lab (IEG), with the Hong Kong University of Science and Technology, the University of Hong Kong, and the University of Texas at Austin, submitted GAE on Sept. 21, 2026. The geometry-native autoencoder reparameterizes multi-level features from a frozen geometry foundation model into a compact latent that a generator can evolve while staying jointly decodable to appearance, depth, cameras, and point maps.

In plain terms, GAE is not another product world-model landing in the OpenWAM, LingBot, or WorldSculpt sense. It is a representation method: put geometry inside the latent the generator already uses, rather than bolting 3D on as a separate output stream.

The codec keeps the DA3 patch grid and compresses only along channels, yielding Cz in {64, 128} instead of the 3,072 channels of a single raw DA3 level. A frozen DPT head reads geometry from decoded features; a learned RGB head shares the same latent. Stage 1 trains the codec with the backbone frozen. Stage 2 freezes the codec and trains a roughly 0.93B DiT-style conditional flow with text, metric Plücker rays, and clean reference tokens.

In controlled comparisons that hold the generator and training protocol fixed, the authors report GAE-64 FVD of 225.7 on RealEstate10K and 287.0 on DL3DV, versus 258.6 and 373.2 for SD-VAE, the strongest non-GAE controlled latent under that matched DiT setup (down 12.7% and 23.1%). Independent VGGT camera ATE falls from 0.0072 to 0.0034 on RealEstate10K (-52.8%, the abstract's roughly halved claim) and from 0.0073 to 0.0056 on DL3DV (-23.3%). Those deltas are author-controlled matched-latent bake-offs, not third-party tests against Gen3R, GLD, or commercial video pipelines.

TencentARC released GAE-D64-1B inference weights on Hugging Face, a roughly 1B-parameter temporal DiT over the 64-channel geometry-native latent at 672×378 with V=81, metric Plücker conditioning, and Qwen3-0.6B text. The paper's 81-view showcase at that resolution uses a separately trained larger-scale model and is qualitative only; it is not part of the controlled ranking that produces the FVD and ATE headlines.

Hugging Face Papers listed the preprint on its Sept. 23, 2026 Daily Papers feed under ARC Lab, Tencent. alphaXiv indexes the same arXiv record with a third-party methods restatement, and the lab project page at jiah-cloud.github.io/GAE.github.io demos joint RGB, depth, camera, and point-cloud decode from one latent. What remains open is independent replication of the controlled FVD and ATE gains outside the authors' matched protocol. Lab claims, community indexes, and project demos should be weighed separately until those points are settled.