Back to Discover

Alaya Lab's WorldSculpt rebuilds cluttered scenes as hundreds of separate, editable 3D objects

WorldSculpt project teaser showing compositional editable object meshes in a shared world frame

WorldSculpt, from Alaya Lab and the University of Tokyo, takes images of a cluttered scene shot from several angles and builds a separate 3D model of each object in it, hundreds per scene, instead of one fused shape. Objects built this way can be moved and edited one by one, which games, virtual reality, simulation and robotics need. It needs the camera positions and each object marked out in advance, and its results so far come from the researchers' own tests.

On Sept. 7, 2026, Alaya Lab and the University of Tokyo posted WorldSculpt, a system that turns images of a cluttered scene, taken from several angles, into hundreds of separate 3D objects. Each object becomes its own 3D model that can be moved or edited on its own, which is what gaming, augmented and virtual reality, simulation and robotics need. A scene stored as one fused 3D shape can't be taken apart object by object.

That sets it apart from AI tools that generate whole 3D worlds, such as Marble and HY-World 2.0, which keep each scene as a single fused piece. WorldSculpt does need help up front. Along with the images, it is given each camera's position, an outline of every object in each image and a rough 3D box around each object, and it places every object it builds into one shared scene. Working out those camera positions, outlines and boxes is outside what WorldSculpt does.

The system builds on Pixal3D, an existing Tencent model that makes a 3D object from a single image, and adds a way to combine what several cameras see. The researchers say they trained it only on single objects, yet it handles crowded scenes with hundreds of objects partly hiding one another, without any training on whole scenes.

In the researchers' own tests, the shapes WorldSculpt built matched the real objects more closely than rival methods such as ShapeR did. That held on scans of real household scenes and on a new set of six detailed scenes the team built in Unreal Engine, a video game engine, with 93 to 701 objects each. When three-quarters of each object was hidden, combining many camera views let WorldSculpt hold up, while the single-image model it builds on did far worse.

The trained model files are posted on Hugging Face under Apache 2.0, a license that lets anyone use, change and share them free of charge, with code on GitHub. In a demonstration, the researchers also used WorldSculpt, with no extra training, to turn 3D worlds made by Marble and HY-World 2.0 into scenes of separate, editable objects.

WorldSculpt builds only the shapes of objects, not their surface textures or materials, and only for scenes that stay still; objects that move or bend are beyond it for now. The exact 3D models behind the team's Unreal Engine scenes are withheld for copyright reasons, and researchers who want to test against them must contact the project lead.

Nobody outside the lab has checked the results yet. It is also unclear how much error in the supplied camera positions, outlines and boxes the system can take before its shapes break down, and whether adding textures and materials later would give its objects a look good enough for simulations.