Back to Discover

Nankai and Rightly Robotics DeformSmith builds sim-ready deformable assets with robot pick-and-place in the loop

DeformSmith project still: robot Approach, Grasp (squeeze), and Lift of a deformable plush with matching mesh twin under gripper contact

On Sept. 16, 2026, Nankai University and Rightly Robotics submitted DeformSmith (arXiv:2609.18620), a physics-harness hierarchical agent stack that turns text or a single image into simulation-ready deformable assets with robot pick-and-place in the loop. Author-reported GPT-6 Astra ratings and human win rates versus PhysGen3D / PhysGM / PhysX-Omni stay lab claims; Hugging Face Papers (Daily Papers Sept. 21) and alphaXiv index the release without metric replication.

Nankai University and Rightly Robotics submitted DeformSmith on Sept. 16, 2026, a physics-harness stack that builds sim-ready deformable assets from text or an image. That matters for robot labs that need soft objects they can grasp and transport in simulation, not only meshes that look right when still.

In plain terms, DeformSmith is not a learned world-model video predictor and not a one-shot image-to-physics guess. It is an asset builder: agents propose geometry, mass and contact, neo-Hookean material parameters, and grasp plans, then a shared physics harness runs material-point-method probes and simulated pick-and-place to accept or revise those choices until the asset is ready for manipulation data.

Four layers run under that harness. L0 reconstructs mesh, particles, Gaussians, and a collision proxy. L1 sets mass, volume, and contact. L2 tunes Young's modulus, Poisson's ratio, and damping with drop, compression, and stability probes. L3 plans pick-and-place on a RealMan RM65-6F in SAPIEN, then feeds action and material revisions back through the harness. The authors say the Planner, Designer, and Critic use GPT-5.6 Sol under a rule-checked contract, with Warp MPM for deformable probes.

On 39 cases (30 text-driven, 9 image-based), authors report GPT-6 Astra mean ratings of physical realism 0.70, photorealism 0.58, and semantic consistency 0.82, versus PhysGen3D at 0.46 / 0.34 / 0.80. Blinded human pairwise win rates versus PhysGen3D, PhysGM, and PhysX-Omni are 68% to 75% for physical plausibility, 88% to 96% for visual quality, and 63% to 76% for semantic consistency (ties excluded). An ablation they report raises pick-and-place task success from 40% to 67%, material pass from 67% to 83%, and joint success from 27% to 57% with manipulation-guided refinement. Those figures are author-reported, not third-party re-runs.

Hugging Face Papers listed the preprint on its Sept. 21, 2026 Daily Papers feed, and alphaXiv indexes the same arXiv record with a third-party methods summary. The live project page at can-lee.github.io/deformsmith-web restates the tagline "Generate the object. Establish its physics. Put it into interaction." and hosts pick-and-place replays. The work is deformable asset generation with robot-in-loop validation, distinct from learned world-model video predictors Pelican-Sim, Odyssey-3, XPACE, and OpenWAM.

Stated limits keep the work to homogeneous volumetric neo-Hookean solids; fluids and granular materials are out of scope, and contact is approximate. The real-world section frames generated geometry and parameters as a potential init for later video-based identification such as DeformMaster or EMPM, not as a completed hardware transfer. What remains open is independent replication of the GPT-6 Astra and human preference scores, how far text or image priors match measured materials, and whether broader material models close the gap. Author benches, community indexes, and project demos should be weighed separately until those points are settled.