Training generalizable embodied agents requires realistic, varied, and simulation-ready indoor environments. Language-guided methods support semantic planning and object-level reasoning, but precise spatial grounding remains difficult. Visual generative models offer rich priors over plausible arrangements, yet their outputs require geometric reasoning to become physically valid 3D scenes. We introduce SceneBloom, an agentic framework that combines visual layout sampling with contract-guided 3D compilation. A hierarchical sampler uses diffusion priors to generate furniture arrangements within functional zones. A compiler then uses relations extracted from each proposal to guide geometric fitting and local pose correction. Across five room types, SceneBloom attains the highest visual quality and physical stability among the compared methods and is the only method with no detected mesh collisions and out-of-bounds objects. Its layouts are also closer to the distribution of real rooms than those of all compared baselines. Object navigation policies trained in these realistic scenes achieve a higher success rate in held-out real scans than those trained on an equal number of real training scans. Code is available at this https URL.
The challenge of generating physically valid indoor scenes
Embodied agents operating in indoor spaces must reason about object locations, spatial relationships, and physical stability. Generating such scenes from scratch is difficult: language-guided methods support semantic planning but struggle with precise spatial grounding, while visual generative models produce plausible arrangements that lack geometric validity when reconstructed as 3D meshes. SceneBloom addresses this by pairing a visual sampler with a compiler that corrects geometry while preserving the functional relationships captured in the original layout.
From visual priors to geometric reality: the two-stage design
SceneBloom operates in two stages. First, a hierarchical sampler generates furniture arrangements within functional zones using diffusion priors. Second, a compiler extracts relations from each proposal and uses them to guide geometric fitting and local pose correction. This division of labor lets the sampler focus on coherent arrangement while the compiler ensures physical validity.
Hierarchical sampling: diffusion priors within functional zones
The sampler divides a room into functional zones — such as sleeping or working areas — using a vision-language model that looks up similar example layouts. For each zone j, the diffusion model produces one image from a short prompt q_j, low-rank adapters Λ_j tuned to the catalog assets, and a random seed ε_j:
I_j = G(q_j, Λ_j; ε_j)This image proposes a coherent furniture group for the zone, and different seeds vary the layout without changing which assets the compiler is meant to build.
The zone-level design means the sampler never generates a full room at once. Instead, it handles one functional group at a time, which reduces the search space and makes each arrangement more likely to be both visually plausible and spatially organized.
Contract-guided 3D compilation: relations as geometric constraints
Before geometric correction begins, a vision-language model extracts a relation contract C_j from the original proposal. Each tuple in C_j names a subject object u, a relation r, a target object or wall v, and a confidence γ. The compiler uses only relations with confidence at least 0.75 — examples include a chair facing a table, a bookshelf beside a cabinet, or an object's back against a wall.
The compiler first fits objects around each zone's anchor while using the contract to guide separation. It then freezes the zone's footprint — the floor area occupied by its furniture — and moves the whole zone as one unit to satisfy the room plan, enlarging the room only if the zones' furniture footprints cannot fit. After assembly, simple geometric checks decide whether each recorded relation still holds; local rotations or translations restore a violated relation only when the room boundary and object footprints allow it. This gives the system a way to correct coordinates while preserving the qualitative structure sampled in the image.
Experimental results: quality, stability, and layout distribution
Across 50 generated scenes in five room types (bedroom, living room, dining room, office, bathroom), SceneBloom leads all compared methods — Holodeck, SceneWeaver, and SAGE — on every evaluated metric.
SceneBloom is the only method with 0.0% detected object and scene collisions and 0.000 out-of-bounds objects per scene. Its movable-object stability rate reaches 75.3%, compared with 45.3% for the next-best method. Four GPT-4o realism scores from 0 to 10 are highest across all axes: realism, functionality, layout coherence, and completeness. Layout Distribution Discrepancy (LDD) against both Matterport3D and Scan2CAD real-layout references is lowest for SceneBloom: 0.1759 against Matterport3D and 0.1340 against Scan2CAD, while the next-best baseline scores 0.2009 and 0.1797 respectively.
Transfer learning: navigation policies in simulated scenes
Object-goal navigation policies trained from scratch in SceneBloom-generated scenes achieve higher success rates in held-out real HM3D scans than policies trained on an equal number of real training scans. At 120 generated houses, SceneBloom-trained policies reach 35.63% success and 16.20% SPL, while real-scan training yields 32.87% success and 15.43% SPL. The untrained policy baseline reaches 29.70% success. SceneBloom also leads at 40 and 80 house budgets.
When fine-tuning a pretrained open-vocabulary policy on 864 real episodes from HM3D-OVON, SceneBloom reaches 48.57% success versus 48.26% with no fine-tuning and 47.57% after real-scan continuation. On held-out generated scenes with unseen goals, SceneBloom achieves 53.19% success and 38.89% SPL, the highest values in that comparison.
Why sampling and repair belong together
The ablation study shows that the combination of visual sampling and contract-guided repair is greater than either stage alone. With visual proposals, adding contracts cuts the share of recorded relations left unsatisfied from 37.45% to 4.25%, raises layout coherence from 4.31 to 4.65, and improves realism from 5.66 to 5.83, while collisions remain at zero. With textual starting layouts, the same step raises scene collisions from 4% to 8% and lowers stability from 98.44% to 94.27%, revealing that contracts cannot make a crowded arrangement safe by themselves. Visual sampling supplies the furniture relationships, while contract-guided compilation carries them through geometric correction — neither stage is presented as a substitute for the other.
Practical considerations
SceneBloom averages 2.7 minutes per layout, compared with 73.6 minutes for SAGE — roughly 27 times faster. Holodeck takes 1.6 minutes but 78% of its evaluated scenes contain a collision. The reported time covers layout generation only; rendering and physical settling are excluded. The paper reports 34,867 language-model tokens and a cost of USD 0.0057 per scene, though it does not report the total compute required to train asset adapters, so the per-scene figure is not the complete setup bill.
The scope of the evaluation is specific: 10 requests per room type, automated GPT-4o judges for visual ratings, and LDD evaluated in only three room types. No collisions were detected in this sample, but 24.7% of movable objects still fall outside the paper's stability threshold. Navigation transfer is evaluated in simulation on HM3D scans, not on a physical robot, and the fine-tuning edge over a pretrained policy is small — 0.31 percentage points in success rate on the full 864-episode real set.
Use SceneBloom when spatial organization matters to training
SceneBloom joins visual arrangement sampling to relation-aware geometric repair. Its evaluated rooms support stronger navigation learning than equal-count real-scan training in the from-scratch setting. Use it to generate diverse, spatially organized indoor scenes for simulation-based navigation training when the target task depends on furniture layout, not just object inventory. Treat zero detected collisions as a result on 50 evaluated rooms, not a universal guarantee. The stability rate is 75.3%, the fine-tuning edge over a pretrained policy is small, tests remain in simulation, and the cost of training asset adapters is not reported.