# Spatial Transfusion: From Structured Spatial Scene Understanding to Generation and Editing

DevFeed: [Spatial Transfusion: From Structured Spatial Scene Understanding to Generation and Editing](<https://devfeed.tech/articles/spatial-transfusion-from-structured-spatial-scene-understanding-to-generation-and-editing-115295.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02330>)

Author: Jaeyeon Kim, Sebastian Braun, Hannes Gamper

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Language Engineering](<https://devfeed.tech/topics/language-engineering.md>), [lora](<https://devfeed.tech/topics/lora.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generation](<https://devfeed.tech/tags/generation.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [structured](<https://devfeed.tech/tags/structured.md>)

## AI overview

Spatial Transfusion combines spatial audio understanding, generation, and editing in a first-order ambisonics framework. It extends a pretrained understanding model using flow matching over VAE latents, while keeping the backbone frozen and routing modality-specific LoRA adapters. Editing is represented as structured scene transformations drawn from a taxonomy of atomic edits; a curriculum progressively introduces generation, editing, and natural-language interaction. The abstract reports that the approach largely preserves spatial understanding and outperforms full fine-tuning on generation and editing. It supports controllable generation and editing through structured or natural-language interfaces.

## Source excerpt

arXiv:2611.02330v1 Announce Type: cross Abstract: Spatial audio conveys not only what sounds are present, but where they originate and how they interact with the environment. Recent audio-language models increasingly understand and reason about such scenes, yet spatial audio understanding, generation, and editing remain largely separate. We present Spatial Transfusion, a unified first-order ambisonics (FOA) framework that extends a pretrained spatial audio understanding model to generation and editing through flow matching over VAE latents. We formulate editing as a structured scene transformation over a taxonomy of atomic edits. Keeping the pretrained backbone frozen, we introduce modality-based LoRA routing and a curriculum that progressively adds generation, editing, and natural-language interaction. Experiments show that Spatial Transfusion largely preserves spatial understanding while outperforming full fine-tuning in generation and editing. The resulting model supports controllable FOA generation and editing through both structured and natural-language interfaces. Audio samples are available at https://microsoft.github.io/spatial-transfusion/