# The balance between feature learning and collapse in dynamical systems-based generative models

DevFeed: [The balance between feature learning and collapse in dynamical systems-based generative models](<https://devfeed.tech/articles/the-balance-between-feature-learning-and-collapse-in-dynamical-systems-based-generative-models-125521.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12156>)

Author: Julian Brandon, Bruno Loureiro, N Alex Cayco-Gajic, Arthur Pellegrino

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [bias](<https://devfeed.tech/tags/bias.md>), [budget](<https://devfeed.tech/tags/budget.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [feature](<https://devfeed.tech/tags/feature.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generative-models](<https://devfeed.tech/tags/generative-models.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [inference](<https://devfeed.tech/tags/inference.md>), [learning](<https://devfeed.tech/tags/learning.md>), [learning-theory](<https://devfeed.tech/tags/learning-theory.md>), [linear](<https://devfeed.tech/tags/linear.md>), [math-ds](<https://devfeed.tech/tags/math-ds.md>), [operators](<https://devfeed.tech/tags/operators.md>), [rank](<https://devfeed.tech/tags/rank.md>), [stable](<https://devfeed.tech/tags/stable.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [systems](<https://devfeed.tech/tags/systems.md>), [trade](<https://devfeed.tech/tags/trade.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

The paper uses dynamical systems and operator theory to express gradient flow for a general class of stochastic differential equations as a composition of linear operators, linking mode collapse to low-rank biases. In variational inference and flow-matching tasks, it reports two mechanisms: collapsed models can be stable fixed points of gradient flow, or sequential feature learning can delay learning smaller modes enough to create effective collapse within a fixed training budget. The authors frame this as a trade-off between feature learning and collapse and propose the operator view as a step toward a unified account of learning dynamics in generative models such as normalising flows and diffusion.

## Source excerpt

arXiv:2611.12156v1 Announce Type: cross Abstract: Over the past decade, a growing body of work has established that implicit biases in learning dynamics fundamentally shape the solutions found by neural networks, governing their learned features and generalisation performance. However, despite these strides in deep learning theory, far less is known about representation learning in generative models based on dynamical systems, from normalising flows to diffusion. Importantly, empirical evidence suggests that continuous-time generative models can be prone to mode collapse, yet a learning theory explaining such biases remains elusive. To address this, we use tools from dynamical systems and operator theory to express the gradient flow of a general class of stochastic differential equations as a composition of linear operators, allowing us to relate collapse to low-rank biases in generative models. In variational inference and flow-matching tasks, we show that this bias can manifest in two different mechanisms: i) where collapsed models are stable fixed points of the gradient flow, or ii) in sequential feature learning, where smaller modes take progressively longer to be learned, producing an effective collapse for a fixed training budget. Together these findings point to a trade-off between feature learning and collapse, suggesting an operator view as a step towards a unified framework for understanding the learning dynamics of dynamical systems-based generative models.