# stable-diffusion

A latent text-to-image diffusion model with reference software for image generation and modification.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Keystone: Judging Previews at Key Steps for Test-Time Search in Diffusion Models

DevFeed: [Keystone: Judging Previews at Key Steps for Test-Time Search in Diffusion Models](<https://devfeed.tech/articles/keystone-judging-previews-at-key-steps-for-test-time-search-in-diffusion-models-145530.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25530>)

Author: Jiajun Fan, Chaoran Cheng, Ziwen Wang, Changyi Yang, Bangji Yang, Qinglong Zheng, Ge Liu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Image](<https://devfeed.tech/topics/image.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [keystone](<https://devfeed.tech/tags/keystone.md>), [preview](<https://devfeed.tech/tags/preview.md>), [search](<https://devfeed.tech/tags/search.md>), [test](<https://devfeed.tech/tags/test.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>)

### AI overview

Keystone is a training-free method for judging diffusion-model previews early in denoising, where the abstract says outcomes are often already determined by the seed or a sampler's initial stochastic steps. Instead of applying reward models designed for finished images--which read early previews near chance on several prompt types--Keystone turns prompts into typed questions about objects, counts, colours, and positions, then uses an off-the-shelf multimodal decision model to answer them. On GenEval with SD3.5-Medium, its adaptive search and pruning reportedly beat best-of-8 with every other verifier using less than half the generator compute. As a drop-in verifier, its score also improved best-of-N, progressive seed pruning, and FK steering there. Without retuning, both searches remained above the strongest other verifier's best-of-N on average across four base models and three benchmarks.

### Source excerpt

arXiv:2611.25530v1 Announce Type: new Abstract: Text-to-image diffusion and flow models yield a preview, their clean-image estimate, at every denoising step. When they miscount, drop or misplace objects, test-time search is in a bind: its verifiers are reward models built for finished images, so judging early misreads blurry previews, and judging late, as best-of-N does, pays to fully denoise doomed candidates. We find that the image is not the problem: the outcome is settled as denoising starts, by the seed or by a stochastic sampler's first steps, and already shows in early previews; what fails on them is the judge, as reward models read them near chance on several kinds of prompts. We introduce Keystone, a training-free judge of previews that compiles a prompt into typed questions about its objects, counts, colours and positions and has an off-the-shelf multimodal decision model answer them. Asked this way, the judge reads counts and positions from early previews better than reward models read them from finished images. Searching at these key steps on GenEval with SD3.5-Medium, Keystone's adaptive search and pruning significantly outperform best-of-8 with every other verifier at under half its generator compute, and as a drop-in verifier there, the Keystone score significantly improves best-of-N, progressive seed pruning and FK steering; without retuning, both searches stay significantly above the strongest other verifier's best-of-N on average over four base models and three benchmarks. Keystone thus moves test-time judgment to the key steps where the outcome already shows.

## Process-Knowledge-Guided Latent Diffusion for Generative Design of Porous Architectures: A Case Study in Aerosol Jet Printing

DevFeed: [Process-Knowledge-Guided Latent Diffusion for Generative Design of Porous Architectures: A Case Study in Aerosol Jet Printing](<https://devfeed.tech/articles/process-knowledge-guided-latent-diffusion-for-generative-design-of-porous-architectures-a-case-study-in-aerosol-jet-printing-145030.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25264>)

Author: Ngoc Dang Khoa Nguyen, Hyunwoong Ko

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [VAE](<https://devfeed.tech/topics/vae.md>)

Tags: [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [cs-ce](<https://devfeed.tech/tags/cs-ce.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [design](<https://devfeed.tech/tags/design.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [flow](<https://devfeed.tech/tags/flow.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generative-design](<https://devfeed.tech/tags/generative-design.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [printing](<https://devfeed.tech/tags/printing.md>), [process](<https://devfeed.tech/tags/process.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [score](<https://devfeed.tech/tags/score.md>), [topology](<https://devfeed.tech/tags/topology.md>)

### AI overview

The study shows that process knowledge can steer latent diffusion toward porous layouts with lower process-dependent utilization scores. Its generator combines a VQ-VAE latent representation, topology U-Net, ControlNet branch for domain conditioning, and a separately trained score regressor whose gradient guides denoising. For aerosol jet printing, atomizer and sheath gas flows enter the score through a flow-focusing relation. Across seven modeled flow settings, the lowest-scoring layout differed between settings in 191 of 500 design domains. In sampling tests on three validation domains and one test domain, mean directly computed scores fell from 0.642 with domain conditioning alone to 0.636 at steering weight 300 and 0.450 at 12,000 (112 scores per weight; lower is better). These findings apply under the paper's stated assumptions.

### Source excerpt

arXiv:2611.25264v1 Announce Type: new Abstract: Printed porous layouts interleave functional material with open channels, and the layout that uses the material best depends on how the process settings shape the deposit. Generative models propose layouts quickly, but a generator that does not receive the process setting samples from the same distribution at every setting, even when the process-dependent score ranks candidate layouts differently. This paper examines whether process knowledge can steer a latent diffusion generator toward layouts suited to a given process setting in a prescribed design domain. We formulate a process-dependent utilization score in which a process model sets the length over which the utility of printed material decays with distance from a channel. The generator combines a vector-quantized variational autoencoder (VQ-VAE) latent representation, a topology U-Net, a ControlNet domain branch, and a separately trained regressor that predicts the score from noised latents; the domain branch conditions each denoising step and the regressor gradient steers it. The method is applied to aerosol jet printing, where the atomizer and sheath gas flows enter the score through the flow-focusing relation. Across seven modeled flow settings, the lowest-scoring candidate layout of a domain differed between settings in 191 of 500 domains. In a sampling study, the mean directly computed score on three validation domains and one test domain was 0.642 with domain conditioning alone, 0.636 at a weight of 300, and 0.450 at 12,000 (112 scores per weight; lower is better). The results support process-knowledge-guided generation as an approach to process-specific porous design under the stated assumptions.

## Erased but Recoverable: On the Fragility of Concept Erasure Under Fine-Tuning

DevFeed: [Erased but Recoverable: On the Fragility of Concept Erasure Under Fine-Tuning](<https://devfeed.tech/articles/erased-but-recoverable-on-the-fragility-of-concept-erasure-under-fine-tuning-143146.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24220>)

Author: Jun Li, Lizhi Xiong, Ziqiang Li, Yu Liu, Zenglin Shi

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [lora](<https://devfeed.tech/topics/lora.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [lora](<https://devfeed.tech/tags/lora.md>), [persistence](<https://devfeed.tech/tags/persistence.md>), [profiling](<https://devfeed.tech/tags/profiling.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [target](<https://devfeed.tech/tags/target.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>)

### AI overview

Concept erasure in text-to-image diffusion models can weaken or reverse after downstream fine-tuning such as DreamBooth, LoRA, or Custom Diffusion. The authors report that fine-tuning preserves certain high-contribution attention heads with stable output subspaces and response magnitudes, and that these heads in erased models can realign with their pretrained representations. Their proposed SHACE method profiles stability-critical heads and keeps their parameter blocks unchanged during erasure. Experiments reportedly show improved resistance to concept recovery across varied fine-tuning settings while maintaining target suppression and non-target generation quality.

### Source excerpt

arXiv:2611.24220v1 Announce Type: new Abstract: Existing concept erasure methods for text-to-image diffusion models primarily focus on suppressing target concepts while preserving non-target generation quality. However, in practical applications, open-source diffusion models are often further adapted through DreamBooth, LoRA, or Custom Diffusion. Such adaptations can reshape internal representations, thereby weakening or even reversing the erasure effects achieved by existing methods. We identify erasure persistence as a key yet underexplored dimension of concept erasure and analyze how downstream fine-tuning reshapes internal representations. Our analysis reveals that fine-tuning of the original model does not uniformly modify the model, but preserves certain high-contribution structures, particularly attention heads with stable output subspaces and response magnitudes. We also observe that, after downstream fine-tuning of concept-erased models, these heads' output representations more readily realign with their pretrained counterparts. Motivated by this observation, we propose Stable-Head-Aware Concept Erasure (SHACE), which improves erasure persistence by preserving stable attention structures during concept erasure. SHACE identifies stability-critical attention heads through Attention-Head Stability Profiling and keeps their corresponding parameter blocks unchanged during erasure via Stable-Head Gradient Preservation. Extensive experiments show that SHACE improves resistance to concept recovery across diverse fine-tuning settings while maintaining strong suppression and non-target generation quality.

## Training-Free Domain Generation for One-Shot Federated Learning with Server-Side Pre-trained Diffusion Models

DevFeed: [Training-Free Domain Generation for One-Shot Federated Learning with Server-Side Pre-trained Diffusion Models](<https://devfeed.tech/articles/training-free-domain-generation-for-one-shot-federated-learning-with-server-side-pre-trained-diffusion-models-142722.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23908>)

Author: Yixuan Tong, Yuanguo Bi, Mingjian Zhi, Wenchao Xu, Yuan Wu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [class](<https://devfeed.tech/tags/class.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classification-accuracy](<https://devfeed.tech/tags/classification-accuracy.md>), [client](<https://devfeed.tech/tags/client.md>), [communication](<https://devfeed.tech/tags/communication.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [data](<https://devfeed.tech/tags/data.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [distributions](<https://devfeed.tech/tags/distributions.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [federated-learning](<https://devfeed.tech/tags/federated-learning.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [guidance](<https://devfeed.tech/tags/guidance.md>)

### AI overview

FedDoG addresses heterogeneous client data in one-shot federated learning by using a server-side pre-trained diffusion model to generate domain-aligned synthetic data. Clients send compact domain representations extracted with a frozen image encoder; a pre-trained image-conditioning adapter uses them as visual guidance. The method adjusts conditioning strength based on domain concentration and filters generated samples for class consistency and predictive entropy. On three benchmarks, the authors report higher-quality synthetic images and better classification accuracy than existing OSFL baselines, alongside 99.9% less client upload and 91.5% less client computation time. These are reported experimental results; the excerpt gives no benchmark names or further evaluation details.

### Source excerpt

arXiv:2611.23908v1 Announce Type: new Abstract: One-shot federated learning (OSFL) has attracted increasing attention for its ability to substantially reduce communication overhead by limiting client-server interaction to a single communication round. Nevertheless, this limited interaction makes learning from heterogeneous client distributions challenging. Diffusion-based OSFL approaches seek to address this challenge by synthesizing data that approximate client distributions. However, existing diffusion-based approaches often impose substantial computational costs on clients and lack effective visual guidance to generate data aligned with client distributions. To address these limitations, we propose FedDoG, a training-free Domain Generation framework that leverages server-side pre-trained diffusion models to synthesize data aligned with client domains. FedDoG extracts compact domain representations using a frozen image encoder, substantially reducing client-side computational and communication costs. These representations are incorporated into the diffusion model through a pre-trained image-conditioning adapter, thereby providing explicit visual guidance for domain generation. To further improve the quality of synthetic data, FedDoG adapts the strength of visual conditioning according to domain concentration and filters generated samples based on class consistency and predictive entropy. Experiments on three benchmarks demonstrate that FedDoG generates higher-quality synthetic images and outperforms existing OSFL baselines in classification accuracy. Meanwhile, FedDoG reduces client upload and computation time by 99.9% and 91.5%, respectively. The code is available at https://github.com/YXTong17/FedDoG.

## Low-Dimensional Adaptation of Rectified Flow: A Diffusion and Stochastic Localization Perspective

DevFeed: [Low-Dimensional Adaptation of Rectified Flow: A Diffusion and Stochastic Localization Perspective](<https://devfeed.tech/articles/low-dimensional-adaptation-of-rectified-flow-a-diffusion-and-stochastic-localization-perspective-142502.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2601.15500>)

Author: Saptarshi Roy, Alessandro Rinaldo, Purnamrita Sarkar

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [complexity](<https://devfeed.tech/tags/complexity.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [drift](<https://devfeed.tech/tags/drift.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [flow](<https://devfeed.tech/tags/flow.md>), [generation](<https://devfeed.tech/tags/generation.md>), [intrinsic](<https://devfeed.tech/tags/intrinsic.md>), [iteration](<https://devfeed.tech/tags/iteration.md>), [localization](<https://devfeed.tech/tags/localization.md>), [math-st](<https://devfeed.tech/tags/math-st.md>), [model](<https://devfeed.tech/tags/model.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [schedule](<https://devfeed.tech/tags/schedule.md>), [simulations](<https://devfeed.tech/tags/simulations.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [stat-th](<https://devfeed.tech/tags/stat-th.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>)

### AI overview

The paper studies whether rectified flow (RF) sampling can exploit low intrinsic dimension in a target distribution. With sufficiently accurate drift estimates and an endpoint-adapted time discretization, it gives an iteration complexity of O(k/ε), up to logarithmic factors, for total variation precision ε and intrinsic dimension k. It also establishes an equivalence between DDPM and a stochastic version of RF through a connection to stochastic localization, then proposes a stochastic RF sampler that adapts to low dimension under milder drift-accuracy requirements and a specific time schedule. Simulations on synthetic and text-to-image data are reported as evidence for the schedules' efficacy.

### Source excerpt

arXiv:2601.15500v5 Announce Type: replace-cross Abstract: In recent years, Rectified flow (RF) has gained considerable popularity largely due to its generation efficiency and state-of-the-art performance. In this paper, we investigate how well RF automatically adapts to the intrinsic low dimensionality of the support of the target distribution to accelerate sampling. We show that, using a carefully designed end-point adapted time-discretization scheme and with sufficiently accurate drift estimates, the RF sampler enjoys an iteration complexity of order $O(k/\varepsilon)$ (up to log factors), where $\varepsilon$ is the precision in total variation distance and $k$ is the intrinsic dimension of the target distribution. In addition, we show that the denoising diffusion probabilistic model (DDPM) procedure is equivalent to a stochastic version of RF by establishing a novel connection between these processes and stochastic localization. Building on this connection, we further design a stochastic RF sampler that also adapts to the low-dimensionality of the target distribution under mild requirements on the accuracy of the drift estimates, and also with a specific time schedule. We illustrate the efficacy of newly designed time-discretization schedules with simulations on the synthetic data and text-to-image (T2I) data experiments.

## Dual-Manifold Variational Autoencoders

DevFeed: [Dual-Manifold Variational Autoencoders](<https://devfeed.tech/articles/dual-manifold-variational-autoencoders-142732.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23964>)

Author: Hongxu Chen, Chun Wang, Yanghao Wang, Hongxiang Li, Zhen Wang, Zhixiang Zhang, Long Chen

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [VAE](<https://devfeed.tech/topics/vae.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [ImageNet](<https://devfeed.tech/topics/imagenet.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [cfg](<https://devfeed.tech/tags/cfg.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [dimension](<https://devfeed.tech/tags/dimension.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [domains](<https://devfeed.tech/tags/domains.md>), [drift](<https://devfeed.tech/tags/drift.md>), [features](<https://devfeed.tech/tags/features.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generator](<https://devfeed.tech/tags/generator.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [inference](<https://devfeed.tech/tags/inference.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [space](<https://devfeed.tech/tags/space.md>), [subspace](<https://devfeed.tech/tags/subspace.md>), [training](<https://devfeed.tech/tags/training.md>), [ts](<https://devfeed.tech/tags/ts.md>)

### AI overview

DM-VAE addresses off-manifold drift in high-dimensional VAE latents by pairing a reconstructive branch with a low-dimensional branch that supplies a smooth topological skeleton to anchor the latent manifold. It avoids relying on a frozen foundation model as its primary regularizer. Topological representation alignment (TRA) supervises intermediate generator features during training; internal guidance with topological skeleton (IG_TS) guides structural components at inference while preserving high-frequency detail. On ImageNet-1K at 256x256, the foundation-model-free method reports rFID 0.12 and FID50K 1.28 without classifier-free guidance (CFG), or 1.20 with CFG. Combined orthogonally with DINOv2, it reports FID50K 1.11.

### Source excerpt

arXiv:2611.23964v1 Announce Type: cross Abstract: In latent diffusion models, expanding the latent dimension of variational autoencoders (VAEs) is essential to push the upper bound of reconstruction quality. However, without explicit structural constraints, such unregularized expansion easily yields a rugged and ill-conditioned latent manifold, causing generated latents to drift off the manifold during generation (i.e., off-manifold problem). Recent works smooth this manifold by using external foundation models (e.g., DINOv2) as a topological regularizer. However, we argue that tying the latent space to frozen external priors inherently restricts generalization: bound by their pre-training distributions and semantic biases, they may fail to capture faithful visual details on out-of-distribution domains. Observing that low-dimensional latents naturally form a smoother, generation-friendly manifold, we propose dual-manifold VAE (DM-VAE) to leverage internal representations as a topological regularizer. Specifically, DM-VAE pairs a high-dimensional reconstructive branch with a low-dimensional branch that extracts a clean topological skeleton, anchoring the high-dimensional manifold to this smooth subspace. Building upon this framework, we further introduce topological representation alignment (TRA) to supervise intermediate generator features during training, and internal guidance with topological skeleton (IG_TS) to guide structural components while preserving high-frequency details at inference. On ImageNet-1K (256x256), our foundation-model-free DM-VAE achieves a state-of-the-art rFID of 0.12 and a generation FID50K of 1.28 (w/o CFG) and 1.20 (w/CFG), and further reaches 1.11 when orthogonally combined with DINOv2.

## FaceRiff: Data-Efficient Unified Text-to-3D Face Asset Synthesis via Geometric Conditioning

DevFeed: [FaceRiff: Data-Efficient Unified Text-to-3D Face Asset Synthesis via Geometric Conditioning](<https://devfeed.tech/articles/faceriff-data-efficient-unified-text-to-3d-face-asset-synthesis-via-geometric-conditioning-139922.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21867>)

Author: Elly Akhoundi, Mathieu Lamarre, Thibault Goyallon, Patrick Anderson, \\'Etienne Danvoye

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [3D](<https://devfeed.tech/topics/3d.md>), [peft](<https://devfeed.tech/topics/peft.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [asset](<https://devfeed.tech/tags/asset.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-gr](<https://devfeed.tech/tags/cs-gr.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [image](<https://devfeed.tech/tags/image.md>), [independent](<https://devfeed.tech/tags/independent.md>), [lighting](<https://devfeed.tech/tags/lighting.md>), [lora](<https://devfeed.tech/tags/lora.md>), [maps](<https://devfeed.tech/tags/maps.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [text](<https://devfeed.tech/tags/text.md>)

### AI overview

FaceRiff generates text-conditioned 3D face assets--mesh, albedo, and displacement--in a shared UV space, using a pretrained SDXL model with task-specific LoRA adapters and UV normal maps for geometric conditioning. It was trained on 2,049 scans from about 350 identities on a single GPU, updating less than 3% of backbone parameters per task, and produces assets without per-instance optimization. The abstract reports greater per-prompt diversity than FaceG2E and DreamFace while maintaining prompt alignment and fixed-topology UV consistency; it also generates lighting-independent albedo for PBR relighting and pore-scale displacement. In a blind study of 45 industry professionals, perceived realism was substantially higher than DreamFace and approached that of real captures.

### Source excerpt

arXiv:2611.21867v1 Announce Type: cross Abstract: High-resolution facial datasets with consistent UV-mapped geometry, albedo, and physically based displacement are scarce, yet such assets are essential for photorealistic digital-human rendering. Existing 3D face generation methods are often slow, single-modality, or require large specialized datasets. Meanwhile, large image diffusion models encode strong facial priors but do not natively produce lighting-independent textures, pore-scale displacement, or UV-aligned geometry. We present FaceRiff, a unified, data- and parameter-efficient framework for generating complete 3D face assets (mesh, albedo, and displacement) directly from text in a shared UV space. FaceRiff adapts a pretrained SDXL backbone with task-specific LoRA adapters and uses UV normal maps as geometric conditioning: per-instance normals guide albedo and displacement synthesis, while a dataset-mean normal provides a canonical spatial anchor for mesh generation. Using only 2,049 scans from approximately 350 identities and training on a single GPU, FaceRiff updates less than 3% of the backbone parameters per task and produces render-ready 3D face assets without per-instance optimization. Compared with FaceG2E and DreamFace, FaceRiff produces greater per-prompt diversity while preserving prompt alignment and fixed-topology UV consistency, and additionally generates lighting-independent albedo suitable for PBR relighting and pore-scale displacement absent from prior text-driven baselines. A blind study with 45 industry professionals shows that FaceRiff achieves substantially higher perceived realism than DreamFace, approaching that of real captures. Overall, FaceRiff demonstrates that pretrained diffusion priors can be adapted to unified 3D face asset synthesis under limited data.

## RiDEx: Reward-Informed Directional Exploration for Diffusion Reinforcement Learning

DevFeed: [RiDEx: Reward-Informed Directional Exploration for Diffusion Reinforcement Learning](<https://devfeed.tech/articles/ridex-reward-informed-directional-exploration-for-diffusion-reinforcement-learning-141725.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22052>)

Author: Song Yan, Chenfeng Wang, Wei Zhai, Xingchen Wang, Tao Zhang, Yawen Shao, Shi Pan, Min Li, Zheng-Jun Zha

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [post-training](<https://devfeed.tech/topics/post-training.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [images](<https://devfeed.tech/tags/images.md>), [influence](<https://devfeed.tech/tags/influence.md>), [local](<https://devfeed.tech/tags/local.md>), [magnitude](<https://devfeed.tech/tags/magnitude.md>), [models](<https://devfeed.tech/tags/models.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [random](<https://devfeed.tech/tags/random.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [response](<https://devfeed.tech/tags/response.md>), [reward](<https://devfeed.tech/tags/reward.md>), [space](<https://devfeed.tech/tags/space.md>), [stable-diffusion](<https://devfeed.tech/tags/stable-diffusion.md>)

### AI overview

RiDEx directs diffusion-model exploration toward reward-relevant subspaces of the denoising velocity field, aiming to use perturbations more efficiently than isotropic random exploration. Conditional residuals or differentiable reward gradients propose directions; equal-and-opposite perturbations from the same generation state measure their terminal reward effect. Those reward differences provide signed local supervision, while all generated images contribute to DiffusionNFT learning. On Stable Diffusion 3.5 Medium, RiDEx achieved the highest peak rewards among evaluated methods on four alignment and preference tasks. Its gradient-direction version reached target rewards 15x faster on HPSv2 and 29x faster on PickScore than DiffusionNFT, measured in H200 GPU hours. Controlled experiments reported stronger reward responses at equal perturbation magnitude and improved curve-average reward across all four tasks when measured credit was applied.

### Source excerpt

arXiv:2611.22052v1 Announce Type: new Abstract: Reward-based diffusion post-training updates model parameters from task rewards. While existing approaches improve sampling and policy optimization, they still rely largely on isotropic random perturbations in a high-dimensional latent space for exploration. Such variation often spends the perturbation budget on changes with little influence on the target reward, making exploration inefficient. We study where diffusion models should explore and how to learn from that exploration. We introduce Reward-Informed Directional Exploration (RiDEx), which concentrates local exploration in reward-relevant subspaces of the denoising velocity field. Conditional residuals or differentiable reward gradients propose the subspace, and equal-and-opposite perturbations at the same generation state measure its terminal reward response. The reward difference supplies signed local supervision in the explored coordinates, while all generated images contribute to global DiffusionNFT learning. RiDEx thereby couples exploration-space selection with feedback about the changes it produces. On Stable Diffusion 3.5 Medium, RiDEx attains the highest peak rewards among the evaluated methods on four alignment and preference tasks. Its gradient-direction instantiation achieves 15x and 29x target-attainment speedups over DiffusionNFT on HPSv2 and PickScore, measured in H200 GPU hours. Controlled experiments confirm that reward-aligned directions elicit stronger responses at equal perturbation magnitude, and that applying their measured credit improves curve-average reward across all four tasks.

## Mix-Opt: Mixed Optimization for Memory-Efficient Personalization of Text-to-Image Diffusion Models

DevFeed: [Mix-Opt: Mixed Optimization for Memory-Efficient Personalization of Text-to-Image Diffusion Models](<https://devfeed.tech/articles/mix-opt-mixed-optimization-for-memory-efficient-personalization-of-text-to-image-diffusion-models-141525.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2507.10029>)

Author: Seokeon Choi, Sunghyun Park, Hyoungwoo Park, Jeongho Kim, Sungrack Yun

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [backpropagation](<https://devfeed.tech/tags/backpropagation.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [dit](<https://devfeed.tech/tags/dit.md>), [edge-devices](<https://devfeed.tech/tags/edge-devices.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [image](<https://devfeed.tech/tags/image.md>), [memory](<https://devfeed.tech/tags/memory.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [overfitting](<https://devfeed.tech/tags/overfitting.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [performance](<https://devfeed.tech/tags/performance.md>), [personalization](<https://devfeed.tech/tags/personalization.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [process](<https://devfeed.tech/tags/process.md>), [quality](<https://devfeed.tech/tags/quality.md>), [resources](<https://devfeed.tech/tags/resources.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>)

### AI overview

Mix-Opt combines backpropagation on low-resolution images (BP-low) with zeroth-order optimization on high-resolution images (ZO-high) to personalize text-to-image diffusion models with lower memory use. BP-low adapts target-specific features efficiently but can distort structure; ZO-high refines high-resolution detail with minimal memory overhead but converges slowly without prior adaptation. A timestep-aware probabilistic function selects between them during training, aiming to limit BP-low overfitting when structural information matters and use ZO-high more effectively as training progresses. Experiments on U-Net- and DiT-based models report competitive personalization with significantly reduced memory consumption, without changing inference or increasing its latency.

### Source excerpt

arXiv:2507.10029v3 Announce Type: replace-cross Abstract: Memory-efficient personalization is essential for adapting text-to-image diffusion models while preserving user privacy and operating within the limited computational resources of edge devices. To this end, we propose Mix-Opt, a novel mixed optimization framework that adaptively chooses between backpropagation on low-resolution images (BP-low) and zeroth-order optimization on high-resolution images (ZO-high), guided by the characteristics of the diffusion process. As observed in our experiments, BP-low efficiently adapts the model to target-specific features but suffers from structural distortions due to resolution mismatch. Conversely, ZO-high refines high-resolution details with minimal memory overhead but faces slow convergence when applied without prior adaptation. By complementing both methods, our framework leverages BP-low for effective personalization while using ZO-high to maintain structural consistency, achieving memory-efficient and high-quality fine-tuning. To maximize the efficacy of both BP-low and ZO-high, we introduce a timestep-aware probabilistic function that dynamically selects the appropriate optimization strategy during training. This function mitigates overfitting from BP-low at high timesteps, where structural information is critical, while ensuring that ZO-high is applied more effectively as training progresses. Experimental results across both U-Net- and DiT-based diffusion models demonstrate that Mix-Opt achieves competitive personalization performance while significantly reducing memory consumption. Mix-Opt enables scalable, high-quality personalization without modifying the inference process or increasing inference latency.

## Fast and Strong Image Protection against Diffusion Personalization

DevFeed: [Fast and Strong Image Protection against Diffusion Personalization](<https://devfeed.tech/articles/fast-and-strong-image-protection-against-diffusion-personalization-141490.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22494>)

Author: Jiwon Park, Jihye Kang, Namhyuk Ahn

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [cost](<https://devfeed.tech/tags/cost.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [fast](<https://devfeed.tech/tags/fast.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [image](<https://devfeed.tech/tags/image.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [personalization](<https://devfeed.tech/tags/personalization.md>), [projection](<https://devfeed.tech/tags/projection.md>), [protection](<https://devfeed.tech/tags/protection.md>)

### AI overview

The protective diffusion model (PDM) generates image-protection perturbations through denoising, avoiding the from-scratch optimization required for each image. It is trained to distill an iterative teacher, then post-trained using target residual projection (TRP) loss as a reward. The abstract reports that PDM outperforms nine baselines on faces and artworks and nearly matches its teacher at one-tenth the cost. These findings are limited to the abstract excerpt.

### Source excerpt

arXiv:2611.22494v1 Announce Type: cross Abstract: Personalized text-to-image diffusion models can reproduce a person from a handful of photographs. Image protection perturbs each image before publication so that fine-tuning on it fails, but it optimizes every perturbation from scratch, so its cost grows with the number of images. We present the protective diffusion model (PDM), which is trained once and samples the perturbation of a new image over many denoising steps. PDM distills an iterative teacher and is then post-trained with the target residual projection (TRP) loss as a reward, a stronger objective than existing ones. PDM outperforms nine baselines on faces and artworks and nearly matches its teacher at a tenth of the cost.

## What Does Weight-Space Programming Add to Diffusion Editing? Program Construction versus Score-Guided Refinement

DevFeed: [What Does Weight-Space Programming Add to Diffusion Editing? Program Construction versus Score-Guided Refinement](<https://devfeed.tech/articles/what-does-weight-space-programming-add-to-diffusion-editing-program-construction-versus-score-guided-refinement-141752.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22228>)

Author: Zitian Gao, Jingyi Xu, Ziqi Han, Chengkai Lei, Ye Liu, Liangliang Shi

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [program](<https://devfeed.tech/tags/program.md>), [score](<https://devfeed.tech/tags/score.md>), [weight-space-programming](<https://devfeed.tech/tags/weight-space-programming.md>)

### AI overview

On 24 TIMED test edits for Stable Diffusion 1.4, the preregistered rewrite pipeline showed no advantage over a strong baseline. Exploratory results suggest that one-shot Astra weight-space programs can improve fresh-scene VLM joint success by 10.0 points over a shared-panel hand-written baseline, with no detectable change in CLIP relative match. The programs organise adjustable update directions around protected text-state subspaces. Probing and rewriting them increased CLIP by 6.2 points, while joint success fell by 9.7 points. Five additional evaluators on a fixed 144-image sample showed point estimates in the same directions under whole-image target matching, but subject-local change checks were less decisive. The abstract therefore treats program construction and score-guided revision as distinct stages that need separate evaluation.

### Source excerpt

arXiv:2611.22228v1 Announce Type: new Abstract: Weight-space programming (WSP) lets a language model construct executable weight updates for another network. We study diffusion concept editing through an interface in which programs define training contexts, preservation constraints and update directions, while a shared numerical fitter selects their coefficients. We compare generated programs with published editors, a shared-panel hand-written baseline and budget-matched search over conventional edits on 24 TIMED test edits for Stable Diffusion 1.4. The preregistered rewrite pipeline shows no advantage over the strong baseline. In an exploratory comparison, one-shot Astra programs improve fresh-scene VLM joint success, which requires subject presence together with whole-image target matching, by +10.0 points over the shared-panel baseline, while CLIP relative match shows no detectable change. These programs organise adjustable directions around protected text-state subspaces. Probing and rewriting the Astra programs then raises CLIP by +6.2 points while joint success changes by -9.7. On a fixed 144-image sample, five additional evaluators give point estimates in the same directions under whole-image target matching; checks of subject-local change give less decisive differences. Together, these results identify useful behaviour in complete generated programs and show that score-guided revision need not preserve it: program construction and revision should be evaluated as distinct stages of a WSP pipeline.

## Spectral Steering: Local Inverse Geometry for Frozen Diffusion Generators

DevFeed: [Spectral Steering: Local Inverse Geometry for Frozen Diffusion Generators](<https://devfeed.tech/articles/spectral-steering-local-inverse-geometry-for-frozen-diffusion-generators-137468.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19572>)

Author: Angelo Moroncelli, Alessandro Leanza, Loris Roveda

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusers](<https://devfeed.tech/topics/diffusers.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [control](<https://devfeed.tech/tags/control.md>), [cosine](<https://devfeed.tech/tags/cosine.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generator](<https://devfeed.tech/tags/generator.md>), [generators](<https://devfeed.tech/tags/generators.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [images](<https://devfeed.tech/tags/images.md>), [local](<https://devfeed.tech/tags/local.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [motion](<https://devfeed.tech/tags/motion.md>), [planning](<https://devfeed.tech/tags/planning.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [scalar](<https://devfeed.tech/tags/scalar.md>), [steering](<https://devfeed.tech/tags/steering.md>)

### AI overview

Spectral steering aims to make structured edits to frozen diffusion generators while preserving other output properties. It predicts directional gains from the interaction between the request and the objective-specific Jacobian of the remaining sampler, then applies damped inverse correction with fixed-time relinearization for finite edits. In 8,923 instances from a retrospectively identified local regime, predicted and realized gains correlated at Spearman 0.79, compared with -0.11 for condition number, suggesting conditioning alone did not predict gains. The study covers images, audio, robotics, and control and planning. In 50 paired inpainting cases, four corrections improved unobserved-pixel PSNR by 3.25 dB over Adam at equal total derivative traversals. Scalar-null and matched-movement controls test the proposed mechanism.

### Source excerpt

arXiv:2611.19572v1 Announce Type: cross Abstract: A frozen diffusion generator may need to change several output properties while preserving others. We study when guidance faithfully realizes such a structured request and whether local inverse correction improves its direction. Our account links this benefit to the interaction between the request and the objective-specific remaining-sampler Jacobian: conditioning alone is insufficient. Spectral steering combines a request-specific cosine-gain predictor with damped inverse correction and fixed-time relinearization for finite edits. Scalar nulls and matched-movement controls test the predicted mechanism. Across 8,923 instances in a retrospectively identified local regime, predicted and realized gains correlate at Spearman 0.79, versus -0.11 for condition number. Experiments span images, audio, robotics, and control and planning, including complex character motion generation. On 50 paired inpainting cases, four corrections improve unobserved-pixel PSNR over Adam by 3.25 dB at equal total derivative traversals. Together, the analysis and controls connect request-dependent geometry to directional gains and establish where these gains translate into downstream performance.

## RAGE: Residual-Adaptive Gating for Flow-Based Image Editing

DevFeed: [RAGE: Residual-Adaptive Gating for Flow-Based Image Editing](<https://devfeed.tech/articles/rage-residual-adaptive-gating-for-flow-based-image-editing-139135.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21177>)

Author: Ruofeng Mei, Xiangyu Xu

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [image editing](<https://devfeed.tech/topics/image-editing.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [flux](<https://devfeed.tech/topics/flux.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [background](<https://devfeed.tech/tags/background.md>), [bench](<https://devfeed.tech/tags/bench.md>), [component](<https://devfeed.tech/tags/component.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [edits](<https://devfeed.tech/tags/edits.md>), [flow](<https://devfeed.tech/tags/flow.md>), [flux](<https://devfeed.tech/tags/flux.md>), [image](<https://devfeed.tech/tags/image.md>), [image-editing](<https://devfeed.tech/tags/image-editing.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [local](<https://devfeed.tech/tags/local.md>), [mask](<https://devfeed.tech/tags/mask.md>), [optimal-control](<https://devfeed.tech/tags/optimal-control.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [source](<https://devfeed.tech/tags/source.md>), [stable-diffusion](<https://devfeed.tech/tags/stable-diffusion.md>), [velocity](<https://devfeed.tech/tags/velocity.md>), [yields](<https://devfeed.tech/tags/yields.md>)

### AI overview

RAGE aims to reduce unintended changes to regions that should remain intact during inversion-free, flow-based image editing. It treats source fidelity as a stochastic optimal control problem and derives residual-adaptive gating: each latent component is pulled toward the source with a gain that decays exponentially as its local editing residual grows, suppressing small deviations while allowing larger edits through. Applied on top of FlowEdit-based editors, it adds one lightweight gated interpolation per sampling step and requires no masks, attention maps, retraining, or architecture-specific changes. On PIE-Bench, the authors report the best overall rank among 17 methods and substantially improved background preservation on Stable Diffusion 3 and FLUX, with competitive editing quality.

### Source excerpt

arXiv:2611.21177v1 Announce Type: cross Abstract: Recent inversion-free, flow-based image editing methods evolve the source image toward the target via a differential velocity field. We observe that this velocity field does not explicitly account for which regions should be preserved, leading to unintended drift in non-editing regions. To address this, we formulate source fidelity control as a stochastic optimal control problem in which a selective fidelity term on the editing residual is enforced along the editing trajectory. Solving this problem and approximating its solution yields the Residual Gating Principle: each latent component is pulled back toward the source with a gain that decays exponentially with the local residual magnitude, so that small deviations are suppressed while large, semantically meaningful edits pass through unimpeded. The proposed method, RAGE, requires only one lightweight gated interpolation per sampling step, involves no mask estimation, attention maps, retraining, or architecture-specific modifications, and is applied on top of FlowEdit-based editors. On PIE-Bench, RAGE achieves the best overall rank among 17 methods and substantially improves background preservation on both Stable Diffusion 3 and FLUX backbones while maintaining competitive editing quality.

## Beyond Semantic Prompts: Unified Watermarking and Access Control for Text-to-Image Diffusion Models

DevFeed: [Beyond Semantic Prompts: Unified Watermarking and Access Control for Text-to-Image Diffusion Models](<https://devfeed.tech/articles/beyond-semantic-prompts-unified-watermarking-and-access-control-for-text-to-image-diffusion-models-139302.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20470>)

Author: Fangming Shi, Li Li, Guorui Feng, Xinpeng Zhang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [text-to-image](<https://devfeed.tech/topics/text-to-image.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [Access Control](<https://devfeed.tech/topics/access-control.md>), [watermarking](<https://devfeed.tech/topics/watermarking.md>)

Tags: [access-control](<https://devfeed.tech/tags/access-control.md>), [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [customizable](<https://devfeed.tech/tags/customizable.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [generation](<https://devfeed.tech/tags/generation.md>), [image-processing](<https://devfeed.tech/tags/image-processing.md>), [images](<https://devfeed.tech/tags/images.md>), [models](<https://devfeed.tech/tags/models.md>), [prompt-structure](<https://devfeed.tech/tags/prompt-structure.md>), [prompts](<https://devfeed.tech/tags/prompts.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [quality](<https://devfeed.tech/tags/quality.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [services](<https://devfeed.tech/tags/services.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>), [watermarking](<https://devfeed.tech/tags/watermarking.md>)

### AI overview

The proposed framework combines multi-bit watermarking and tiered access control in a single text prompt. It encodes watermark messages as designated words with dedicated learnable tokens, then jointly adapts the diffusion U-Net and token embeddings using a frozen reference model and pretrained watermark codec. A prompt-structure recognition network controls the VAE decoder to produce clear watermarked images, blurred previews, or black outputs according to access state. The abstract reports reliable watermark recovery, robustness to tested image-processing operations and model modifications, and preserved decoding fidelity and watermark recovery under full access. It does not provide quantitative results in the supplied excerpt.

### Source excerpt

arXiv:2611.20470v1 Announce Type: new Abstract: Text-to-image diffusion services require both traceability of generated content and differentiated access to generation capabilities. However, existing watermarking methods commonly expose watermark control through mechanisms beyond the text-prompt interface, while credential-based access control does not inherently enforce distinct output quality levels. We propose a unified framework that enables customizable multi-bit watermark embedding and tiered access control through a single text prompt. A unified prompt represents watermark messages using designated words and associates protocol structures with different access states. The framework adopts a two-stage training strategy. First, watermark payload partitioning represents messages as designated watermark words, while vocabulary extension assigns each protocol word a dedicated token with a learnable input embedding. Guided by a frozen reference model and a pretrained watermark codec, joint adaptation of the U-Net and protocol-token embeddings transfers message-dependent watermark patterns into the generation process. Second, a prompt-structure recognition network controls feature modulation within the variational autoencoder decoder, which is trained to produce clear watermarked images, blurred previews, or black outputs according to the access state. Experiments demonstrate reliable watermark recovery, robustness to common image-processing operations and evaluated model modifications, and consistent performance across diverse generation settings. The controlled decoder also achieves distinct output behaviors under restricted access while closely preserving reference decoding fidelity and watermark recoverability under full access. Our open-source code and model weights will be public after acceptance.

## Certify, Then Select: Viability-Aware Runtime Steering of Frozen Diffusion Policies

DevFeed: [Certify, Then Select: Viability-Aware Runtime Steering of Frozen Diffusion Policies](<https://devfeed.tech/articles/certify-then-select-viability-aware-runtime-steering-of-frozen-diffusion-policies-138826.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19047>)

Author: Angelo Moroncelli, Loris Roveda

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [a-safety](<https://devfeed.tech/tags/a-safety.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [actions](<https://devfeed.tech/tags/actions.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [availability](<https://devfeed.tech/tags/availability.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [cast](<https://devfeed.tech/tags/cast.md>), [certification](<https://devfeed.tech/tags/certification.md>), [condition](<https://devfeed.tech/tags/condition.md>), [constraint](<https://devfeed.tech/tags/constraint.md>), [control](<https://devfeed.tech/tags/control.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [edits](<https://devfeed.tech/tags/edits.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [policies](<https://devfeed.tech/tags/policies.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [select](<https://devfeed.tech/tags/select.md>), [steering](<https://devfeed.tech/tags/steering.md>)

### AI overview

CAST steers frozen diffusion policies by sampling a few action chunks, certifying each against a barrier condition, then choosing the certified chunk ranked highest by a learned value ensemble. The authors say certification makes a step safe regardless of the critic, while selection stays within log N nats of the policy conditioned on feasibility when a certified sample exists; a safety-filter projection can be singular to that feasible policy. Because safe actions now may lead to no safe actions later, CAST-V adds policy-conditioned viability, estimated with a model under a finite-sample guarantee or learned from controller logs (CAST-VL). Across robomimic manipulation, a planar quadrotor, a MuJoCo humanoid and a unicycle, CAST reportedly improves joint task-and-constraint success over an output filter in nearly every setting; viability removes most dead ends missed by one-step certification.

### Source excerpt

arXiv:2611.19047v1 Announce Type: cross Abstract: Diffusion policies, which generate short action sequences by denoising random noise, are a leading way to learn robot skills from demonstrations. Once deployed, they meet constraints the demonstrations never contained, and retraining for each is impractical. A safety filter edits the policy's action to satisfy them, but the edit ignores the task, and a large edit can leave the policy where it cannot continue. We propose CAST (Certified Action Selection at Test time): draw a few action chunks from the frozen policy, certify each with a barrier condition, and execute the certified chunk that a learned value ensemble ranks highest. We prove that a certified step is safe whatever the critic does; that, whenever a certified sample exists, the executed chunk stays within log N nats of the policy conditioned on feasibility, while a safety-filter projection can be singular to it; and that an exact closed-loop identity splits the gain of ranking into availability, spread and the critic's target. Certifying the next chunk is not enough: a generative controller can have safe actions available now and none later. We characterise the safe control authority of a frozen policy under finite sampling, and CAST-V preserves it through a policy-conditioned viability, estimated with a model under a finite-sample guarantee or learned from the controller's own logs (CAST-VL). On robomimic manipulation, a planar quadrotor, a MuJoCo humanoid and a unicycle, with frozen policies, CAST raises joint task-and-constraint success above the output filter in nearly every setting; certifying before ranking is decisive where a filter's edit is large, and viability removes most dead ends that one-step certification cannot see. Project page: https://clmoro.github.io/cast.

## TileCache: Serving Multi-Round Diffusion Image Editing with Compressed Activation Caches

DevFeed: [TileCache: Serving Multi-Round Diffusion Image Editing with Compressed Activation Caches](<https://devfeed.tech/articles/tilecache-serving-multi-round-diffusion-image-editing-with-compressed-activation-caches-136989.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20246>)

Author: Yuyang Tian, Desen Sun, Shuncheng Jie, Lishan Yang, Sihang Liu

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [image editing](<https://devfeed.tech/topics/image-editing.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [image](<https://devfeed.tech/tags/image.md>), [image-editing](<https://devfeed.tech/tags/image-editing.md>), [serving](<https://devfeed.tech/tags/serving.md>)

### AI overview

TileCache combines compressed activation caches with a scheduler for multi-round diffusion image editing. It addresses the memory burden of caching activations--compressing only after editing would require buffering over 100 GB per request--and supports selective reuse for masked edits. Its scheduler balances faster cached edits against initial generations, which naive policies can delay. Against the state-of-the-art image editing baseline, TileCache reportedly sustains over twice the request arrival rate at 90% SLO attainment, completes edits 2.4-4.5x faster and initial generations 1.3-2.2x faster, while preserving image quality.

### Source excerpt

arXiv:2611.20246v1 Announce Type: new Abstract: Multi-round diffusion image editing can reduce computation by reusing intermediate activations, but retaining these activations across editing rounds introduces substantial memory and bandwidth overheads. Compression can reduce these overheads, but efficient cache construction is challenging: compressing only after editing requires buffering over 100 GB of activations per request. Moreover, the compressed cache must support selective reuse, as future masked edits require only a subset of the cached activations. It also exposes a scheduling challenge, as cached edits are faster than initial generations, causing naive policies to severely delay either initial generations or editing requests. In this paper, we aim to address both challenges through compression-informed serving. We develop TileCache, a serving system for multi-round diffusion editing. It compresses reusable activations and incorporates a scheduler that balances both edits and initial generation requests. Compared with the state-of-the-art image editing baseline, TileCache sustains over 2x the request arrival rate at 90% SLO attainment. It completes edits 2.4-4.5x and initial generations 1.3-2.2x faster than the baseline, while preserving image quality.

## Finite speed Ornstein-Uhlenbeck-Kac processes and their applications in generative modeling

DevFeed: [Finite speed Ornstein-Uhlenbeck-Kac processes and their applications in generative modeling](<https://devfeed.tech/articles/finite-speed-ornstein-uhlenbeck-kac-processes-and-their-applications-in-generative-modeling-135810.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17625>)

Author: Richard Duong, Janis Auffenberg, Gabriele Steidl

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [3D](<https://devfeed.tech/topics/3d.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [polynomials](<https://devfeed.tech/topics/polynomials.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [applications](<https://devfeed.tech/tags/applications.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [cifar-10](<https://devfeed.tech/tags/cifar-10.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [fields](<https://devfeed.tech/tags/fields.md>), [flow](<https://devfeed.tech/tags/flow.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generative](<https://devfeed.tech/tags/generative.md>), [image](<https://devfeed.tech/tags/image.md>), [image-generation](<https://devfeed.tech/tags/image-generation.md>), [law](<https://devfeed.tech/tags/law.md>), [math-ap](<https://devfeed.tech/tags/math-ap.md>), [math-pr](<https://devfeed.tech/tags/math-pr.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [processes](<https://devfeed.tech/tags/processes.md>), [speed](<https://devfeed.tech/tags/speed.md>)

### AI overview

The Ornstein-Uhlenbeck-Kac (OU-K) process replaces Brownian driving noise with a finite-speed Kac process. The authors relate it to a rescaled, time-inhomogeneous Kac process whose law solves the Euler-Poisson-Darboux wave equation; for specified switching rates, they derive an explicit solution using Legendre polynomials between the wave fronts. Using piecewise-deterministic Markov processes, they obtain conditional velocity fields bounded by the Kac speed and apply them in a generative flow model, removing the need for the usual diffusion-model time truncations. They report sharper generation on singular data sets than diffusion and flow-matching models, and competitive CIFAR-10 image-generation results.

### Source excerpt

arXiv:2611.17625v1 Announce Type: cross Abstract: We study the Ornstein-Uhlenbeck-Kac (OU-K) process, where instead of a Brownian motion, a finite-speed Kac process defines the driving process. The OU-K process can be analyzed through a rescaled, time-inhomogeneous Kac process, characterized by a switching rate from the Kac part, and a convergence rate from the Ornstein-Uhlenbeck (OU) part. The law of this inhomogeneous Kac process describes the solution of a wave equation, the Euler-Poisson-Darboux (EPD) equation. For Kac switching rates being multiples of the OU convergence rate, we can explicitly derive this EPD solution via the sum of (the derivative of) two Legendre polynomials, restricted between two Dirac points representing the wave fronts. Using the framework of piecewise-deterministic Markov processes (PDMP), we describe the conditional velocity fields of the OU-K process via our polynomial-law-formula. This allows us to apply the OU-K process in a generative flow model. Since the OU-K velocity fields prove to be bounded by the finite Kac speed, the usual time truncations of diffusion models become unnecessary. We demonstrate advantages over diffusion and flow matching models in terms of generation sharpness on singular data sets, while achieving competitive results on CIFAR-10 image generation.

## JaRS: Contrast-Free Localization of Memorized Regions in Diffusion Models via Jacobian Row Sums

DevFeed: [JaRS: Contrast-Free Localization of Memorized Regions in Diffusion Models via Jacobian Row Sums](<https://devfeed.tech/articles/jars-contrast-free-localization-of-memorized-regions-in-diffusion-models-via-jacobian-row-sums-135858.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18024>)

Author: Saisuchith Mahajan, Aditey Nandan, Rohit Singh, Nishanth Shetty, Chandra Sekhar Seelamantula

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [localization](<https://devfeed.tech/tags/localization.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [regions](<https://devfeed.tech/tags/regions.md>)

### AI overview

JaRS localizes memorized image regions in diffusion models without the reference map required by contrast-based methods. Those methods use denoiser-Jacobian diagonal entries, but small values can indicate either memorization or averaging across neighboring pixels in smooth or predictable regions; resolving that ambiguity requires a second evaluation with a null prompt or earlier checkpoint. Jacobian row sums distinguish memorization from local averaging without that reference, including for unconditional models. JaRS+ pairs the statistic with an image-level detection score computed at an early denoising step. The abstract reports that JaRS+ outperforms contrast-based localization methods on Stable Diffusion v1.4 and v2.1, Realistic Vision v5.1, and an unconditional diffusion Transformer.

### Source excerpt

arXiv:2611.18024v1 Announce Type: cross Abstract: Diffusion models sometimes memorize and reproduce regions of their training images, raising privacy and copyright concerns. We aim to localize these regions. Existing contrast-based methods use the diagonal of the denoiser Jacobian, which reflects the posterior variance of each pixel. However, a small diagonal entry is ambiguous: it signals either memorization or averaging over many neighboring pixels in smooth or predictable regions. To resolve this ambiguity, these methods subtract a reference map, which requires a second evaluation with a null prompt or an earlier checkpoint. Unconditional models have no null prompt, and earlier checkpoints are unavailable in general. We show that Jacobian row sums (JaRS), unlike the diagonal, separate memorization from local averaging without a reference map. Pairing JaRS with an image-level detection score that uses the same statistic at an early denoising step yields JaRS+. Across Stable Diffusion v1.4 and v2.1, Realistic Vision v5.1, and an unconditional diffusion Transformer (DiT), JaRS+ outperforms contrast-based localization methods.

## Rethinking Floorplan Generation with Space Syntax: From Design Theory to Model Improvement

DevFeed: [Rethinking Floorplan Generation with Space Syntax: From Design Theory to Model Improvement](<https://devfeed.tech/articles/rethinking-floorplan-generation-with-space-syntax-from-design-theory-to-model-improvement-132760.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.22507>)

Author: Zhuoyang Jiang, Dongqing Zhang, Qingxiang Liu, Yuxuan Liang

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [ai](<https://devfeed.tech/tags/ai.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [applications](<https://devfeed.tech/tags/applications.md>), [applying](<https://devfeed.tech/tags/applying.md>), [approach](<https://devfeed.tech/tags/approach.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [call](<https://devfeed.tech/tags/call.md>), [capture](<https://devfeed.tech/tags/capture.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [design](<https://devfeed.tech/tags/design.md>), [generation](<https://devfeed.tech/tags/generation.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [model](<https://devfeed.tech/tags/model.md>), [space](<https://devfeed.tech/tags/space.md>), [syntax](<https://devfeed.tech/tags/syntax.md>), [theory](<https://devfeed.tech/tags/theory.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

The paper uses space syntax to make functional organization in generated floorplans measurable. Its Space Syntax Integration Operator (SSIO) recovers convex-space access graphs across heterogeneous formats as a shared design representation. Applying SSIO to residential corpora from 15 countries, the authors find that public spaces have the highest integration, with distributions varying by corpus. They use these patterns to build RPLAN-SSI, which evaluates population-level functional-profile alignment (IPD) and the distribution of public-space advantage within plans (PIAD). Against this reference, generated floorplans show a smaller integration advantage for public spaces. Space Syntax-Guided Post-Training (SSPT) uses SSIO-derived reinforcement-learning feedback to reduce IPD and PIAD across five heterogeneous diffusion backbones while preserving their native mechanisms.

### Source excerpt

arXiv:2602.22507v4 Announce Type: replace Abstract: Architectural design increasingly adopts generative AI (GenAI) for floorplan generation, an early-stage design task exploring how spaces should be organized for use. Yet fidelity objectives inherited from native visual and geometric generators do not automatically capture this design purpose. This organization remains largely implicit in generated representations, limiting direct evaluation and feedback. Drawing on space syntax, an established architectural design theory, the Space Syntax Integration Operator (SSIO) recovers convex-space access graphs across heterogeneous formats. This theory-grounded shared intermediate representation, which we call a design container, supports evaluation and model understanding of spatial design. Applying SSIO to four residential corpora spanning 15 countries reveals public spaces' integration levels are highest, with corpus-dependent distributions. These patterns underpin the RPLAN-SSI benchmark, an empirical design reference system assessing population-level functional-profile alignment (IPD) and distributional alignment of within-plan public-space advantage (PIAD). Comparison with this residential reference reveals a design gap: public spaces in generated floorplans have a smaller integration advantage over other spaces. To address this gap, Space Syntax-Guided Post-Training (SSPT) provides a plug-and-play design-learning interface: SSIO-derived RL feedback reduces IPD and PIAD across five heterogeneous diffusion backbones while preserving native mechanisms. We further discuss how this approach may inform GenAI applications in other design domains.

## Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL

DevFeed: [Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL](<https://devfeed.tech/articles/diff-instruct-with-diffused-reward-towards-principled-one-step-generator-rl-128113.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.24001>)

Author: Junyi Wu, Weijian Luo, Haoyang Zheng, Ruizhe Zhang, Guang Lin

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [rlhf](<https://devfeed.tech/topics/rlhf.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [drp](<https://devfeed.tech/tags/drp.md>), [drs](<https://devfeed.tech/tags/drs.md>), [image](<https://devfeed.tech/tags/image.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [rlhf](<https://devfeed.tech/tags/rlhf.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>)

### AI overview

DIDR addresses a mismatch in prior one-step generator RL, where terminal image-reward optimization combined with noisy-space distribution matching can exploit stochasticity and reduce image fidelity. The proposed data-free, trajectory-level framework uses Integral KL minimization to propagate the reward-tilted clean-image distribution across diffusion noise levels. Its objective has the same minimizer as clean-image RLHF and induces a Diffused Reward Score correction to the reference score. For practical estimation, Diffused Reward Proxy approximates that correction using differentiable short-step denoising. The authors report Pareto dominance over existing one-step SDXL baselines and, on the 6B Z-Image backbone, higher preference alignment than its 50-step teacher while generating in one step.

### Source excerpt

arXiv:2605.24001v3 Announce Type: replace-cross Abstract: Recent advances in one-step text-to-image generation have enabled real-time synthesis with remarkable efficiency and quality. Previous reinforcement learning methods for one-step generators combine image-space reward optimization with diffusion noisy-space distribution matching. This paradigm brings challenges due to a mismatch between terminal reward optimization and the underlying generative dynamics. As a result, optimization tends to exploit stochastic degrees of freedom, often improving reward at the expense of image fidelity. To address this issue, we propose Diff-Instruct with Diffused Reward (DIDR), a data-free trajectory-level alignment framework derived from Integral KL minimization. DIDR propagates the RLHF-optimal reward-tilted clean-image distribution across all noise levels along the diffusion trajectory. We show that this objective admits the same minimizer as clean-image RLHF, while naturally inducing the Diffused Reward Score (DRS), which acts as a reward-driven correction to the reference score function. To make this practical, we further introduce the Diffused Reward Proxy (DRP), an efficient estimator of DRS based on differentiable short-step denoising. Extensive experiments demonstrate that DIDR consistently Pareto-dominates existing one-step SDXL baselines. Moreover, when transferred to a 6B DiT backbone (Z-Image), DIDR surpasses its 50-step teacher in preference alignment while requiring only a single generation step. Code: https://github.com/Junyi-W/DIDR

## Trajectory-Regularized Distribution Distillation for One-Step Generation

DevFeed: [Trajectory-Regularized Distribution Distillation for One-Step Generation](<https://devfeed.tech/articles/trajectory-regularized-distribution-distillation-for-one-step-generation-128223.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12978>)

Author: An Zhao, Yixiang Zhou, Zejian Li, Yi Guo, Pei Chen, Lingyun Sun

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [3](<https://devfeed.tech/tags/3.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [but](<https://devfeed.tech/tags/but.md>), [combines](<https://devfeed.tech/tags/combines.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [distilling](<https://devfeed.tech/tags/distilling.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [estimator](<https://devfeed.tech/tags/estimator.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [field](<https://devfeed.tech/tags/field.md>), [flow](<https://devfeed.tech/tags/flow.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generation](<https://devfeed.tech/tags/generation.md>), [one](<https://devfeed.tech/tags/one.md>)

### AI overview

Trajectory-Regularized Distribution Distillation (TRD) combines trajectory supervision from a teacher's probability-flow ODE with direct distribution-level optimization. Its momentum-mixture policy is predicted in one student forward pass and serves as the generator, an estimator of the student's marginal velocity, and a time-dependent velocity field for trajectory supervision--avoiding the separate trainable score-tracking network typically used for distribution matching. On SANA-1.6B and Stable Diffusion 3.5, the authors report better generation quality and training efficiency than representative consistency and distribution-matching baselines for one-step and few-step generation, with a trade-off in sample diversity.

### Source excerpt

arXiv:2611.12978v1 Announce Type: cross Abstract: Distilling flow matching models for one-step generation generally follows two paradigms. Trajectory alignment supervises the student with the teacher's probability flow ODE, but its performance degrades under very small sampling budgets. Distribution matching directly optimizes the generated distribution, but estimating its reverse Kullback-Leibler gradient typically requires a separate network to track the evolving student score. We propose trajectory-regularized distribution distillation (TRD), which combines distribution-level and trajectory-level supervision through a momentum-mixture policy predicted in a single student forward pass. This policy serves simultaneously as a one-step generator, an estimator of the student's marginal velocity that removes the need for a second trainable network in distribution matching, and a closed-form time-dependent velocity field supervised along the teacher ODE trajectory. Experiments on SANA-1.6B and Stable Diffusion 3.5 show that TRD outperforms representative consistency and distribution matching baselines in generation quality and training efficiency for both one-step and few-step generation, with a trade-off in sample diversity.

## Adaptive Reparametrized Time for Score-Based Diffusion Sampling

DevFeed: [Adaptive Reparametrized Time for Score-Based Diffusion Sampling](<https://devfeed.tech/articles/adaptive-reparametrized-time-for-score-based-diffusion-sampling-128127.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.02137>)

Author: Yilie Huang, Wenpin Tang, Xun Yu Zhou

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [actor](<https://devfeed.tech/tags/actor.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [error](<https://devfeed.tech/tags/error.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [generation](<https://devfeed.tech/tags/generation.md>), [grid](<https://devfeed.tech/tags/grid.md>), [image](<https://devfeed.tech/tags/image.md>), [learning](<https://devfeed.tech/tags/learning.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [quality](<https://devfeed.tech/tags/quality.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [score](<https://devfeed.tech/tags/score.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

ART learns how quickly to advance the sampling clock so a uniform grid in learned time creates adaptive timesteps for score-based diffusion sampling. It optimizes timestep allocation using a leading-order Euler error surrogate. ART-RL recasts this control problem with Gaussian policies; the paper states that the optimal policy mean recovers the optimal ART time-warping rate and derives actor-critic updates from trajectory-based moment identities. The reported experiments span controlled low-dimensional settings and image generation: changing only the timestep grid reportedly improves sample quality over baseline schedules at matched budgets, and learned schedules transfer without retraining across budgets, datasets, solvers, pipelines, and representation spaces.

### Source excerpt

arXiv:2607.02137v4 Announce Type: replace-cross Abstract: We study timestep allocation for score-based diffusion sampling, where a learned reverse-time dynamics is discretized on a finite grid. Uniform and hand-crafted schedules are standard choices, but they rely on fixed prescriptions and can therefore be suboptimal. To address this limitation, we propose Adaptive Reparameterized Time (ART), a continuous-time control formulation that learns a time change by treating the speed of the sampling clock as the control, so that a uniform grid on the learned clock induces adaptive timesteps in the original diffusion time. Based on a leading-order Euler error surrogate, ART provides a principled objective for allocating timesteps along the sampling trajectory. To solve this deterministic control problem, we introduce ART-RL, an auxiliary randomized formulation with Gaussian policies that turns schedule learning into a continuous-time reinforcement learning problem. We prove that the randomized ART-RL formulation is equivalent to ART at the optimizer level, in the sense that its optimal Gaussian policy recovers the optimal ART time-warping rate through its mean. We further establish policy evaluation and policy improvement characterizations and derive trajectory-based moment identities that yield implementable actor--critic updates for learning the schedule. Across experiments ranging from controlled low-dimensional settings to image generation, ART-RL can be plugged into existing diffusion samplers by changing only the timestep grid, consistently improving sample quality over strong baseline schedules at matched budgets while leaving the rest of the sampling pipeline unchanged. The learned schedules also exhibit broad generalization, transferring without retraining across sampling budgets, datasets, solvers, pipelines, and representation spaces.

## Stylization as Geodesic Navigation on Diffusion Manifolds

DevFeed: [Stylization as Geodesic Navigation on Diffusion Manifolds](<https://devfeed.tech/articles/stylization-as-geodesic-navigation-on-diffusion-manifolds-128799.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14097>)

Author: Yan Li, Zehao Li, Xinlong Sun, Fuchen Long, Xi Chen, Wenhan Luo, Yike Guo

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [Image](<https://devfeed.tech/topics/image.md>), [Color Grading](<https://devfeed.tech/topics/color-grading.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [approximation](<https://devfeed.tech/tags/approximation.md>), [attention](<https://devfeed.tech/tags/attention.md>), [content](<https://devfeed.tech/tags/content.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [control](<https://devfeed.tech/tags/control.md>), [controls](<https://devfeed.tech/tags/controls.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [curved](<https://devfeed.tech/tags/curved.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [energy](<https://devfeed.tech/tags/energy.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [fine](<https://devfeed.tech/tags/fine.md>), [flat](<https://devfeed.tech/tags/flat.md>), [framework](<https://devfeed.tech/tags/framework.md>), [free](<https://devfeed.tech/tags/free.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [identity](<https://devfeed.tech/tags/identity.md>), [image](<https://devfeed.tech/tags/image.md>), [iteration](<https://devfeed.tech/tags/iteration.md>), [navigation](<https://devfeed.tech/tags/navigation.md>)

### AI overview

Stylus performs training-free facial style transfer by navigating a pretrained diffusion model's curved latent manifold to balance artistic style with identity preservation. It refines an interpolated content-style point by reducing a ratio-weighted discrete geodesic energy under the pullback metric, using a rank-one power-iteration approximation of the Jacobian. Two controls support the process: latent interpolation for smooth semantic fusion and self-attention query optimization for spatially adaptive style injection. The abstract reports continuous control over stylization strength and the strongest identity and content preservation at competitive stylization strength in experiments; it provides no quantitative results.

### Source excerpt

arXiv:2611.14097v1 Announce Type: cross Abstract: Facial stylization asks for a delicate transformation: the image should look unmistakably artistic, yet the person should remain unmistakably the same. Existing training-free and instruction-driven methods often fail to expose such control, either drifting away from identity or producing only weak stylization. We introduce Stylus, a training-free framework that treats facial style transfer as navigation on the curved latent manifold of a pretrained diffusion model. Rather than interpolating in flat Euclidean space, \method refines the interpolated content--style point by lowering a ratio-weighted discrete geodesic energy under the pullback metric, made tractable by a rank-one power-iteration approximation of the Jacobian. This principle drives two complementary controls: content--style latent interpolation for smooth semantic fusion, and self-attention query optimization for spatially adaptive style injection. The resulting dual-control mechanism provides continuous, fine-grained control over stylization strength while preserving identity-critical facial structure, all without additional training. Experiments show that Stylus achieves the strongest identity and content preservation at competitive stylization strength.

## Location Matters: Sparse Latent-Variance Attacks for Face Privacy against Diffusion Personalization

DevFeed: [Location Matters: Sparse Latent-Variance Attacks for Face Privacy against Diffusion Personalization](<https://devfeed.tech/articles/location-matters-sparse-latent-variance-attacks-for-face-privacy-against-diffusion-personalization-128922.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14254>)

Author: Shuting Yang, Shuxin Li, Linyan Dai, Xinwei Zhang, Dong Huang, Renyang Liu, See-Kiong Ng

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Security & Privacy](<https://devfeed.tech/topics/security-privacy.md>), [VAE](<https://devfeed.tech/topics/vae.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [dispersion](<https://devfeed.tech/tags/dispersion.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [identity-protection](<https://devfeed.tech/tags/identity-protection.md>), [image](<https://devfeed.tech/tags/image.md>), [masking](<https://devfeed.tech/tags/masking.md>), [models](<https://devfeed.tech/tags/models.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [personalization](<https://devfeed.tech/tags/personalization.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [regions](<https://devfeed.tech/tags/regions.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>), [vae](<https://devfeed.tech/tags/vae.md>), [variance](<https://devfeed.tech/tags/variance.md>)

### AI overview

SEAL aims to protect facial photos from identity learning during diffusion-model personalization by changing only 5% of image locations. It jointly selects locations and perturbation values with a differentiable top-k mask, optimizes variance in the frozen VAE encoder's latent distribution, and uses a saliency-guided penalty to spread changes away from perceptually prominent facial regions. On CelebA-HQ and VGGFace2, the authors report the best protected-image quality among compared methods (over 41 dB PSNR, 0.97 SSIM, and at most 0.07 LPIPS) while reducing identity similarity after personalization to at most 0.15. These are reported experimental results; the supplied text does not specify broader deployment performance.

### Source excerpt

arXiv:2611.14254v1 Announce Type: cross Abstract: Text-to-image diffusion models can learn and reproduce an individual's identity from only a few reference photos, creating risks of unauthorized and misleading content generation. Proactive defenses address this risk by adding adversarial perturbations to facial images before release, thereby disrupting identity acquisition during downstream personalization. However, existing methods typically rely on dense $L_\infty$-bounded perturbations, whose accumulated distortions can become visible around facial structures. Recent methods improve protected-image quality, but often at the cost of reduced protection effectiveness. In this paper, we propose Sparse Encoder Attack on Latent Variance (SEAL), a sparse adversarial protection framework that modifies only 5% of the spatial locations in a facial image. SEAL combines a differentiable top-k masking scheme that jointly optimizes perturbation locations and values with a variance-oriented objective over the frozen VAE encoder's latent distribution to disrupt identity-consistent representations. A saliency-guided dispersion penalty further discourages perturbation concentration in perceptually salient facial regions. Experiments on CelebA-HQ and VGGFace2 show that SEAL achieves the best protected-image quality among the compared methods, exceeding 41 dB PSNR with 0.97 SSIM and at most 0.07 LPIPS, while reducing identity similarity to at most 0.15 after diffusion personalization. These results demonstrate that sparse location optimization enables strong identity protection while preserving high visual quality. Code and data are available at https://github.com/Lvxan/SEAL

[Next page](<https://devfeed.tech/topics/stable-diffusion.md?cursor=WyIyMDI2LTExLTExVDA1OjAwOjAwKzAwOjAwIiwgIjA4MmUzMzIxLTk4NmQtNDQ4YS1iMDQ4LTY0MGY2OTIzNTFlMSJd>)