# diffusion-transformers

Transformer-based diffusion-model architecture that replaces a U-Net backbone with a transformer operating on latent image patches.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## RL Post-Training in Diffusion Transformers Is Activation Steering

DevFeed: [RL Post-Training in Diffusion Transformers Is Activation Steering](<https://devfeed.tech/articles/rl-post-training-in-diffusion-transformers-is-activation-steering-145203.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26627>)

Author: Tuna Han Salih Meral, Kaan Oktay, Alec Helbling, Atakan Kara, Duen Horng Chau, Pinar Yanardag

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [activation-steering](<https://devfeed.tech/tags/activation-steering.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [rl](<https://devfeed.tech/tags/rl.md>), [steering](<https://devfeed.tech/tags/steering.md>)

### AI overview

The study finds that fixed activation shifts can reproduce most reported gains from RL post-training diffusion transformers, without adding prompt-dependent transformations. The authors add steering vectors to outputs of a frozen base model and fit them to match RL-tuned checkpoints' velocity predictions, without using reward in that fitting objective. On SD3.5-M, the vectors use 0.29M parameters versus 18.8M for the RL-tuned LoRA. Despite being identical across prompts, tokens, and sampling seeds, they recover 90.0-96.3% of reward gains for six image models and 89.8% and 111.9% for two video models. In separate online RL experiments, zero-initialized vectors exceed weight-based RL's reported gains on OCR text rendering and PickScore human preference for SD3.5-M. The abstract's conclusion is that a constant shift can steer the pretrained generator toward capabilities it already computes.

### Source excerpt

arXiv:2611.26627v1 Announce Type: cross Abstract: Reinforcement learning (RL) post-training substantially improves the visual quality, text rendering, and compositional accuracy of diffusion transformers. Because capabilities such as rendering requested text or composing multiple objects are inherently prompt-dependent, a natural hypothesis is that improving them requires learning complex, input-dependent transformations. To isolate how much of these gains an input-independent shift explains, we probe each RL-tuned model with a deliberately simple parameterization: a fixed set of steering vectors added to the block outputs of the frozen base model. On SD3.5-M, these vectors hold 65x fewer parameters than the RL-tuned LoRA (0.29M against 18.8M). We fit them to match the velocity predictions of public RL-tuned checkpoints, with no reward in the objective. Although the vectors are identical for every prompt, token, and sampling seed, they recover 90.0-96.3% of the reward gains of six RL-tuned image models, and 89.8% and 111.9% on two RL-tuned video models. Initialized at zero and trained with online RL, the vectors alone exceed the reward gains of weight-based RL on text rendering (OCR) and human preference (PickScore) on SD3.5-M. A constant shift thus suffices for most of the gain of policy-gradient RL post-training: without new input-dependent transformations, it steers the pretrained generator toward what it can already compute.

## Online Temporal Rotations for 2 Bit KV-Cache Quantization in Video Diffusion Transformers

DevFeed: [Online Temporal Rotations for 2 Bit KV-Cache Quantization in Video Diffusion Transformers](<https://devfeed.tech/articles/online-temporal-rotations-for-2-bit-kv-cache-quantization-in-video-diffusion-transformers-145656.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26334>)

Author: Ryan Lucas, Petros Prastakos, Rahul Mazumder

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [cache](<https://devfeed.tech/tags/cache.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [online](<https://devfeed.tech/tags/online.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [rotation](<https://devfeed.tech/tags/rotation.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [triton](<https://devfeed.tech/tags/triton.md>), [video](<https://devfeed.tech/tags/video.md>), [video-generation](<https://devfeed.tech/tags/video-generation.md>)

### AI overview

Temporal Rotation Quantization (TRQ) reduces quantization error in video diffusion KV caches by rotating temporally adjacent cache frames before 2-bit quantization. For each block, it fits an orthogonal transform to flatten per-channel means, using a tree of two-bit planar rotations followed by a fixed Hadamard transform. The rotations are fitted online for each video without gradients or model-weight changes; the authors say a greedy fit is globally optimal for this objective and provide fused Triton inference kernels. At matched compression, TRQ raises VBench subject consistency over QVG from 90.38 to 92.02 on LongCat-Video-13B and from 86.33 to 87.61 on HY-WorldPlay-8B; the abstract also reports gains on other metrics without specifying them.

### Source excerpt

arXiv:2611.26334v1 Announce Type: cross Abstract: Video diffusion transformers reuse key--value (KV) caches across denoising steps, which quickly becomes a bottleneck for long video generation. We find that quantizing temporally distant cache vectors leads to a large range and high quantization error, but that nearby video tokens in the KV-cache share a persistent pattern per channel. These patterns also vary across videos, motivating rotations that are adapted to each video's unique cache. We propose Temporal Rotation Quantization (TRQ), a method that rotates frames in the cache that are adjacent in time before quantization. For each block of temporally adjacent cache vectors, TRQ fits an orthogonal transform that maps the block's per-channel means to a vector that is close to a constant across channels. To keep the quantization format lightweight, we parameterize this rotation as a tree of $2 \times 2$ planar rotations, each stored with only two bits, followed by a fixed Hadamard transform shared across all blocks. The rotations are fit without gradients or changes to model weights, and we prove that a simple greedy fit is globally optimal for our 'mean-flattening' objective. The transforms are fit online for each new video, and we release fused Triton kernels to make this process efficient at inference. At 2-bit precision and matched compression, TRQ improves VBench subject consistency over the state-of-the-art method QVG from 90.38 to 92.02 on LongCat-Video-13B and from 86.33 to 87.61 on HY-WorldPlay-8B, as well as improving other metrics.

## Chameleon: Dynamic Format Adapter for Efficient Diffusion

DevFeed: [Chameleon: Dynamic Format Adapter for Efficient Diffusion](<https://devfeed.tech/articles/chameleon-dynamic-format-adapter-for-efficient-diffusion-144937.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.33496>)

Author: Arnab Sanyal, Sandeep Chinchali

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [adapter](<https://devfeed.tech/tags/adapter.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [format](<https://devfeed.tech/tags/format.md>), [framework](<https://devfeed.tech/tags/framework.md>), [int8](<https://devfeed.tech/tags/int8.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [kurtosis](<https://devfeed.tech/tags/kurtosis.md>), [layer](<https://devfeed.tech/tags/layer.md>), [layers](<https://devfeed.tech/tags/layers.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [memory](<https://devfeed.tech/tags/memory.md>), [snr](<https://devfeed.tech/tags/snr.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [table](<https://devfeed.tech/tags/table.md>), [time](<https://devfeed.tech/tags/time.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

Chameleon improves diffusion-model post-training quantization by selecting the number format at a fixed bit-width to fit each weight channel and activation layer, with optional refinement by timestep. It chooses activation formats from kurtosis and diffusion SNR statistics, and weight formats offline by reconstruction error. The abstract reports that per-layer selection provides most of the gain, while timestep refinement helps mainly with 6-bit activations. Across SDXL, SDXL-Turbo and PixArt-α on COCO-2014, Chameleon achieved the best FID in six backbone and bit-width settings; CLIP scores were within 0.24 of the FP16 reference. On PixArt-α, a mixed-format kernel ran 1.09x faster than FP16 end-to-end with weights three times smaller. These are results reported in the supplied abstract excerpt.

### Source excerpt

arXiv:2609.33496v3 Announce Type: replace Abstract: Post-training quantization (PTQ) is the standard way to run modern diffusion models on memory-constrained accelerators, yet every existing diffusion PTQ scheme fixes the $\mathit{number\ format}$ in advance and only tunes the scale, zero point, or per-layer bit-width. At a fixed bit-width the best format depends on the distribution being encoded, and that distribution differs across weight channels, across layers, and along the diffusion timestep, where activation distributions slide from heavy-tailed and noise-dominated to tightly clustered and structured. We propose Chameleon, a PTQ framework that holds the bit-width fixed and treats the format itself as a discrete variable, chosen per weight channel and per activation layer, optionally refined per timestep bucket. Activation formats come from {INT8, FP8 E4M3, FP8 E5M2, MXFP8, MXINT8}, selected ahead of time from two cheap statistics (empirical kurtosis and the closed-form diffusion SNR) and stored in a lookup table; weight formats come from {INT8, MXINT8} at 8 bits or {INT4, NF4, FP4 E2M1, MXINT4, MXFP4} at 4 bits, selected offline by reconstruction error. An architectural fork adapts the same selection layer to multi-step UNets, single-step distilled models, and Diffusion Transformers. Most of the gain comes from the per-layer choice; the timestep refinement helps mainly at 6-bit activations. Across SDXL, SDXL-Turbo, and PixArt-$\alpha$ on COCO-2014, Chameleon achieves the best FID in all six backbone $\times$ bit-width settings, with CLIP within 0.24 of the FP16 reference and the best of all quantized methods at $W_{4}A_{8}$. On PixArt-$\alpha$, a real mixed-format kernel runs 1.09$\times$ faster than FP16 end-to-end with 3$\times$ smaller weights.

## Inject Where It Matters: Objective-Dependent Feature Control in Rectified-Flow Image Editing

DevFeed: [Inject Where It Matters: Objective-Dependent Feature Control in Rectified-Flow Image Editing](<https://devfeed.tech/articles/inject-where-it-matters-objective-dependent-feature-control-in-rectified-flow-image-editing-145589.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25811>)

Author: Nina Goncharova, Alexandra Ivanova, Aibek Alanov

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [image editing](<https://devfeed.tech/topics/image-editing.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [bench](<https://devfeed.tech/tags/bench.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [feature-injection](<https://devfeed.tech/tags/feature-injection.md>), [flux](<https://devfeed.tech/tags/flux.md>), [image-editing](<https://devfeed.tech/tags/image-editing.md>), [inference](<https://devfeed.tech/tags/inference.md>), [masking](<https://devfeed.tech/tags/masking.md>), [masks](<https://devfeed.tech/tags/masks.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [top-k](<https://devfeed.tech/tags/top-k.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

The method controls source-feature injection in prompt-driven FLUX image editing through adaptive spatial masks and REINFORCE-selected Top-K transformer blocks. Masks gate injection using differences between source- and target-conditioned outputs at matched inversion states; selected blocks remain fixed at inference, with no online search. On PIE-Bench with FLUX.1-dev, masking controls the alignment-preservation trade-off. Across 60 validation examples, a learned nine-block set achieved lower whole-image LPIPS than RF-Edit's 18-block set in the same masked pipeline, with slightly lower alignment. It had the lowest LPIPS among 50 random nine-block sets, though not the highest reward. Objective sweeps favored different blocks for preservation and alignment.

### Source excerpt

arXiv:2611.25811v1 Announce Type: new Abstract: Many prompt-driven FLUX editing methods rely on injecting source features into target-conditioned generation, creating an inherent trade-off between source preservation and the desired edit. We address this problem by controlling where source features are injected and which transformer blocks are used. First, we introduce adaptive masks that spatially gate feature injection using differences between source- and target-conditioned block outputs at matched inversion states. Second, we use REINFORCE to select Top-K injection blocks under different preservation-editability objectives. The selected blocks are fixed at inference and require no online search. On PIE-Bench with FLUX.1-dev, spatial masking controls the alignment-preservation trade-off. On 60 validation examples, a learned nine-block set gives lower whole-image LPIPS than the 18-block RF-Edit set within the same masked pipeline, with slightly lower alignment. Among 50 random nine-block sets, it has the lowest LPIPS but not the highest reward. Separate objective sweeps reveal different block preferences for preservation and alignment.

## HiUni: Harmonizing Understanding and Generation in Shared Visual Space

DevFeed: [HiUni: Harmonizing Understanding and Generation in Shared Visual Space](<https://devfeed.tech/articles/hiuni-harmonizing-understanding-and-generation-in-shared-visual-space-145631.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26107>)

Author: Jiahao Hu, Jingdong Du, Yichen Zhang, Yi Song, Zhenyu Zhang, Jiaxin Niu, Ruihang Li, Xiangzhao Hao, Yiqian Zhang, Yong Huang, Zefeng Zhang, Zheyuan Gu, Shuohuan Wang

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [comparisons](<https://devfeed.tech/tags/comparisons.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [generation](<https://devfeed.tech/tags/generation.md>), [image-generation](<https://devfeed.tech/tags/image-generation.md>), [key](<https://devfeed.tech/tags/key.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [pixel](<https://devfeed.tech/tags/pixel.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [sharing](<https://devfeed.tech/tags/sharing.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [transformer](<https://devfeed.tech/tags/transformer.md>), [vision-language-model](<https://devfeed.tech/tags/vision-language-model.md>), [vit](<https://devfeed.tech/tags/vit.md>)

### AI overview

HiUni extends a pretrained vision-language model for understanding and image generation using the same language-aligned ViT and dense Transformer backbone, with task-specific computation confined to the output. To address the difficulty of optimizing pixel-space generation and the loss of spatial detail in the pretrained ViT, it adds a Pixel Diffusion Head and a lightweight HiGate pathway. The abstract reports an average understanding score similar to the pretrained model and competitive image generation and editing performance; it provides no specific scores or benchmark details.

### Source excerpt

arXiv:2611.26107v1 Announce Type: cross Abstract: Unified multimodal models aim to support visual understanding and generation within a single framework. However, prevailing approaches often separate their visual representations or backbone computation to accommodate the different requirements of these two tasks, which limits the degree of unification. In this paper, we present HiUni, which revisits this design choice and explores whether a pretrained vision language model can be directly extended into a unified model with extensive parameter sharing. Specifically, HiUni reuses the same language-aligned ViT and dense Transformer backbone for both capabilities, while confining task-specific computation to the output side. Yet such a design introduces two key challenges. Generation in pixel space is difficult to optimize, and semantic abstraction in pretrained ViT discards fine spatial details required for synthesis. To address these challenges, HiUni introduces a \emph{Pixel Diffusion Head} tailored to direct pixel-space optimization and a lightweight \emph{HiGate} pathway that provides fine-grained spatial details near the output stage. Extensive experiments show that HiUni achieves a similar average understanding score to the pretrained VLM while attaining competitive performance on image generation and editing. Controlled comparisons and analyses further demonstrate that a single dense backbone can effectively support both capabilities without task-specific backbone computation, highlighting a simpler path toward unified multimodal modeling.

## Every Query Routes Itself: Accelerating Video Diffusion Transformers with Per-Query Routed Sparse Attention

DevFeed: [Every Query Routes Itself: Accelerating Video Diffusion Transformers with Per-Query Routed Sparse Attention](<https://devfeed.tech/articles/every-query-routes-itself-accelerating-video-diffusion-transformers-with-per-query-routed-sparse-attention-143528.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24479>)

Author: Zhen Huang, Shuwei Li, Ruizhe Yao, Xinrui Chen, Danyi Liu, Haoyuan Liang, Zhiyu Ye, Qiuyan Sun, Bingwei Chen, Yiyuan Li, Xu Zhou, Haohuan Fu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [inference](<https://devfeed.tech/tags/inference.md>), [query](<https://devfeed.tech/tags/query.md>), [sparse-attention](<https://devfeed.tech/tags/sparse-attention.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

PQRS reduces video diffusion transformer attention cost by clustering keys only and allowing each query to select its own key clusters; centroids compensate for unselected clusters. To handle these distinct selections efficiently, it traverses key clusters, groups queries with identical selections into dense tasks, and merges partial results in one pass using a softmax offset carried from the previous denoising step. The training-free method was evaluated on four text-to-video and image-to-video models from the Wan and HunyuanVideo families. The authors report it was the fastest evaluated sparse attention method and closest to dense output, with 28.9 dB PSNR on Wan2.1-T2V and 35.4 dB on HunyuanVideo at 1.66-2.10x end-to-end speedup.

### Source excerpt

arXiv:2611.24479v1 Announce Type: cross Abstract: Self-attention over long token sequences dominates the inference cost of video diffusion transformers. Clustering-based sparse attention clusters both queries and keys and lets all queries in a cluster share one selection of key clusters. We show that this sharing limits accuracy: with key clusters and density fixed, attention grows more accurate as the query clusters are refined, and is most accurate when each query selects on its own, which needs no query clustering at all. We therefore propose Per-Query Routed Sparse Attention (PQRS), a training-free method that clusters only keys, lets each query select its own key clusters, and compensates for the unselected ones with their centroids. Per-query selections leave no query tile with a shared key list, so PQRS traverses key clusters instead of queries, buckets queries with identical selections into dense tasks, and merges their partial results in one pass under a fixed softmax offset carried from the previous denoising step. On four text-to-video and image-to-video models from the Wan and HunyuanVideo families, PQRS is the fastest of all the sparse attention methods we evaluate and the closest to the dense output, reaching $28.9$ dB PSNR on Wan2.1-T2V and $35.4$ dB on HunyuanVideo at $1.66$--$2.10\times$ end-to-end speedup.

## Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving

DevFeed: [Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving](<https://devfeed.tech/articles/multimodal-masked-diffusion-autoencoder-for-modeling-impaired-driving-141861.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23211>)

Author: John Gideon, Anshul Gupta, Megan Applegate-Kenton, Kayli Battel, Todd J Rowell, Patricio Reyes Gomez, Laporsha Dees, Emily Sumner, Avinash Balachandran, Guy Rosman

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [attention](<https://devfeed.tech/tags/attention.md>), [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [design](<https://devfeed.tech/tags/design.md>), [detection](<https://devfeed.tech/tags/detection.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [driving](<https://devfeed.tech/tags/driving.md>), [effects](<https://devfeed.tech/tags/effects.md>), [environment](<https://devfeed.tech/tags/environment.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [inference](<https://devfeed.tech/tags/inference.md>), [input](<https://devfeed.tech/tags/input.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>)

### AI overview

ID-MAE combines impairment-label prediction with self-supervised reconstruction of masked gaze and trajectory, aiming to improve impairment inference when cognitive-state labels are costly. The authors introduce DIVERT, a driving-simulator dataset with 103 hours from 109 participants, covering ten cognitive, visual, and emotional impairment inductions and recording gaze, vehicle state, and scene video. On location-matched DIVERT pairs, ID-MAE reaches a paired AUC of 0.706 for impairment detection, versus 0.590 without reconstruction loss; the abstract also reports improvements on existing datasets and strong trajectory and gaze prediction. The authors report that visually demanding distractions are separable from cognitive distractions. Code and data are planned for release upon acceptance.

### Source excerpt

arXiv:2611.23211v1 Announce Type: cross Abstract: Human cognitive state (e.g., distraction, stress), gaze, and behavior interact with the driving environment in subtle ways that AI could use to assist drivers, but ground-truth labels for human cognitive state levels are costly to obtain at scale. We propose a multimodal masked-diffusion autoencoder, ID-MAE, that adds a self-supervised objective to impairment prediction: alongside predicting the impairment label as a masked token, it reconstructs masked gaze and trajectory. To probe the effects of different modalities and types of impairments on model inference, we introduce DIVERT, a driving-simulator dataset containing 103 hours of driving data (n = 109) with ten impairment inductions spanning cognitive, visual, and emotional states, recorded with gaze, vehicle state, and scene video. We demonstrate how ID-MAE affords significant improvements in impairment inference results on both the DIVERT dataset and existing datasets, and enables strong results in additional tasks, such as trajectory and gaze prediction. For location-matched pairs, our model achieves a paired AUC of 0.706 for impairment detection on DIVERT, compared to 0.590 without reconstruction loss, improving over existing baselines. We probe the design space in terms of input modalities, masking scheme, positional encoding, attention mechanism, and decoding. We also investigate the interaction between various impairments in training and inference, demonstrating that visually demanding distractions are separable from cognitive distractions. Code and data will be available upon acceptance.

## Hierarchical Diffusion Transformer for End-to-End Image Generation

DevFeed: [Hierarchical Diffusion Transformer for End-to-End Image Generation](<https://devfeed.tech/articles/hierarchical-diffusion-transformer-for-end-to-end-image-generation-140096.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22962>)

Author: Xiaoyu Wu, Yifei Wang, Weihang Guo, Chen Wei, Lydia E Kavraki, Niloofar Mireshghallah, Steven Wu

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [Image](<https://devfeed.tech/topics/image.md>), [ImageNet](<https://devfeed.tech/topics/imagenet.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [generation](<https://devfeed.tech/tags/generation.md>), [hierarchy](<https://devfeed.tech/tags/hierarchy.md>), [hit](<https://devfeed.tech/tags/hit.md>), [image](<https://devfeed.tech/tags/image.md>), [image-generation](<https://devfeed.tech/tags/image-generation.md>), [images](<https://devfeed.tech/tags/images.md>), [jit](<https://devfeed.tech/tags/jit.md>), [pixel](<https://devfeed.tech/tags/pixel.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [quality](<https://devfeed.tech/tags/quality.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>), [token](<https://devfeed.tech/tags/token.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [training](<https://devfeed.tech/tags/training.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

HiT improves pixel-space diffusion efficiency by progressively grouping related image tokens into compact group tokens for the Transformer bottleneck, then restoring features to their original positions. A compression budget lets group membership and counts vary by image and diffusion timestep. The model learns coherent object and part groupings without segmentation labels; for text-to-image, it also aligns caption words with corresponding image patches without region annotations. On ImageNet 256x256, HiT reports 1.48 FID and generation quality comparable to JiT at about 40% of its cumulative training FLOPs. Across five text-to-image benchmarks, it exceeds UniLiP's average score with roughly 6x fewer denoiser parameters and 3x fewer forward FLOPs. The abstract reports these results for end-to-end pixel-space generation without external visual pretraining.

### Source excerpt

arXiv:2611.22962v1 Announce Type: cross Abstract: Pixel-space diffusion learns visual representations and image generation jointly in a single end-to-end training stage, without a separately pretrained visual encoder. However, plain pixel-space Transformers still trail leading two-stage latent diffusion models in generation quality and training efficiency. We hypothesize that part of this gap arises from retaining one token per patch throughout the backbone, which does not explicitly encourage a visual hierarchy that summarizes related content at progressively coarser levels. To address this, we introduce Hierarchical Diffusion Transformers (HiT), which progressively group related tokens and represent each group with a single token. The main Transformer bottleneck processes these compact group tokens, and ungrouping restores features to their original image positions. A compression budget regulates the average compression ratio across the batch, allowing group memberships and counts to vary across images and diffusion timesteps during both training and generation. The model exhibits emergent semantic structure, learning to group coherent objects and object parts without segmentation labels. In text-to-image generation, HiT further learns to group caption words with image patches depicting the same concepts, without region annotations. HiT achieves 1.48 FID on ImageNet $256\times256$ and reaches comparable generation quality to JiT using approximately 40\% of its cumulative training FLOPs. For text-to-image generation, HiT exceeds UniLiP's average score across five benchmarks with roughly $6\times$ fewer denoiser parameters and $3\times$ fewer forward FLOPs. These results suggest that learning a visual hierarchy within the generator breaks down the barrier to stronger image generation without external visual pretraining.

## WoRD: Diffusion-Based Posterior Inference for Online Goal Recognition

DevFeed: [WoRD: Diffusion-Based Posterior Inference for Online Goal Recognition](<https://devfeed.tech/articles/word-diffusion-based-posterior-inference-for-online-goal-recognition-140129.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23192>)

Author: Almog Anschel, Sarah Keren

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>), [Calibrated uncertainty](<https://devfeed.tech/topics/calibrated-uncertainty.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [bayes](<https://devfeed.tech/tags/bayes.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [inference](<https://devfeed.tech/tags/inference.md>), [motion](<https://devfeed.tech/tags/motion.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [online](<https://devfeed.tech/tags/online.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [robot](<https://devfeed.tech/tags/robot.md>), [vehicle](<https://devfeed.tech/tags/vehicle.md>)

### AI overview

WoRD infers an agent's goal from partial continuous trajectories by using a learned trajectory prior rather than requiring an explicit dynamics model. For each candidate goal, it runs a warm-started denoising chain, steers samples toward observations with a closed-form log-likelihood gradient, then applies Bayes' rule to form a calibrated goal posterior. In evaluations on simulated robots, pedestrians, and vehicles, the method reportedly preserved multiple plausible goals through ambiguous prefixes, while filter- and classifier-based observers sometimes collapsed to one mode or became overconfident. The supplied text is an abstract and does not provide metric names or detailed results.

### Source excerpt

arXiv:2611.23192v1 Announce Type: new Abstract: Goal recognition is the task of inferring an agent's intended goal from its observed behavior. Most existing work focuses on high-level symbolic recognition, abstracting away the continuous motion through which agents reach their goal. Methods that do reason over continuous trajectories rely on an explicit dynamics model that is hard to acquire and rarely transfers across settings. Neither approach explicitly learns how agents move toward each candidate goal, limiting accuracy on ambiguous trajectory prefixes where such knowledge is most needed. We present WoRD (Waypoint Recognition with Diffusion), which uses a learned trajectory prior to infer the agent's goal from partial observations. At each step, WoRD runs one warm-started denoising chain per candidate goal, steers each chain toward the observed trajectory through a closed-form log-likelihood gradient, and reweights the resulting samples by Bayes' rule into a calibrated posterior over goals. We evaluate WoRD on simulated robot settings, pedestrian trajectories, and vehicle trajectories using four metrics targeting distinct failure modes. Across all three settings, the same inference procedure preserves calibrated multimodal beliefs through ambiguous prefixes where filter- and classifier-based observers collapse to a single mode or grow overconfident.

## LoopPrune: Recurrent Pruning for Looped Diffusion Transformers

DevFeed: [LoopPrune: Recurrent Pruning for Looped Diffusion Transformers](<https://devfeed.tech/articles/loopprune-recurrent-pruning-for-looped-diffusion-transformers-141806.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22757>)

Author: Weizhong Huang, Huaijia Kou, Yan Zhang, Fei Chao, Xiawu Zheng, Liujuan Cao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [pruning](<https://devfeed.tech/tags/pruning.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

LoopPrune prunes shared structures in looped diffusion Transformers by estimating recurrent importance from shared-removal effects and position variation, then reconstructing weights from pooled recurrent input statistics. During recovery, it matches complete denoising predictions and text guidance at identical student states, updating the student against a fixed dense teacher. The abstract reports that Looped-DiT B/16 retains 93.14% of dense average accuracy across six benchmarks at 30% structured parameter pruning, 9.49 percentage points above the best baseline. For pruned SDXSv3, results across PyTorch, vLLM-Omni and SGLang Diffusion show 34.36-44.64% lower peak allocated GPU memory and 1.46-1.74x faster denoising than dense models. These are abstract-level claims; further experimental details are not provided.

### Source excerpt

arXiv:2611.22757v1 Announce Type: new Abstract: Looped diffusion Transformers reuse shared blocks to deepen computation and match larger models with fewer parameters, reducing storage and deployment burdens. Yet repeated attention and MLP computations remain costly. Pruning reduces storage and computation by removing redundant parameters or units. Training-free methods struggle to resolve recurrent contributions, while training-based methods lack explicit constraints on position-specific errors affecting later inputs and final outputs. We propose LoopPrune, a pruning and recovery framework addressing these requirements. Recurrent importance estimation combines shared-removal effects and position variation to select a shared structure; shared reconstruction pools recurrent input statistics to compensate weights. Loop-position recovery compares complete denoising predictions at identical student states and matches text guidance, updating all student parameters against a fixed dense teacher. Extensive experiments across four open-source models, six benchmarks, and three pruning paradigms demonstrate effectiveness. At 30% structured parameter pruning, Looped-DiT B/16 retains 93.14% of dense six-benchmark average accuracy, exceeding the best baseline by 9.49%. Across PyTorch, vLLM-Omni, and SGLang Diffusion, pruned SDXSv3 reduces peak allocated GPU memory by 34.36-44.64% and accelerates denoising 1.46-1.74x over dense models.

## Mix-Opt: Mixed Optimization for Memory-Efficient Personalization of Text-to-Image Diffusion Models

DevFeed: [Mix-Opt: Mixed Optimization for Memory-Efficient Personalization of Text-to-Image Diffusion Models](<https://devfeed.tech/articles/mix-opt-mixed-optimization-for-memory-efficient-personalization-of-text-to-image-diffusion-models-141525.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2507.10029>)

Author: Seokeon Choi, Sunghyun Park, Hyoungwoo Park, Jeongho Kim, Sungrack Yun

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [backpropagation](<https://devfeed.tech/tags/backpropagation.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [dit](<https://devfeed.tech/tags/dit.md>), [edge-devices](<https://devfeed.tech/tags/edge-devices.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [image](<https://devfeed.tech/tags/image.md>), [memory](<https://devfeed.tech/tags/memory.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [overfitting](<https://devfeed.tech/tags/overfitting.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [performance](<https://devfeed.tech/tags/performance.md>), [personalization](<https://devfeed.tech/tags/personalization.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [process](<https://devfeed.tech/tags/process.md>), [quality](<https://devfeed.tech/tags/quality.md>), [resources](<https://devfeed.tech/tags/resources.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>)

### AI overview

Mix-Opt combines backpropagation on low-resolution images (BP-low) with zeroth-order optimization on high-resolution images (ZO-high) to personalize text-to-image diffusion models with lower memory use. BP-low adapts target-specific features efficiently but can distort structure; ZO-high refines high-resolution detail with minimal memory overhead but converges slowly without prior adaptation. A timestep-aware probabilistic function selects between them during training, aiming to limit BP-low overfitting when structural information matters and use ZO-high more effectively as training progresses. Experiments on U-Net- and DiT-based models report competitive personalization with significantly reduced memory consumption, without changing inference or increasing its latency.

### Source excerpt

arXiv:2507.10029v3 Announce Type: replace-cross Abstract: Memory-efficient personalization is essential for adapting text-to-image diffusion models while preserving user privacy and operating within the limited computational resources of edge devices. To this end, we propose Mix-Opt, a novel mixed optimization framework that adaptively chooses between backpropagation on low-resolution images (BP-low) and zeroth-order optimization on high-resolution images (ZO-high), guided by the characteristics of the diffusion process. As observed in our experiments, BP-low efficiently adapts the model to target-specific features but suffers from structural distortions due to resolution mismatch. Conversely, ZO-high refines high-resolution details with minimal memory overhead but faces slow convergence when applied without prior adaptation. By complementing both methods, our framework leverages BP-low for effective personalization while using ZO-high to maintain structural consistency, achieving memory-efficient and high-quality fine-tuning. To maximize the efficacy of both BP-low and ZO-high, we introduce a timestep-aware probabilistic function that dynamically selects the appropriate optimization strategy during training. This function mitigates overfitting from BP-low at high timesteps, where structural information is critical, while ensuring that ZO-high is applied more effectively as training progresses. Experimental results across both U-Net- and DiT-based diffusion models demonstrate that Mix-Opt achieves competitive personalization performance while significantly reducing memory consumption. Mix-Opt enables scalable, high-quality personalization without modifying the inference process or increasing inference latency.

## Amortizing Tempered Sampling with Corrected Self-Training

DevFeed: [Amortizing Tempered Sampling with Corrected Self-Training](<https://devfeed.tech/articles/amortizing-tempered-sampling-with-corrected-self-training-141838.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23072>)

Author: Hoang-Viet Nguyen, Hai-Dang Dau, Alexandre H. Thi\\'ery

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [energy](<https://devfeed.tech/tags/energy.md>), [markov-chain-monte-carlo](<https://devfeed.tech/tags/markov-chain-monte-carlo.md>), [model](<https://devfeed.tech/tags/model.md>), [regression](<https://devfeed.tech/tags/regression.md>), [runtimes](<https://devfeed.tech/tags/runtimes.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [target](<https://devfeed.tech/tags/target.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The paper proposes corrected self-training across tempered targets to build a reusable neural sampler, using importance sampling, Markov chain Monte Carlo, and replay buffers to correct and retain samples. Its Sequential Score Sampler (SSS) uses a score-based diffusion proposal trained by regression on energy gradients, with importance weights applied to diffusion paths. After training, samples can come directly from the model, from importance-corrected proposals, or from the final-temperature buffer. The authors report competitive accuracy across benchmarks and shorter runtimes with fewer energy evaluations than competing samplers on several targets.

### Source excerpt

arXiv:2611.23072v1 Announce Type: cross Abstract: Learning a neural sampler can reduce the cost of repeated sampling from an un-normalized distribution, but obtaining good training data remains difficult. We develop a corrected self-training scheme across a sequence of tempered targets, making use of importance sampling and Markov chain Monte Carlo to correct the samples and replay buffers to store them. We introduce Sequential Score Sampler (SSS) as a particular case. In SSS, the proposal is a score-based diffusion trained by regression on energy gradients; and the samples are corrected by importance weights on the path space. After training, samples can be collected directly from the model, from its importance-corrected proposals, or from the final-temperature buffer. The method amortizes repeated sampling by encoding the global structure of the target in the model. SSS achieves competitive sampling accuracy across the benchmarks and, on several targets, uses shorter runtimes with fewer energy evaluations than competing samplers.

## Stochastic Diversity Windows: Diagnosing Noise Influence in Robotic Diffusion Sampling

DevFeed: [Stochastic Diversity Windows: Diagnosing Noise Influence in Robotic Diffusion Sampling](<https://devfeed.tech/articles/stochastic-diversity-windows-diagnosing-noise-influence-in-robotic-diffusion-sampling-141805.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22743>)

Author: Tina Lakhani, Jing Wu, Yu-Kun Lai, Ze Ji

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robotics](<https://devfeed.tech/topics/robotics.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [express](<https://devfeed.tech/tags/express.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [impact](<https://devfeed.tech/tags/impact.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [noise](<https://devfeed.tech/tags/noise.md>), [robotic](<https://devfeed.tech/tags/robotic.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [solution](<https://devfeed.tech/tags/solution.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

A per-timestep counterfactual diagnostic measures how changing sampled noise affects robotic diffusion model outputs. Across five model-sampler configurations and three benchmarks, noise effects vary over denoising: they can amplify, contract, or be nearly erased. The authors use these profiles to identify stochastic diversity windows (SDWs), where noise has the strongest measured influence. Two heuristic interventions follow: targeted stochastic amplification to increase exploration, and an SDW-hybrid sampler that replaces DDPM steps with DDIM skips in low-impact regions. The hybrid reports up to 40-60% fewer network evaluations and up to 85% lower wall-clock time; held-out tasks showed no consistent changes in feasibility, cost, or diversity.

### Source excerpt

arXiv:2611.22743v1 Announce Type: cross Abstract: Diffusion models are increasingly adopted in robotics and control due to their capability to express multiple feasible solutions to a given task. However, the role of stochasticity in the diffusion model-sampler across denoising remains poorly understood. We introduce a per-timestep counterfactual diagnostic that tests how changing the sampled noise affects the final output. Across five model-sampler configurations spanning three robotics benchmarks, diverse architectures, domains, and conditioning mechanisms, we reveal highly varied temporal sensitivity profiles, where noise-induced deviations can amplify, partially contract, or undergo near-total attenuation downstream. From these profiles, we define stochastic diversity windows (SDWs): compact intervals that identify where sampler stochasticity has the strongest measured influence on the final output. Guided by these windows, we apply two heuristic interventions: targeted stochastic amplification, which increases sample diversity exploration of the solution space, and an SDW-hybrid sampler, which replaces DDPM steps with DDIM skips in low-impact regions, resulting in network evaluation reductions of up to 40-60% and wall-clock time by up to 85%, with no consistent changes detected in feasibility, cost, or diversity across held-out tasks.

## Latent Fusion: Channel Stacking for Structure-Preserving Image Editing

DevFeed: [Latent Fusion: Channel Stacking for Structure-Preserving Image Editing](<https://devfeed.tech/articles/latent-fusion-channel-stacking-for-structure-preserving-image-editing-137333.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18727>)

Author: Mohit Dulani

Published: 2026-11-17T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [image editing](<https://devfeed.tech/topics/image-editing.md>), [Image](<https://devfeed.tech/topics/image.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [flux](<https://devfeed.tech/topics/flux.md>)

Tags: [appearance](<https://devfeed.tech/tags/appearance.md>), [attention](<https://devfeed.tech/tags/attention.md>), [channel](<https://devfeed.tech/tags/channel.md>), [concatenation](<https://devfeed.tech/tags/concatenation.md>), [condition](<https://devfeed.tech/tags/condition.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [editor](<https://devfeed.tech/tags/editor.md>), [edits](<https://devfeed.tech/tags/edits.md>), [flux-2](<https://devfeed.tech/tags/flux-2.md>), [image](<https://devfeed.tech/tags/image.md>), [image-editing](<https://devfeed.tech/tags/image-editing.md>), [layout](<https://devfeed.tech/tags/layout.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [projection](<https://devfeed.tech/tags/projection.md>), [recipe](<https://devfeed.tech/tags/recipe.md>), [sd3-5](<https://devfeed.tech/tags/sd3-5.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [source](<https://devfeed.tech/tags/source.md>), [structure](<https://devfeed.tech/tags/structure.md>)

### AI overview

For structure-preserving appearance edits under a small-data, 4B-backbone, 2K-frame regime, channel-wise latent stacking offers lower computational cost and preserved pixel alignment in the reported runs. Compared with sequence concatenation, it keeps sequence length unchanged and uses two rather than three forward passes per step; the matched training setup found sequence concatenation displaced structure, but the authors do not claim a general ranking. On 29 held-out images, the editor scored 20.3 dB PSNR and 0.198 LPIPS against human edits (unedited source: 15.0 dB). Counterfactual tests exposed a limitation: it largely learned a global source grade rather than selective editing.

### Source excerpt

arXiv:2611.18727v1 Announce Type: cross Abstract: Instruction-based image editing models must condition on the source image, either by sequence concatenation, which appends source tokens to the target sequence (FLUX.1 Kontext), or by channel-wise concatenation, which stacks the source latent onto the noisy latent and widens the input projection (InstructPix2Pix, FLUX.1 Fill). The FLUX.1 Kontext report found channel-wise concatenation worse in initial experiments at 1K resolution with a 12B model and a large corpus. We explore channel-wise concatenation as the lower-cost option for a common production regime: structure-preserving edits that change appearance (sky, exposure, foliage, cleanup) but not layout, about 1,400 training pairs, a 4B non-distilled backbone, and 2K frames. It keeps the sequence at its original length, where sequence concatenation doubles it and roughly quadruples attention cost, and needs only single-scale classifier-free guidance (two forward passes per step instead of three). Starting from FLUX.2 Klein 4B base, we widen only the input projection and train both schemes with the same trainer, data, and hyperparameters. Channel-wise concatenation keeps the output pixel-aligned with the source; in our runs sequence concatenation, which must learn the correspondence between 10,880 target and 10,880 source tokens from fewer than 1,500 pairs, displaced structure, an observation we report without claiming a general ranking. On 29 held-out images the editor reaches 20.3 dB PSNR and 0.198 LPIPS against the human edit (unedited source: 15.0 dB), and counterfactual conditioning tests show it largely learned a global grade of the source rather than a selective edit, which we report as the open problem of this recipe. We also compare against an SD3.5-Medium editor trained with the same surgery and report the full training recipe, including failure modes. Code: https://github.com/complete-dope/LatentFusion

## PlanDiv: Test-Time Scaling with Latent Plans for Vision-Language-Action Models

DevFeed: [PlanDiv: Test-Time Scaling with Latent Plans for Vision-Language-Action Models](<https://devfeed.tech/articles/plandiv-test-time-scaling-with-latent-plans-for-vision-language-action-models-138975.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20010>)

Author: Hanbo Zhang, Lianlei Shan, Liuhaichen Yang, Zezhi Tang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [VAE](<https://devfeed.tech/topics/vae.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [vision-language-action](<https://devfeed.tech/tags/vision-language-action.md>), [vision-language-action-models](<https://devfeed.tech/tags/vision-language-action-models.md>)

### AI overview

PlanDiv improves test-time scaling for vision-language-action policies by making action candidates represent different latent plans instead of resampling noise from one decoder. A lightweight planner proposes plans learned from demonstrations, and a shared masked-diffusion executor produces an action chunk for each; the policy selects the chunk with the highest likelihood. With the backbone, data, and training recipe held fixed, eight candidates raised CALVIN five-task chain success from 56.9% to 69.9%, while matched alternatives reached at most 64.9%. On LIBERO-Long, four plans removed 40% of single-candidate failures, versus at most 21% for matched resampling. On a real Franka arm, PlanDiv matched the single-output baseline on a task with one solution and succeeded in 51 of 54 trials versus 40 of 54 across two tasks with two solutions.

### Source excerpt

arXiv:2611.20010v1 Announce Type: cross Abstract: Test-time scaling improves vision-language-action (VLA) policies by drawing several action candidates and executing the best one. We find that its returns diminish quickly when the candidates are resampled from one decoder: they differ by noise rather than by plan, and for each of three resampling sources, going from four to eight candidates adds only 0.06-0.07 to the average number of completed tasks on CALVIN. We introduce PlanDiv, an action head that makes the candidates differ by plan: a lightweight planner proposes K latent plan hypotheses, a shared masked-diffusion executor decodes one action chunk per plan from a single backbone pass, and the policy executes the chunk that its own likelihood ranks highest. The plans are learned from demonstrations alone, without plan labels or rewards. With the backbone, data and training recipe held fixed, eight plan-conditioned candidates raise five-task chain success on CALVIN ABC$\rightarrow$D from 56.9% to 69.9%, while eight candidates from autoregressive, plan-free or same-plan resampling reach at most 64.9%. The same-plan baseline is PlanDiv's own checkpoint with the plan held fixed, so this gap comes from the plans, not from drawing more samples. On LIBERO-Long, four plans remove 40% of single-candidate failures, against at most 21% for matched resampling. On a real Franka arm the gain appears where a task admits more than one solution: PlanDiv is on par with a single-output OpenVLA-OFT+ baseline on a one-solution task and succeeds in 51 of 54 trials against 40 of 54 on two tasks with two solutions.

## Memorization Lives in a Few Attention Heads: Mitigating Memorization in Diffusion Models via Projection-Based Suppression

DevFeed: [Memorization Lives in a Few Attention Heads: Mitigating Memorization in Diffusion Models via Projection-Based Suppression](<https://devfeed.tech/articles/memorization-lives-in-a-few-attention-heads-mitigating-memorization-in-diffusion-models-via-projection-based-suppression-139274.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19427>)

Author: Puwei Lian, Yujun Cai, Songze Li, Yuezu Lv, Guanghui Wen

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [attention](<https://devfeed.tech/tags/attention.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [mitigation](<https://devfeed.tech/tags/mitigation.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [projection](<https://devfeed.tech/tags/projection.md>), [quality](<https://devfeed.tech/tags/quality.md>), [score](<https://devfeed.tech/tags/score.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [top-k](<https://devfeed.tech/tags/top-k.md>)

### AI overview

The paper finds that diffusion-model memorization is driven mainly by a small subset of shallow cross-attention heads with unusually strong semantic control, rather than cross-attention as a whole. It proposes Projection-based Top-k Attention Head Suppression: identify the most memorization-related heads with a semantic-control score, then remove their condition-specific components by projecting between conditional and unconditional representations. The authors report reduced memorization and a better balance with image quality than existing approaches across multiple models and datasets. The evidence provided is an abstract, so it does not specify the experimental details or limitations.

### Source excerpt

arXiv:2611.19427v1 Announce Type: cross Abstract: Diffusion models can memorize and reproduce training samples during inference, raising serious privacy and copyright concerns. Existing inference-time mitigation methods often suppress conditional information globally, which may reduce memorization but can also harm semantic alignment and image quality. In this paper, we revisit memorization from the perspective of cross-attention heads. Through systematic analysis and targeted ablation, we find that memorization is mainly driven by a small subset of shallow-layer attention heads with abnormally strong semantic control, rather than by the entire cross-attention mechanism. Motivated by this finding, we propose Projection-based Top-k Attention Head Suppression, a targeted inference-time mitigation method. Our method first identifies the top-k attention heads most related to memorization using a semantic-control score, and then removes their condition-specific components through directional projection between conditional and unconditional representations. This design selectively weakens memorization-related features while preserving useful semantic information. Extensive experiments on different diffusion models and datasets show that our method effectively reduces memorization and achieves a better trade-off between memorization mitigation and image quality than existing approaches.

## A COMPASS to the Galaxy: Joint nucleosynthetic yield set comparison and parameter inference in Galactic chemical evolution

DevFeed: [A COMPASS to the Galaxy: Joint nucleosynthetic yield set comparison and parameter inference in Galactic chemical evolution](<https://devfeed.tech/articles/a-compass-to-the-galaxy-joint-nucleosynthetic-yield-set-comparison-and-parameter-inference-in-galactic-chemical-evolution-139134.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21175>)

Author: Berkay G\\"unes, Giuseppe Viterbo, Sven Buder, Tobias Buck

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [astro-ph-ga](<https://devfeed.tech/tags/astro-ph-ga.md>), [astro-ph-im](<https://devfeed.tech/tags/astro-ph-im.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [classifier](<https://devfeed.tech/tags/classifier.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [framework](<https://devfeed.tech/tags/framework.md>), [galaxy](<https://devfeed.tech/tags/galaxy.md>), [gce](<https://devfeed.tech/tags/gce.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [ia](<https://devfeed.tech/tags/ia.md>), [imf](<https://devfeed.tech/tags/imf.md>), [independent](<https://devfeed.tech/tags/independent.md>), [inference](<https://devfeed.tech/tags/inference.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [physics-data-an](<https://devfeed.tech/tags/physics-data-an.md>), [yield](<https://devfeed.tech/tags/yield.md>)

### AI overview

Compass combines amortized Bayesian model comparison and parameter inference for Galactic chemical evolution models with intractable likelihoods. A transformer-based conditional score diffusion model learns posterior and likelihood from forward simulations, allowing yield prescriptions to be ranked alongside parameter estimation. In controlled tests, it identifies the generating yield set and recovers parameters at accuracy comparable to existing simulation-based inference methods, with computational cost reduced by orders of magnitude. Applied to Milky Way data, it favors a hybrid NuGrid + IllustrisTNG prescription, independently confirmed by a classifier. The inferred IMF slope is −2.65 ± 0.02 and the SN Ia normalization is log(N_Ia) = −2.81 ± 0.02; the IMF slope varies by about 0.3 across the top-ranked yield sets. The abstract also reports that attention highlights physically meaningful abundance signatures as drivers of the inferred parameters.

### Source excerpt

arXiv:2611.21175v1 Announce Type: cross Abstract: Galactic Chemical Evolution (GCE) models connect stellar nucleosynthesis to observed abundance patterns, but their interpretation requires simultaneous parameter inference and comparison between competing stellar yield prescriptions. This remains challenging due to intractable likelihoods and growing dataset complexity. Using compass, a novel simulation-based inference framework, we perform, for the first time, amortized joint Bayesian model comparison and parameter inference for GCE models with intractable likelihoods within a unified architecture. Compass employs a conditional score-based diffusion model with a transformer architecture to learn both the posterior and likelihood directly from forward simulations, enabling efficient inference and likelihood-based ranking of competing nucleosynthetic yield sets. In controlled tests, compass identifies the generating yield set and recovers ground-truth parameters with accuracy comparable to existing simulation-based inference (SBI) methods, while reducing computational cost by orders of magnitude. Applied to Milky Way data, the framework favors a hybrid NuGrid + IllustrisTNG yield prescription, confirmed by an independent classifier, and infers an initial mass function slope of $\alpha_{\rm IMF} = -2.65 \pm 0.02$ and a SN$_{\rm Ia}$ normalization of $\log(N_{\rm Ia}) = -2.81 \pm 0.02$, with a systematic spread of $\approx 0.3$ in $\alpha_{\rm IMF}$ between the best-ranked yield sets, consistent with independent constraints. The attention mechanism confirms that the inferred parameters are driven by physically meaningful abundance signatures. By unifying amortized parameter inference, Bayesian model comparison, and interpretability in a single probabilistic framework, compass provides a scalable approach for precision chemical evolution studies with current and forthcoming spectroscopic surveys.

## Role-Separated Diffusion Transformers: Global Rule Isolation and Deterministic Sampling Manifold Compression

DevFeed: [Role-Separated Diffusion Transformers: Global Rule Isolation and Deterministic Sampling Manifold Compression](<https://devfeed.tech/articles/role-separated-diffusion-transformers-global-rule-isolation-and-deterministic-sampling-manifold-compression-137375.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19009>)

Author: Zhiguang Liu, Yi Shang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [compression](<https://devfeed.tech/tags/compression.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [image](<https://devfeed.tech/tags/image.md>), [isolation](<https://devfeed.tech/tags/isolation.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

RoSeDiT separates the global conditioning rule from image tokens by adding an "adaLN listening" mechanism, so adaLN follows a controller channel; the abstract says a controller channel alone is insufficient when class and noise conditioning already use adaLN. Under the same SiT protocol and parameter counts, RoSeDiT reportedly beats SiT at all four scales after 400K steps. RoSeDiT-XL/2 reaches FID 7.98 at 1.4M steps versus SiT-XL/2's 8.3 at 7M steps; with the two-stage Deterministic Sampling Manifold Compression schedule, it reports FID 5.84 at 2.4M steps, compared with SiT + REPA's 5.9 at 4M. The authors also report large FID differences despite similar denoising loss, suggesting that loss and image-manifold quality can be misaligned.

### Source excerpt

arXiv:2611.19009v1 Announce Type: cross Abstract: We present Role-Separated Diffusion Transformers (RoSeDiT), a new diffusion transformer architecture that isolates the global rule from the image tokens. RoSeViT showed in visual reasoning that a vanilla ViT spreads a shared rule over all $L$ tokens at an entropy of order $Ld$, whereas an enforced controller route reduces it to order $d$. In class-conditional DiT and SiT, the class and the noise level are explicitly conditioned by adaLN, so a RoSeViT-style controller channel on the ViT side is insufficient to establish the full role separation. By introducing the adaLN listening mechanism, which makes adaLN compute under the controller's command, RoSeDiT brings the entropy theorem and role separation into the latent diffusion domain. Under the same SiT training and evaluation protocol at the same level of parameter count, RoSeDiT surpasses SiT at all four model scales at 400K steps, and RoSeDiT-L/2 even surpasses SiT-XL/2's 17.2 FID with 15.34 and 32% fewer parameters. RoSeDiT-XL/2 surpasses SiT-XL/2's 7M-step FID of 8.3 at 1.4M steps with 7.98, using one-fifth as many training steps. We further observe that models with the same denoising loss can have a large FID gap, which indicates a misalignment between the denoising objective and the image manifold. We formalize this phenomenon into Deterministic Sampling Manifold Compression (DSMC), a two-stage training schedule with which RoSeDiT-XL/2 surpasses SiT + REPA's 4M-step FID of 5.9 at 2.4M steps with 5.84.

## Beyond Alignment: Supervising the Generator for EEG-to-Image Reconstruction

DevFeed: [Beyond Alignment: Supervising the Generator for EEG-to-Image Reconstruction](<https://devfeed.tech/articles/beyond-alignment-supervising-the-generator-for-eeg-to-image-reconstruction-137421.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19232>)

Author: Sangmin Jo, Wootaek Jeong, Heung-Il Suk

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Image processing](<https://devfeed.tech/topics/image-processing.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [brain](<https://devfeed.tech/tags/brain.md>), [condition](<https://devfeed.tech/tags/condition.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [eeg](<https://devfeed.tech/tags/eeg.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [features](<https://devfeed.tech/tags/features.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generator](<https://devfeed.tech/tags/generator.md>), [generators](<https://devfeed.tech/tags/generators.md>), [image](<https://devfeed.tech/tags/image.md>), [images](<https://devfeed.tech/tags/images.md>), [inference](<https://devfeed.tech/tags/inference.md>), [neural](<https://devfeed.tech/tags/neural.md>), [process](<https://devfeed.tech/tags/process.md>), [quality](<https://devfeed.tech/tags/quality.md>)

### AI overview

Stronger EEG-to-vision alignment alone did not yield comparable gains in image reconstruction, and EEG-conditioned generators struggled to follow the fixed targets used by Representation Alignment (REPA). The paper introduces generation-derived supervision (GDS): an image-conditioned teacher maps static image-encoder features into a diffusion transformer's internal representation space, and an EEG-conditioned student tracks those dynamic targets during denoising. The teacher is discarded at inference. On THINGS-EEG and THINGS-MEG, the method reportedly outperforms conditioning-centric baselines in reconstruction fidelity. The supplied text is an abstract, so it does not provide implementation details or quantitative results.

### Source excerpt

arXiv:2611.19232v1 Announce Type: cross Abstract: Visual brain decoding translates neural activity into human-interpretable images. While recent EEG-to-image reconstruction methods rely on aligning EEG signals with pretrained visual embeddings to condition an image generator, we empirically demonstrate that stronger cross-modal alignment do not translate into comparable gains in reconstruction quality. This indicates that improving the input condition alone is insufficient; the generative process itself requires explicit supervision. Although Representation Alignment (REPA) offers a solution by guiding diffusion transformers with pretrained vision encoder features, we find that EEG-conditioned generators struggle to follow these fixed targets. To address this mismatch, we introduce generation-derived supervision (GDS): an image-conditioned teacher that translates static image-encoder features into the internal representation space of a diffusion transformer. Our EEG-conditioned student generator closely tracks these dynamic targets during denoising, discarding the teacher at inference. Evaluated on THINGS-EEG and THINGS-MEG, our approach consistently outperforms conditioning-centric baselines, achieving state-of-the-art reconstruction fidelity.

## High-quality cost-effective AI image generation on Amazon ECS

DevFeed: [High-quality cost-effective AI image generation on Amazon ECS](<https://devfeed.tech/articles/high-quality-cost-effective-ai-image-generation-on-amazon-ecs-136885.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/containers/high-quality-cost-effective-ai-image-generation-on-amazon-ecs/>)

Author: Nick Biso

Published: 2026-11-17T00:19:06Z

Content type: article

Language: en

Sources: [Containers](<https://devfeed.tech/sources/containers.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai-image-generation](<https://devfeed.tech/tags/ai-image-generation.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [amazon-elastic-container-service](<https://devfeed.tech/tags/amazon-elastic-container-service.md>), [auto-scaling](<https://devfeed.tech/tags/auto-scaling.md>), [ci-cd](<https://devfeed.tech/tags/ci-cd.md>), [cost](<https://devfeed.tech/tags/cost.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [ecs](<https://devfeed.tech/tags/ecs.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>)

### AI overview

The described Amazon ECS service runs Z-Image-Turbo on AWS Inferentia2, with one service per image resolution and one request at a time per instance because inference saturates the accelerator. Its key deployment mechanism is compiling model artifacts once in CI/CD, caching them on Amazon EFS, and loading them into containers at startup; the post reports this cuts startup to about two minutes from over 30 minutes and avoids recompilation when changes do not affect the model graph. The excerpt estimates $0.003 per 512px image and $0.011 per 1024px image at $1.97/hour for an inf2.8xlarge in us-east-1, while noting costs vary with utilization and other AWS resources still incur charges. It also describes resolution-based routing and streaming intermediate image frames over WebSockets. Deployment requires AWS quotas for Inf2 instances, CDK bootstrap, and listed local tools; first deployment takes about 45 minutes, including compilation.

### Source excerpt

Learn how to run Z-Image-Turbo, a 6-billion-parameter diffusion transformer, on AWS Inferentia2 and Amazon ECS to generate high-quality images at a fraction of a cent each, with auto-scaling, arbitrary resolution support, and a complete CI/CD pipeline.

## Comparative Analysis of 3D Convolutional and 2.5D Slice-Conditioned U-Net Architectures for MRI Super-Resolution via Elucidated Diffusion Models

DevFeed: [Comparative Analysis of 3D Convolutional and 2.5D Slice-Conditioned U-Net Architectures for MRI Super-Resolution via Elucidated Diffusion Models](<https://devfeed.tech/articles/comparative-analysis-of-3d-convolutional-and-2-5d-slice-conditioned-u-net-architectures-for-mri-super-resolution-via-elucidated-diffusion-models-134968.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.14667>)

Author: Hendrik Chiche, Eshaan Gulati, Ludovic Corcos, Logan Rouge

Published: 2026-11-16T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [2-5d](<https://devfeed.tech/tags/2-5d.md>), [3d](<https://devfeed.tech/tags/3d.md>), [3d-model](<https://devfeed.tech/tags/3d-model.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [acquisitions](<https://devfeed.tech/tags/acquisitions.md>), [added](<https://devfeed.tech/tags/added.md>), [all](<https://devfeed.tech/tags/all.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [best](<https://devfeed.tech/tags/best.md>), [better](<https://devfeed.tech/tags/better.md>), [brain](<https://devfeed.tech/tags/brain.md>), [changes](<https://devfeed.tech/tags/changes.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [models](<https://devfeed.tech/tags/models.md>), [slice](<https://devfeed.tech/tags/slice.md>), [super-resolution](<https://devfeed.tech/tags/super-resolution.md>)

### AI overview

The study finds no overall winner between 3D and 2.5D slice-conditioned U-Nets for 2x in-plane brain MRI super-resolution with an elucidated diffusion model. Trained on 500 NKI/FOMO60K subjects and evaluated on held-out data, they tie under the training degradation (37.72 vs. 37.70 dB PSNR), each about 0.2 dB ahead of EDSR. Results depend on the test setting: EDSR matches the diffusion models under k-space truncation, the 2.5D model is most accurate with added noise, and the 3D model leads on 70 external IXI subjects, by 0.4 dB over the best baseline and 0.9 dB over 2.5D in foreground PSNR. The evidence supports context-dependent performance rather than a uniformly superior architecture.

### Source excerpt

arXiv:2603.14667v2 Announce Type: replace-cross Abstract: Magnetic resonance imaging (MRI) super-resolution (SR) methods that computationally enhance low-resolution acquisitions offer a software alternative to higher-field scanners. We study an elucidated diffusion model (EDM) framework for 2x in-plane brain MRI SR and compare two U-Net backbones: (i) a 3D convolutional U-Net that processes volumetric patches and (ii) a 2.5D slice-conditioned U-Net that super-resolves each slice while conditioning on its two neighbouring slices. All models are trained from scratch to convergence on 500 subjects of the NKI cohort of FOMO60K, with a separate 6-subject validation set for checkpoint selection, and evaluated on 70 held-out subjects with subject-level statistics against baselines (EDSR, Swin2SR, Res-SRDiff) trained on the same data. Neither architecture is better overall. On the degradation used in training the two are tied (37.72 dB PSNR for the 3D model and 37.70 dB for the 2.5D model), 0.2 dB ahead of EDSR. When the test degradation changes to k-space truncation, EDSR matches the diffusion models, and with added noise the 2.5D model is the most accurate method. When the test cohort changes to 70 subjects of an external dataset (IXI), the 3D model is the most accurate, 0.4 dB ahead of the best baseline and 0.9 dB ahead of the 2.5D model in foreground PSNR.

## JaRS: Contrast-Free Localization of Memorized Regions in Diffusion Models via Jacobian Row Sums

DevFeed: [JaRS: Contrast-Free Localization of Memorized Regions in Diffusion Models via Jacobian Row Sums](<https://devfeed.tech/articles/jars-contrast-free-localization-of-memorized-regions-in-diffusion-models-via-jacobian-row-sums-135858.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18024>)

Author: Saisuchith Mahajan, Aditey Nandan, Rohit Singh, Nishanth Shetty, Chandra Sekhar Seelamantula

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [stable-diffusion](<https://devfeed.tech/topics/stable-diffusion.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [localization](<https://devfeed.tech/tags/localization.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [regions](<https://devfeed.tech/tags/regions.md>)

### AI overview

JaRS localizes memorized image regions in diffusion models without the reference map required by contrast-based methods. Those methods use denoiser-Jacobian diagonal entries, but small values can indicate either memorization or averaging across neighboring pixels in smooth or predictable regions; resolving that ambiguity requires a second evaluation with a null prompt or earlier checkpoint. Jacobian row sums distinguish memorization from local averaging without that reference, including for unconditional models. JaRS+ pairs the statistic with an image-level detection score computed at an early denoising step. The abstract reports that JaRS+ outperforms contrast-based localization methods on Stable Diffusion v1.4 and v2.1, Realistic Vision v5.1, and an unconditional diffusion Transformer.

### Source excerpt

arXiv:2611.18024v1 Announce Type: cross Abstract: Diffusion models sometimes memorize and reproduce regions of their training images, raising privacy and copyright concerns. We aim to localize these regions. Existing contrast-based methods use the diagonal of the denoiser Jacobian, which reflects the posterior variance of each pixel. However, a small diagonal entry is ambiguous: it signals either memorization or averaging over many neighboring pixels in smooth or predictable regions. To resolve this ambiguity, these methods subtract a reference map, which requires a second evaluation with a null prompt or an earlier checkpoint. Unconditional models have no null prompt, and earlier checkpoints are unavailable in general. We show that Jacobian row sums (JaRS), unlike the diagonal, separate memorization from local averaging without a reference map. Pairing JaRS with an image-level detection score that uses the same statistic at an early denoising step yields JaRS+. Across Stable Diffusion v1.4 and v2.1, Realistic Vision v5.1, and an unconditional diffusion Transformer (DiT), JaRS+ outperforms contrast-based localization methods.

## Ultra Fast PDE Solving via Physics Guided Few-step Diffusion

DevFeed: [Ultra Fast PDE Solving via Physics Guided Few-step Diffusion](<https://devfeed.tech/articles/ultra-fast-pde-solving-via-physics-guided-few-step-diffusion-132750.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.03627>)

Author: Cindy Xiangrui Kong, Yueqi Wang, Haoyang Zheng, Weijian Luo, Guang Lin

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [fast](<https://devfeed.tech/tags/fast.md>), [generative-models](<https://devfeed.tech/tags/generative-models.md>), [inference](<https://devfeed.tech/tags/inference.md>), [partial-differential-equations](<https://devfeed.tech/tags/partial-differential-equations.md>), [physics](<https://devfeed.tech/tags/physics.md>)

### AI overview

Phys-Instruct distills a pre-trained diffusion PDE solver into a few-step generator, using PDE residual constraints and distillation guidance to improve physical consistency while reducing sampling cost. Across five PDE benchmarks, it reportedly runs orders of magnitude faster than corresponding 200-step EDM teachers and reduces PDE error by factors of 2.4 to 15.5. The unconditional student is also described as a compact prior for downstream conditional inference. These claims are based on the supplied abstract.

### Source excerpt

arXiv:2602.03627v2 Announce Type: replace Abstract: Diffusion-based models have demonstrated impressive accuracy and generalization in solving partial differential equations (PDEs). However, their practical utility still suffers from high sampling costs and insufficient physical consistency, mainly due to many-step iterative sampling and insufficient physics guidance. To address these issues, we propose \emph{Phys-Instruct}, a physics-guided distillation framework that compresses a pre-trained diffusion PDE solver into a few-step generator while explicitly enforcing PDE consistency through distillation guidance. We reformulate the training objective as a constrained problem with PDE residual, and derive a score-function identity that leads to a practical physics-guided generator update. Across five PDE benchmarks, Phys-Instruct achieves orders-of-magnitude faster inference while reducing the PDE Error by factors of $2.4$ to $15.5$, compared with the corresponding 200-step EDM teachers. Moreover, the resulting unconditional student model functions as a compact prior, enabling efficient and physically consistent inference for various downstream conditional tasks. Our results indicate that Phys-Instruct is an effective and efficient framework for ultra-fast PDE solving powered by generative models.

## RAIN: Region-Aware Inversion Network for Semantic Watermark Extraction

DevFeed: [RAIN: Region-Aware Inversion Network for Semantic Watermark Extraction](<https://devfeed.tech/articles/rain-region-aware-inversion-network-for-semantic-watermark-extraction-131967.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.14856>)

Author: Zilai Li

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [GPU](<https://devfeed.tech/topics/gpu.md>)

Tags: [conditional](<https://devfeed.tech/tags/conditional.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [github](<https://devfeed.tech/tags/github.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [noise](<https://devfeed.tech/tags/noise.md>), [one](<https://devfeed.tech/tags/one.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [watermark](<https://devfeed.tech/tags/watermark.md>)

### AI overview

RAIN is a prompt-free, one-step method for extracting Gaussian-Shading semantic watermarks without iterative diffusion inversion or repeated evaluation of a diffusion-scale U-Net. It estimates a useful noise statistic near the high-SNR image endpoint, where the method argues that Gaussian Shading only requires the recovered latent to fall within the correct watermark decision region. RAIN splits endpoint recovery into an image-like anchor and a noise-oriented residual to make greater use of GPU parallelism. The abstract reports lower noise-extraction cost than OSI and FARI, but provides no quantitative comparison in the supplied excerpt.

### Source excerpt

arXiv:2609.14856v3 Announce Type: replace-cross Abstract: Semantic watermarks for diffusion models embed ownership information into the generative process while preserving perceptual quality, but Gaussian-Shading extraction conventionally requires multi-step diffusion inversion to recover the initial noise. Recent one-step methods show that this cost can be reduced substantially. We study this problem through extended flow matching and conditional regression. The key observation is that, near the high-SNR image endpoint, recovering a useful noise statistic given by the first-step output of the extended flow matching in the high-SNR regime is much simpler than reconstructing the full inverse trajectory, and Gaussian Shading only requires the recovered latent to remain in the correct watermark decision region. Based on this observation, we propose a lightweight, prompt-free extractor that decomposes endpoint recovery into an image-like anchor and a noise-oriented residual, which increases the capability of the model to utilize GPU parallel computation. The resulting method avoids iterative inversion and repeated evaluation of a diffusion-scale U-Net, providing an efficient one-step extraction pipeline with a concise theoretical interpretation. The computational cost of extracting noise is lower than that of both OSI and FARI. The github repo is there: https://github.com/TheLovesOfLadyPurple/RAIN-lightweight-NN-for-one-step-semantic-watermark-extraction

[Next page](<https://devfeed.tech/topics/diffusion-transformers.md?cursor=WyIyMDI2LTExLTEzVDA1OjAwOjAwKzAwOjAwIiwgImE3YjU3MzYxLThlY2ItNGIxNC1iZGI1LWE2ODUwMTI3MjhlOCJd>)