# DeltaForge: A Post-hoc Mixture-of-Experts VLA for Multi-Domain Autonomous Driving

DevFeed: [DeltaForge: A Post-hoc Mixture-of-Experts VLA for Multi-Domain Autonomous Driving](<https://devfeed.tech/articles/deltaforge-a-post-hoc-mixture-of-experts-vla-for-multi-domain-autonomous-driving-125459.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11856>)

Author: Bingzhang Wang, Mo Poorsartep, Yasaman Esfandiari

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [approach](<https://devfeed.tech/tags/approach.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [autonomous-driving](<https://devfeed.tech/tags/autonomous-driving.md>), [base](<https://devfeed.tech/tags/base.md>), [cars](<https://devfeed.tech/tags/cars.md>), [clip](<https://devfeed.tech/tags/clip.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [domain](<https://devfeed.tech/tags/domain.md>), [driving](<https://devfeed.tech/tags/driving.md>), [experts](<https://devfeed.tech/tags/experts.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [post](<https://devfeed.tech/tags/post.md>), [vla](<https://devfeed.tech/tags/vla.md>)

## AI overview

DeltaForge combines independently fine-tuned driving VLA experts using their weights and a small proxy dataset. It applies joint SVD to separate shared update structure from small domain-specific components, then trains two cooperating clip-level blind routers with proxy-set soft labels: one for VLM reasoning experts and one for flow-matching dynamics experts. Across six urban, off-road and highway domains involving cars, trucks and ATVs, the abstract reports accuracy near that of per-domain experts and better results than merging, routing, centralized and federated baselines, at close to single-model parameter and compute cost. A new domain can be added by projecting its update onto the shared basis, without end-to-end retraining; the model also generalizes to domain combinations not explicitly trained. These are abstract-level claims; no detailed measurements are provided here.

## Source excerpt

arXiv:2611.11856v1 Announce Type: new Abstract: Fleets of driving vision-language-action (VLA) models are developed one domain at a time: each operator finetunes a shared base model on private data, producing specialized experts that are costly to consolidate into a single model. Weight merging loses accuracy as heterogeneous domains accumulate, and dense mixture-of-experts routing is prohibitively expensive to combine them. We present DeltaForge, to our knowledge the first post-hoc mixture-of-experts VLA for driving, which consolidates N independently finetuned per-domain models into a single multi-domain model from their weights and a small proxy set alone. When different operators finetune the same base model, their weight updates end up sharing a lot of structure. We exploit this by decomposing all the updates together using singular value decomposition (SVD), splitting each one into a shared part (the same across all domains) and a tiny domain-specific part. The router then only needs to work with those tiny domain-specific pieces, not the full expert weights, which is what keeps the approach efficient. Then two cooperating clip-level blind routers, one over the VLM reasoning experts and one over the flow-matching dynamics experts, are trained from soft labels on the proxy dataset. Across six diverse domains spanning urban, off-road, and highway driving on cars, trucks, and all-terrain vehicles (ATVs), DeltaForge approaches the accuracy of the per-domain experts and surpasses every merging, routing, centralized, and federated baseline at close to single-model parameter and compute cost. Adding a new domain requires only projecting its weight update onto the shared basis with no end-to-end retraining needed. The resulting model also generalizes to domain combinations it was never explicitly trained on.