# asr

Automatic speech recognition is the task of transcribing audio into text.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Robustness of Text-Only Adapted Speech LLMs for Spoken Semantic Parsing

DevFeed: [Robustness of Text-Only Adapted Speech LLMs for Spoken Semantic Parsing](<https://devfeed.tech/articles/robustness-of-text-only-adapted-speech-llms-for-spoken-semantic-parsing-142533.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.21453>)

Author: Youngwon Choi, Donghyuk Jung, Hyeonyu Kim, Taeyoun Kwon, Myeongkyun Cho, Moon Hwan Kim

Published: 2026-11-19T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [asr](<https://devfeed.tech/topics/asr.md>), [function calling](<https://devfeed.tech/topics/function-calling.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [command](<https://devfeed.tech/tags/command.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [function-calling](<https://devfeed.tech/tags/function-calling.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llms](<https://devfeed.tech/tags/llms.md>), [noise](<https://devfeed.tech/tags/noise.md>), [parsing](<https://devfeed.tech/tags/parsing.md>), [performance](<https://devfeed.tech/tags/performance.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text](<https://devfeed.tech/tags/text.md>)

### AI overview

Across matched text-only supervision, Speech LLMs generally outperform ASR-LLM cascades on spoken command understanding, intent prediction, and function calling, especially under severe babble noise. The study compares 3B and 7B models on three datasets using clean speech and five noise SNR levels. Gains on fine-grained entity prediction are less consistent. Speech LLMs also have lower average inference time per sample at both model scales. These findings are limited to the evaluated tasks, datasets, and conditions.

### Source excerpt

arXiv:2606.21453v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate structured outputs from natural language instructions, including semantic frames and function calls. When the input is speech, this task becomes spoken semantic parsing, which can be performed using either Speech LLMs or ASR-LLM cascades. Although both approaches can be adapted with text-only task supervision, their relative robustness to acoustic degradation remains unclear. We compare these approaches under matched text-form supervision using 3B and 7B models across three datasets, evaluated on clean speech and babble noise at five SNR levels. Results show that Speech LLMs generally achieve better performance in command understanding, intent prediction, and function calling, particularly under severe noise, while improvements in fine-grained entity prediction are less consistent. An additional evaluation shows that Speech LLMs achieve lower average inference time per sample than ASR-LLM cascades at both model scales.

## Reson8 releases Rumble-2-Flash speech model as a free open-weights download

DevFeed: [Reson8 releases Rumble-2-Flash speech model as a free open-weights download](<https://devfeed.tech/articles/reson8-releases-rumble-2-flash-speech-model-as-a-free-open-weights-download-140709.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/18/reson8-releases-rumble-2-flash-speech-model-as-a-free-open-weights-download/>)

Author: Duncan Riley

Published: 2026-11-18T14:00:57Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [releases](<https://devfeed.tech/topics/releases.md>), [Whisper](<https://devfeed.tech/topics/whisper.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-act](<https://devfeed.tech/tags/ai-act.md>), [amsterdam](<https://devfeed.tech/tags/amsterdam.md>), [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [balderton-capital](<https://devfeed.tech/tags/balderton-capital.md>), [developers](<https://devfeed.tech/tags/developers.md>), [download](<https://devfeed.tech/tags/download.md>), [european-union](<https://devfeed.tech/tags/european-union.md>), [free](<https://devfeed.tech/tags/free.md>), [general-data-protection-regulation](<https://devfeed.tech/tags/general-data-protection-regulation.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [iso-27001](<https://devfeed.tech/tags/iso-27001.md>), [james-wise](<https://devfeed.tech/tags/james-wise.md>), [jarno-verhagen](<https://devfeed.tech/tags/jarno-verhagen.md>), [medical-transcription](<https://devfeed.tech/tags/medical-transcription.md>), [model](<https://devfeed.tech/tags/model.md>), [netherlands](<https://devfeed.tech/tags/netherlands.md>), [news](<https://devfeed.tech/tags/news.md>), [np-hard-ventures](<https://devfeed.tech/tags/np-hard-ventures.md>), [open-asr-leaderboard](<https://devfeed.tech/tags/open-asr-leaderboard.md>), [open-weights](<https://devfeed.tech/tags/open-weights.md>), [openai](<https://devfeed.tech/tags/openai.md>), [pre-seed-funding](<https://devfeed.tech/tags/pre-seed-funding.md>), [raoul-ritter](<https://devfeed.tech/tags/raoul-ritter.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [reson8](<https://devfeed.tech/tags/reson8.md>), [resonate-labs](<https://devfeed.tech/tags/resonate-labs.md>), [rumble-2](<https://devfeed.tech/tags/rumble-2.md>), [rumble-2-flash](<https://devfeed.tech/tags/rumble-2-flash.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-models](<https://devfeed.tech/tags/speech-models.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [startup](<https://devfeed.tech/tags/startup.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [thomas-kluiters](<https://devfeed.tech/tags/thomas-kluiters.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>), [whisper-large-v3](<https://devfeed.tech/tags/whisper-large-v3.md>)

### AI overview

Reson8 has released Rumble-2-Flash as a free open-weights download under Apache 2.0. The 500-million-parameter model is designed for fast transcription of prerecorded audio and supports more than 40 languages. In Reson8's reported tests, it averaged 7.09% word error rate across seven languages and 17.6% across 10 lower-resource languages; Reson8 reports Whisper Large V3 at 39.82% on the latter test. These are company-reported results. Custom vocabulary support for up to 50,000 terms remains exclusive to the larger Rumble-2 model. Rumble-2-Flash is also available through Reson8's API, starting at EUR 0.01 per hour of audio.

### Source excerpt

Amsterdam-based speech recognition startup Resonate Labs B.V. today released its Rumble-2-Flash transcription model as a free download with open weights. Designed for fast transcription of prerecorded audio, the 500 million-parameter model supports more than 40 languages. Reson8 is pitching the release at developers and researchers who need speech recognition built for languages other than English. According [...] The post Reson8 releases Rumble-2-Flash speech model as a free open-weights download appeared first on SiliconANGLE.

## Marimba: A System One Model for Speech

DevFeed: [Marimba: A System One Model for Speech](<https://devfeed.tech/articles/marimba-a-system-one-model-for-speech-139871.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18330>)

Author: Solano Amadori

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [detection](<https://devfeed.tech/tags/detection.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [language](<https://devfeed.tech/tags/language.md>), [lexical](<https://devfeed.tech/tags/lexical.md>), [meetings](<https://devfeed.tech/tags/meetings.md>), [model](<https://devfeed.tech/tags/model.md>), [personal-information](<https://devfeed.tech/tags/personal-information.md>), [pii](<https://devfeed.tech/tags/pii.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [speaker](<https://devfeed.tech/tags/speaker.md>), [speech](<https://devfeed.tech/tags/speech.md>), [window](<https://devfeed.tech/tags/window.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

Marimba uses frozen Qwen3-ASR 0.6B features to make typed speech decisions: zero-shot transcript scoring for lexical questions, a question-conditioned head for non-lexical questions, a separate layer-18 turn-end readout, and the ASR model's language scores for language identification. Results vary by task and dataset: decoder AUROC is 0.80-0.92 on private conversational tests and 0.92-0.97 on public read speech; speaker-change readouts exceed an ECAPA baseline, and speaker counts outperform pyannote 3.1. Turn-end AUROC is 0.930, below Smart Turn v3.2's 0.982 on its own test set, while ICSI next-speaker results are close (0.716 vs. 0.712) with a paired confidence interval including zero. PII detection depends on a private curated pipeline applied to greedy transcripts; its item-level F1 is 0.904 on synthetic Brazilian Portuguese speech and 1.000 on reference transcripts.

### Source excerpt

arXiv:2611.18330v1 Announce Type: cross Abstract: Speech applications require decisions about term presence, turn completion, and language within an audio window. We present Marimba, an interface for typed speech decisions built on frozen Qwen3-ASR 0.6B features. Lexical questions use zero-shot scoring of ASR transcripts; non-lexical questions use a question-conditioned head and a separate layer-18 turn-end readout. Language identification uses the ASR model's own language scores. On private conversational test sets, decoder AUROC is 0.80-0.92; on public read speech it is 0.92-0.97. Speech Commands four-way accuracy is 0.93; binary lexical ECE is at most 0.08. Personal-information detection, which passes the greedy transcript to a private, curated PII pipeline, reaches item-level F1 0.904 on synthetic Brazilian Portuguese speech (1.000 on reference transcripts). Layer-12 speaker readouts detect speaker change with AUROC 0.966-0.980 on AMI and VoxConverse, above an ECAPA baseline, and count speakers (0, 1, 2, or 3+) more accurately than pyannote 3.1. The turn-end readout reaches AUROC 0.930 versus Smart Turn v3.2 at 0.982 on Smart Turn's own test set. On a subset of ICSI meetings labeled by next speaker, AUROC is 0.716 versus 0.712; the paired confidence interval includes zero.

## Does the Voice Matter? Evaluating Frustration Detection on Deployed Clinical Voice-Agent Calls

DevFeed: [Does the Voice Matter? Evaluating Frustration Detection on Deployed Clinical Voice-Agent Calls](<https://devfeed.tech/articles/does-the-voice-matter-evaluating-frustration-detection-on-deployed-clinical-voice-agent-calls-137644.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21001>)

Author: Annika Michael, Zachary Ellis, Yuxiang Zhou, Samuel Thio, Yajie Vera He, Spencer Hazel, Jared Joselowitz

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [asr](<https://devfeed.tech/topics/asr.md>), [speech-to-speech](<https://devfeed.tech/topics/speech-to-speech.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [detection](<https://devfeed.tech/tags/detection.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [transcript](<https://devfeed.tech/tags/transcript.md>), [triage](<https://devfeed.tech/tags/triage.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>)

### AI overview

In 30 deployed clinical voice-agent calls, patient turns were often too short for transcripts alone to reliably detect frustration: about 59% were one or two words, and transcript-based detection flagged only 3 or 4 of 39 frustrated one-word turns. Adding audio did not consistently improve results. Two Gemini 2.5 models gained 3.0 percentage points when prompted to attend to vocal delivery, but no model showed a statistically clear audio advantage when given the same prompt for both inputs; some models performed worse with audio under input-specific prompts. A WavLM probe reached 71.6% overall accuracy yet missed every frustrated one-word turn in 24 added calls. The findings indicate that a short transcript cannot rule out frustration, while audio benefits depend on the model and prompt.

### Source excerpt

arXiv:2611.21001v1 Announce Type: cross Abstract: Autonomous voice agents now conduct routine clinical follow-up calls, so some automated triage is unavoidable. A patient becoming frustrated is one thing that triage should catch. These systems typically read only the transcript the speech-to-text stage produces, not the voice, and whether that is enough is largely untested on real clinical speech. We study per-turn frustration detection on real patient calls from a deployed clinical voice agent: 150 turns from six calls labelled by three annotators, with a length-matched acted corpus as a control, and 714 turns from 30 calls. Patient turns are overwhelmingly short: across the deployed calls, roughly 59% are one or two words, against 13% in the full acted corpus, and a corpus accuracy gap remains even after the acted set is length-matched. On one-word turns, transcript-only detection does not exclude chance. Across five audio-capable language models, the transcript flags 3 or 4 of the 39 frustrated one-word turns in the 30 calls, and a classifier given only the word count reaches 57.3%. Whether audio helps depends on the model and on the instruction. Asked to attend to delivery, two Gemini 2.5 models gain 3.0 pp over the transcript across 30 calls (-1.0 to 7.5 pp). Of the turns they confidently clear, 6.9% were in fact frustrated for audio against 13.0% for the transcript, a difference of 6.1 pp (2.2 to 10.4 pp). Across the 30 calls, given one prompt identical for both inputs, no model's audio-text interval excludes zero. Under the input-specific prompts, a newer Gemini model and two smaller open models do worse with audio than with the transcript (-3.5, -8.6 and -10.6 pp). A WavLM probe trained on the clinical calls matches the best prompted model overall (71.6%) but misses every frustrated one-word turn in the 24 added calls. When a patient's turn is short, the transcript alone is not enough to rule out frustration.

## Decode Early, Verify Late: Streaming Offline Speech Recognizers Without Accuracy Loss

DevFeed: [Decode Early, Verify Late: Streaming Offline Speech Recognizers Without Accuracy Loss](<https://devfeed.tech/articles/decode-early-verify-late-streaming-offline-speech-recognizers-without-accuracy-loss-138984.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20064>)

Author: Doyeob Kim

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [Real-time AI interactions](<https://devfeed.tech/topics/real-time-ai-interactions.md>)

Tags: [1-5](<https://devfeed.tech/tags/1-5.md>), [22](<https://devfeed.tech/tags/22.md>), [a100](<https://devfeed.tech/tags/a100.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [after](<https://devfeed.tech/tags/after.md>), [agents](<https://devfeed.tech/tags/agents.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [asr](<https://devfeed.tech/tags/asr.md>), [audio](<https://devfeed.tech/tags/audio.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [complete](<https://devfeed.tech/tags/complete.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decode](<https://devfeed.tech/tags/decode.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [equals](<https://devfeed.tech/tags/equals.md>), [falls](<https://devfeed.tech/tags/falls.md>), [offline](<https://devfeed.tech/tags/offline.md>), [speech](<https://devfeed.tech/tags/speech.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [verify](<https://devfeed.tech/tags/verify.md>)

### AI overview

Decode Early, Verify Late (DEVL) streams offline ASR by showing partial transcripts as drafts, then verifying the latest draft against the complete audio in one forward pass when the turn ends. Under stated input and decoding conditions, its final output equals greedy offline decoding in exact arithmetic. Across five recognizers, 95-100% of finals were byte-identical to offline transcripts, with word error rate within 0.13 points; single-stream time to final fell 1.5-7.3x. For Qwen3-ASR-1.7B on one A100, median final latency was 22-26 ms after turn end at up to 64 streams, 2.7-3.9x sooner than its official streaming mode. The tested system met a 100 ms p99 target at up to 80 streams on Pipecat-STT and 64 on WildASR.

### Source excerpt

arXiv:2611.20064v1 Announce Type: new Abstract: Voice agents need accurate transcripts soon after a user stops speaking. We propose Decode Early, Verify Late (DEVL), a training-free framework for streaming inference with offline ASR models. The server keeps partial transcripts as drafts and verifies the latest against the complete audio in one forward pass at turn end. Under stated input and decoding conditions, the final equals the model's greedy offline output in exact arithmetic. We adapt DEVL to four model families. In our five-recognizer comparison, 95-100% of final transcripts are byte-identical to the offline transcripts, WER stays within 0.13 points of offline, and single-stream time to final falls 1.5-7.3x. Serving Qwen3-ASR-1.7B on one A100, DEVL returns finals a median 22-26 ms after turn end at up to 64 streams, 2.7-3.9x sooner than the official streaming mode. It is the only system we tested that meets a 100 ms p99 target, up to 80 streams on Pipecat-STT and 64 on WildASR.

## Latency, Speech Recognition and Turn-Taking in Voice Agents for Spanish Phone Negotiation: An Externally Measured Comparison of Pipecat, LiveKit and GPT-Live

DevFeed: [Latency, Speech Recognition and Turn-Taking in Voice Agents for Spanish Phone Negotiation: An Externally Measured Comparison of Pipecat, LiveKit and GPT-Live](<https://devfeed.tech/articles/latency-speech-recognition-and-turn-taking-in-voice-agents-for-spanish-phone-negotiation-an-externally-measured-comparison-of-pipecat-livekit-and-gpt-live-137399.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19118>)

Author: Jovan Popoca Moran, Eduardo de Avila-Armenta, Ra\\'ul Alejandro Aquino-Ibarra, Jorge Octavio Nicol\\'as-D\\'iaz, Umberto Leon-Dominguez

Published: 2026-11-17T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [livekit](<https://devfeed.tech/topics/livekit.md>), [asr](<https://devfeed.tech/topics/asr.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [gpt-live](<https://devfeed.tech/tags/gpt-live.md>), [latency](<https://devfeed.tech/tags/latency.md>), [livekit](<https://devfeed.tech/tags/livekit.md>), [negotiation](<https://devfeed.tech/tags/negotiation.md>), [phone](<https://devfeed.tech/tags/phone.md>), [pipecat](<https://devfeed.tech/tags/pipecat.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

In simulated Spanish freight-rate calls, GPT-Live had the lowest median listener-measured response latency (2,444.8 ms), but its 171.2 ms advantage over Pipecat was not statistically conclusive; LiveKit measured 3,090.0 ms. Tool calls added 0.9-1.1 seconds across all three architectures, and framework-reported latency understated listener wait. GPT-Live also had lower word and amount error rates than the two cascades. With default settings, Pipecat and LiveKit often treated an overlapping "ok" as the caller's answer, sometimes recording agreement; no such case occurred for GPT-Live in the reported calls. The study recommends measuring latency from the listener's side and adding safeguards for amounts and backchannels in cascades. Results come from 77 calls per architecture using recordings from one speaker without a telephone network, so they do not establish performance in live phone negotiations.

### Source excerpt

arXiv:2611.19118v1 Announce Type: cross Abstract: A voice agent that negotiates by phone must answer quickly, understand spoken amounts and cope with a caller who talks over it. We compared three architectures for such an agent, negotiating freight rates in Spanish: two cascades (speech recognition, language model and speech synthesis) orchestrated by Pipecat and by LiveKit Agents, and GPT-Live, an OpenAI speech model that listens and speaks without separate stages. A program played the carrier with the same 47 recordings of one speaker for all agents, without the telephone network, in 77 calls per architecture over five negotiation scenarios, and measured latency on the listener's side with a validated instrument. Median latency to the content response was 2,616.0 ms (Pipecat), 3,090.0 ms (LiveKit) and 2,444.8 ms (GPT-Live); the confidence interval of GPT-Live's 171.2 ms advantage over Pipecat includes zero. According to its own timestamps, GPT-Live began to answer 800 ms after the end of the line, close to the 0.8 s published by OpenAI; most of the remaining wait, about 1,367 ms and nearly constant, is the audio's round trip through OpenAI's service. A tool call added 0.9-1.1 s in every architecture, more than any difference between them, and the frameworks' self-reported latency fell short of the listener's wait by 9% (Pipecat) and 22% (LiveKit). Word error rates were low (6.03, 4.93 and 1.47%), but the error on amounts was 13.01% and 12.61% in the cascades, against 2.86% in GPT-Live. With default settings, Pipecat and LiveKit stopped at almost every backchannel and often took it as the carrier's answer: an "ok" spoken over the agent ended in a registered agreement in 3 of 15 Pipecat calls, 2 of 15 LiveKit calls and no GPT-Live call (descriptive counts, not tested). Latency should therefore be measured on the listener's side, and a cascade needs safeguards for spoken amounts and backchannels.

## FETTUCCINE: Fast and efficient brain-to-text decoding on mobile devices

DevFeed: [FETTUCCINE: Fast and efficient brain-to-text decoding on mobile devices](<https://devfeed.tech/articles/fettuccine-fast-and-efficient-brain-to-text-decoding-on-mobile-devices-135920.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18561>)

Author: Jonathan D. McCart, Pranav I. Deevi, Mehdi Azabou, Nanda H. Krishna, Caleb A. McKinney, Nicholas S. Card, Sergey D. Stavisky, Chethan Pandarinath

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [bci](<https://devfeed.tech/topics/bci.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [ai-on-mobile](<https://devfeed.tech/topics/ai-on-mobile.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [brain](<https://devfeed.tech/tags/brain.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [decode](<https://devfeed.tech/tags/decode.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [device](<https://devfeed.tech/tags/device.md>), [devices](<https://devfeed.tech/tags/devices.md>), [error](<https://devfeed.tech/tags/error.md>), [fast](<https://devfeed.tech/tags/fast.md>), [features](<https://devfeed.tech/tags/features.md>), [first](<https://devfeed.tech/tags/first.md>), [framework](<https://devfeed.tech/tags/framework.md>), [future](<https://devfeed.tech/tags/future.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [mobile-devices](<https://devfeed.tech/tags/mobile-devices.md>), [q-bio-nc](<https://devfeed.tech/tags/q-bio-nc.md>), [text](<https://devfeed.tech/tags/text.md>)

### AI overview

FETTUCCINE uses automatic speech recognition models for brain-to-text decoding, aiming to reduce the compute burden and variable inference latency associated with language-modeling stacks. On public data from one intracortical speech neuroprosthesis user, it outperformed prior end-to-end methods, with word error rates as low as 4.39%. The models ran on a commercial mobile device at up to 370 times real-time throughput, and fine-tuning extended usable performance to later days. The abstract also reports that gradient-based saliency maps identified relevant neural features consistent with physiological priors across ASR model families. These findings are based on the described user data; the excerpt does not establish performance across a broader population.

### Source excerpt

arXiv:2611.18561v1 Announce Type: cross Abstract: Recent advances in speech neuroprostheses have enabled new communication avenues for people who have lost the ability to speak due to neurological illness or injury. These systems can decode neural activity during attempted speech into text by relying on language priors to achieve high decoding accuracy. This performance currently comes at the cost of using a compute-intensive language modeling stack, presenting a major bottleneck for real-world use. Further, current language models incur highly variable and disruptive inference latencies, limiting communication throughput and preventing users from reliably participating in natural conversation. We introduce FETTUCCINE, an end-to-end brain-to-text framework that leverages the speed and portability of automatic speech recognition (ASR) models to address the aforementioned practical challenges, while achieving usable (<10%) word error rates (WER), efficient adaptation to neural non-stationarities, and reliable performance on mobile devices. When evaluated using publicly available data from an intracortical speech neuroprosthesis user, FETTUCCINE outperforms previous end-to-end methods, achieving as low as 4.39% WER. Most notably, our models successfully run on a commercially-available mobile device with throughput reaching up to 370x real-time, providing the first demonstration of brain-to-text decoding on a mobile device. We also show that our models can be successfully finetuned to extend usable performance to future days. Beyond performance, our approach enables localization of the neural features most relevant for decoding via gradient-based saliency maps, which we show align with well-established physiological priors across different ASR model families. Taken together, our results show that FETTUCCINE overcomes core infrastructural and computational barriers, yielding a new class of portable brain-to-text communication.

## Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot

DevFeed: [Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot](<https://devfeed.tech/articles/cost-optimizing-assemblyai-speech-to-text-inference-on-aws-with-amazon-ec2-spot-132652.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/cost-optimizing-assemblyai-speech-to-text-inference-on-aws-with-amazon-ec2-spot/>)

Author: Belair Albahagwi

Published: 2026-11-13T19:35:35Z

Content type: news

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [cloud cost management](<https://devfeed.tech/topics/cloud-cost-management.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [amazon-sqs](<https://devfeed.tech/tags/amazon-sqs.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [assemblyai](<https://devfeed.tech/tags/assemblyai.md>), [aws](<https://devfeed.tech/tags/aws.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cost-savings](<https://devfeed.tech/tags/cost-savings.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [ec2](<https://devfeed.tech/tags/ec2.md>), [ecs](<https://devfeed.tech/tags/ecs.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [regions](<https://devfeed.tech/tags/regions.md>), [retry](<https://devfeed.tech/tags/retry.md>), [scale](<https://devfeed.tech/tags/scale.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [spot](<https://devfeed.tech/tags/spot.md>), [sqs](<https://devfeed.tech/tags/sqs.md>)

### AI overview

AssemblyAI reports 70% lower inference compute costs than On-Demand pricing using GPU Spot Instances, while maintaining SLA commitments. Its Amazon ECS clusters combine an always-on On-Demand baseline with diversified Spot capacity across g4dn, g5, and g6 GPU families and Availability Zones. Amazon SQS decouples job submission from execution: tasks finish work within Spot's two-minute interruption notice or return unfinished jobs to the queue for another worker to retry. Custom images with model artifacts and containers take about two minutes to create, reducing recovery time. Independent deployments in each AWS Region keep audio within required geographic boundaries. The article says compute demand grew nearly fourfold year over year; interruptions may add processing time, and exhausted retries can be routed to a dead-letter queue for investigation.

### Source excerpt

Learn how AssemblyAI runs cost-optimized speech-to-text inference at scale on AWS, using Amazon ECS, Amazon EC2 Spot GPU instances, and an Amazon SQS-based retry architecture to achieve 70% cost savings while maintaining SLA commitments across European Regions.

## Speech Tokenizers are Vulnerable: Transferable Semantic Attack and Robust Tokenizer

DevFeed: [Speech Tokenizers are Vulnerable: Transferable Semantic Attack and Robust Tokenizer](<https://devfeed.tech/articles/speech-tokenizers-are-vulnerable-transferable-semantic-attack-and-robust-tokenizer-132067.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14929>)

Author: Zhisheng Zhang, Yifan Mi, Yixuan Zhou, Haiyun Li, Shengbo Cai, Dongfu Song, Ziang Xu, Jie Hao, Zhiyong Wu

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [asr](<https://devfeed.tech/tags/asr.md>), [attack](<https://devfeed.tech/tags/attack.md>), [audio](<https://devfeed.tech/tags/audio.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [based](<https://devfeed.tech/tags/based.md>), [black](<https://devfeed.tech/tags/black.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [box](<https://devfeed.tech/tags/box.md>), [building](<https://devfeed.tech/tags/building.md>), [can](<https://devfeed.tech/tags/can.md>), [cause](<https://devfeed.tech/tags/cause.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [robust](<https://devfeed.tech/tags/robust.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [speech](<https://devfeed.tech/tags/speech.md>), [tokenizers](<https://devfeed.tech/tags/tokenizers.md>)

### AI overview

Speech tokenizers can be disrupted by small perturbations, degrading downstream ASR and large audio-language model performance. The proposed T-SemAttack jointly shifts representations across an ensemble of semantic encoders while preserving perceptual quality. The authors report substantial drift in deep representations and directional alignment between surrogate-model input gradients, which they suggest may help explain transferability. ROSETok combines noisy training with robust semantic distillation to target reconstruction fidelity and downstream robustness. Experiments across multiple datasets, nine open-source and black-box ASR systems, and four LALMs report strong transferable attacks and robustness for ROSETok.

### Source excerpt

arXiv:2611.14929v1 Announce Type: cross Abstract: Speech tokenizers serve as speech front ends, encoding input speech signals into representations for downstream models, such as automatic speech recognition (ASR) systems and large audio-language models (LALMs). In this paper, we find that speech tokenizers are fragile at the semantic level: small perturbations can disrupt their semantic encoding process and consequently cause severe degradation in downstream ASR and LALM performance. To study this vulnerability, we propose T-SemAttack, a transferable semantic attack based on a semantic encoder ensemble. By jointly drifting the representation spaces of multiple semantic encoders, T-SemAttack effectively disrupts the semantic content of speech while preserving perceptual quality, thereby inducing error features. We further analyze layer-wise representation drift and cross-model gradient alignment. Our findings show that perturbations can induce substantial shifts in deep semantic representations and that the input gradients of different surrogate encoders exhibit directional alignment. These observations suggest that shared adversarially sensitive directions in the input space may be a key factor underlying cross-model attack transferability. Building on these, we introduce ROSETok, a robust speech tokenizer that combines noisy training with robust semantic distillation to improve reconstruction fidelity and downstream SpeechLLM robustness. Extensive experiments on multiple datasets, 9 open-source and black-box ASR systems, and 4 LALMs demonstrate that T-SemAttack achieves strong transferable attack performance, while the proposed Robust Speech Tokenizer exhibits robustness under high-fidelity reconstruction and downstream tasks. Our code and demo are available at https://wxzyd123.github.io/t-semattack.

## Swanada: Split-Encoder Capability-Routed Conformer CTC and Sub-30 MB On-Device Speech Recognition Across Twelve Scripts

DevFeed: [Swanada: Split-Encoder Capability-Routed Conformer CTC and Sub-30 MB On-Device Speech Recognition Across Twelve Scripts](<https://devfeed.tech/articles/swanada-split-encoder-capability-routed-conformer-ctc-and-sub-30-mb-on-device-speech-recognition-across-twelve-scripts-128358.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13814>)

Author: Hemanth HM

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Node.js](<https://devfeed.tech/topics/node-js.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [on-device-speech-recognition](<https://devfeed.tech/tags/on-device-speech-recognition.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [scripts](<https://devfeed.tech/tags/scripts.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

Swanada uses a shared INT8 Conformer encoder with hot-swappable, script-specific CTC heads to run streaming 16 kHz speech recognition for 12 languages across browser WebAssembly, Node.js, and Python. Its Studio tier is 177.38 MiB; the distilled Mobile tier is 25.35 MiB. The abstract reports 12-language storage reductions of 11.6x for Studio and 67.8x for Mobile, with 25.85 MiB required on Mobile's first load. Runtime changes synthesize positional encodings and replace pointwise convolutions with MatMul. On the reported held-out and open test clips, Studio achieved 5.43% and 8.87% CER, while Mobile achieved 18.56% and 19.23%; the abstract reports 0.0097x-0.0264x real-time factor and 30.6 ms median time to first token.

### Source excerpt

arXiv:2611.13814v1 Announce Type: cross Abstract: Client-side automatic speech recognition (ASR) across Indic and Classical scripts eliminates cloud transcription costs, network tail latency, and server-side audio retention, yet neural exports face four systems bottlenecks in browsers and mobile runtimes: (1) monolithic per-language Conformer checkpoints require ~177 MiB per script (> 2.1 GiB across 12 scripts), exhausting CacheStorage quotas; (2) static relative positional encoding tables embed 10.2-20.5 MB of FP32 constants while 1x1 Conv1d layers trigger slow or unsupported ConvInteger kernels in WebAssembly SIMD; (3) sliding-window CTC flickers on boundary tokens; and (4) mobile WebAudio pipelines suffer 44.1/48 kHz sample-rate mismatches and silent AudioWorklet stalls under iOS WKWebView. We present Swanada, an open-source 16 kHz streaming ASR system spanning Classical Sanskrit, Indian English, and ten Indian languages (sa, en, hi, kn, mr, bn, ta, te, ml, gu, pa, or) across browser WebAssembly, Node.js, and Python. Swanada pairs capability-routed shared INT8 Conformer encoders (a 17-layer 177.38 MiB Studio tier at $d = 512$ for desktop/servers and a distilled 14-layer 25.35 MiB Mobile tier at $d = 256$ for phones) with twelve hot-swappable 515.0 KiB native-script linear CTC heads ($257 \times 512$), cutting 12-language storage by $11.6\times$ (183.4 MiB) and $67.8\times$ (31.39 MiB; 25.85 MiB first load). Runtime protobuf tail surgery exposes the shared representation in $< 2$ ms, while dynamic positional synthesis, MatMul pointwise convolutions, and post-quantization QInt8 head calibration preserve sub-30 MB accuracy. Across $N = 480$ held-out and $N = 1,280$ open public test clips (FLEURS, LibriSpeech, GLOBE, AMI, wild Sanskrit), Swanada achieves 5.43% held-out and 8.87% 12-language open CER on Studio (18.56% and 19.23% on Mobile) at $0.0097\times$-$0.0264\times$ RTF, with 30.6 ms median TTFT and 0.00% committed-prefix churn.

## TransferBreaker: Breaking Adversarial Transferability in Fine-Tuned Speech Recognition

DevFeed: [TransferBreaker: Breaking Adversarial Transferability in Fine-Tuned Speech Recognition](<https://devfeed.tech/articles/transferbreaker-breaking-adversarial-transferability-in-fine-tuned-speech-recognition-128584.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.09109>)

Author: Mojtaba Nafez, Aref Mousavi, Mohammad Ebrahim Mahdavi, Mobina Poulaei, Kiarash Kiani Feriz, Mohammad Hossein Rohban

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [invariance](<https://devfeed.tech/tags/invariance.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [performance](<https://devfeed.tech/tags/performance.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [safety](<https://devfeed.tech/tags/safety.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

Adversarial audio perturbations crafted against a public pretrained ASR model can transfer to fine-tuned target models, undermining the protection expected from black-box deployment. TransferBreaker combines Base Adversarial Fine-Tuning, which limits training to perturbations effective on the base model; Latent Jacobian Regularization, which suppresses adversarially sensitive latent directions; and HybridGrad-AFT, which interpolates perturbations from base and target gradients to address adaptive attacks. The abstract reports evaluation across three languages and four large ASR models, with adversarial word error rate reduced from 92.6 to 27.8. It does not specify the evaluation setup or variation across models and languages.

### Source excerpt

arXiv:2610.09109v2 Announce Type: replace Abstract: Many organizations fine-tune publicly available pretrained Automatic Speech Recognition (ASR) models and deploy them in black-box settings, assuming limited access provides protection. We show this assumption is fragile: adversarial perturbations crafted on the public base model transfer effectively to fine-tuned target models, severely degrading performance and posing concerns for safety-critical applications. We propose TransferBreaker, a unified fine-tuning framework that suppresses adversarial transfer by integrating Base Adversarial Fine-Tuning, which restricts adversarial training to base-effective perturbations; Latent Jacobian Regularization, which enforces latent-space invariance by suppressing adversarially sensitive directions; and HybridGrad-AFT, which improves robustness against adaptive attacks by interpolating transferable perturbations from base and target gradients. We theoretically justify all components and evaluate TransferBreaker across three languages and four large ASR models, reducing adversarial WER from 92.6 to 27.8. Our code is publicly available at https://github.com/rohban-lab/TransferBreaker.

## Nepali-English Code-Mixed Speech Recognition: Data Preparation and Adaptation of a Speech Foundation Model

DevFeed: [Nepali-English Code-Mixed Speech Recognition: Data Preparation and Adaptation of a Speech Foundation Model](<https://devfeed.tech/articles/nepali-english-code-mixed-speech-recognition-data-preparation-and-adaptation-of-a-speech-foundation-model-128321.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13512>)

Author: Jeevan Neupane, Janardan Bhetwal, Kshitiz Paudel, Anup Aryal, Bipin Bashyal

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Whisper](<https://devfeed.tech/topics/whisper.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

Full fine-tuning of SraVaani on a 172.8-hour, six-domain Nepali speech corpus reduced test WER from 48.6% to 17.1%; a 12-epoch run reached 16.3%, close to fine-tuned Whisper large-v3-turbo at 16.4%. To handle the model tokenizer's lack of support for digits and common symbols, the study generates Nepali and English number readings and uses zero-shot ASR output to choose a plausible form. Freezing the encoder worsened WER to 25.5%, and excluding numeric clips raised WER on numeric test speech from 14.3% to 21.3%. FLEURS evaluation with original and mixed-script references indicates transcription conventions account for some, but not all, out-of-domain performance loss.

### Source excerpt

arXiv:2611.13512v1 Announce Type: new Abstract: Nepali speakers regularly mix English words into conversation, but multilingual speech recognisers often struggle with language switches and the writing script used for English words. We study how to adapt SraVaani, a 449M-parameter hybrid TDT-CTC model, using a 172.8-hour corpus covering six Nepali speech domains. Its pretrained tokenizer does not support digits and several common symbols. Rather than replace it or discard these recordings, we generate Nepali and English readings for numbers and use zero-shot ASR output to select a plausible form. Full fine-tuning reduces test word error rate (WER) from 48.6% to 17.1%; a 12-epoch run reaches 16.3%, compared with 16.4% for fine-tuned Whisper large-v3-turbo. Freezing the encoder increases WER to 25.5%, while excluding numeric clips raises WER on numeric test speech from 14.3% to 21.3%. We also evaluate on FLEURS using both its original references and mixed-script alternatives. The comparison shows that transcription conventions explain part, but not all, of the out-of-domain performance loss.

## Representational Alignment between Deep Neural Networks and Human Brain in Speech Processing under Audiovisual Noise

DevFeed: [Representational Alignment between Deep Neural Networks and Human Brain in Speech Processing under Audiovisual Noise](<https://devfeed.tech/articles/representational-alignment-between-deep-neural-networks-and-human-brain-in-speech-processing-under-audiovisual-noise-126255.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12291>)

Author: Cheng Luo, Jian Zhu, Zhoujian Sun, Yunyi Qi, Yu Qi, Wenhui Sun, Wenyuan Yu

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [after](<https://devfeed.tech/tags/after.md>), [align](<https://devfeed.tech/tags/align.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [applying](<https://devfeed.tech/tags/applying.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [asr](<https://devfeed.tech/tags/asr.md>), [brain](<https://devfeed.tech/tags/brain.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-neural-networks](<https://devfeed.tech/tags/deep-neural-networks.md>), [human-brain](<https://devfeed.tech/tags/human-brain.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural](<https://devfeed.tech/tags/neural.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [noise](<https://devfeed.tech/tags/noise.md>), [processing](<https://devfeed.tech/tags/processing.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-processing](<https://devfeed.tech/tags/speech-processing.md>), [under](<https://devfeed.tech/tags/under.md>)

### AI overview

The study reports that a pretrained automatic speech recognition model's shallow-layer embeddings align more closely with brain responses to acoustic features, while deeper-layer embeddings align more closely with responses to linguistic features. Researchers used neural encoding models for speech and noise envelopes, word onset, and surprisal, then applied representational similarity analysis to compare those responses with model embeddings before and after fine-tuning on audiovisual noisy data. Fine-tuning strengthened the reported alignment: shallow layers showed improved noise processing and deeper layers more refined linguistic representations. These findings suggest the model can develop brain-like patterns of speech processing in noise.

### Source excerpt

arXiv:2611.12291v1 Announce Type: new Abstract: Speech comprehension in the human brain is an incremental process that begins with acoustic processing and advances to linguistic processing. While recent studies have revealed that the hierarchy of deep neural networks (DNNs) correlates with the ascending auditory pathway, the exact nature of this DNN-brain alignment remains underexplored. In this study, we investigate how DNN representations align with the brain's acoustic-to-linguistic processing. Specifically, we employed neural encoding models to simulate neural responses to acoustic (i.e., speech and noise envelope) and linguistic features (i.e., word onset and surprisal). By applying representational similarity analysis (RSA), we quantified the similarity between these neural responses and the DNN embeddings generated by a pre-trained automatic speech recognition (ASR) model, both before and after fine-tuning on audiovisual noisy data. Our results demonstrate significant DNN-brain alignment: embeddings from shallow layers exhibit higher similarity to neural responses associated with acoustic features, while those from deeper layers align more closely with neural responses related to linguistic features. Importantly, the fine-tuning process enhances this alignment by improving noise processing in shallower layers and refining linguistic representations in deeper layers. These results suggest that fine-tuned DNN models can naturally develop human-like processing patterns in noisy environments, highlighting a functional alignment between the human brain and DNNs in speech representation.

## TID System for Speech-COSER Challenge 2026: ASR Track

DevFeed: [TID System for Speech-COSER Challenge 2026: ASR Track](<https://devfeed.tech/articles/tid-system-for-speech-coser-challenge-2026-asr-track-126122.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11497>)

Author: Fernando L\\'opez, Jordi Luque

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [asr](<https://devfeed.tech/tags/asr.md>), [audio](<https://devfeed.tech/tags/audio.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [combined](<https://devfeed.tech/tags/combined.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [elderly](<https://devfeed.tech/tags/elderly.md>), [lora](<https://devfeed.tech/tags/lora.md>), [pruning](<https://devfeed.tech/tags/pruning.md>), [spain](<https://devfeed.tech/tags/spain.md>), [speech](<https://devfeed.tech/tags/speech.md>)

### AI overview

For spontaneous speech from elderly speakers recorded in rural Spain, Telefónica Innovación Digital's Speech-COSER Challenge 2026 ASR system uses CTC-based pseudo-forced alignment to score audio-text match quality and guide a two-stage LoRA curriculum. The team then applies magnitude pruning to the LoRA matrices, aiming to improve generalization. A seven-system ensemble combined with ROVER reports 36.25% unnormalized and 22.17% normalized WER.

### Source excerpt

arXiv:2611.11497v1 Announce Type: cross Abstract: We describe the system submitted by Telef\'onica Innovaci\'on Digital to the Speech-COSER Challenge 2026, which features spontaneous speech from elderly speakers recorded in rural Spain. For ASR, we perform CTC-based pseudo-forced alignment scores audio-text match quality to guide a two-stage LoRA curriculum, followed by post-hoc magnitude pruning of the LoRA matrices to enhance generalization. An ensemble of seven systems combined via ROVER achieves 36.25\% unnormalized and 22.17\% normalized WER.

## KeyASRBench: Measuring Whole-Phrase Keyword Preservation in Real-World ASR

DevFeed: [KeyASRBench: Measuring Whole-Phrase Keyword Preservation in Real-World ASR](<https://devfeed.tech/articles/keyasrbench-measuring-whole-phrase-keyword-preservation-in-real-world-asr-126030.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10852>)

Author: Guanzheng Wei, Ziyi Zhao

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [measuring](<https://devfeed.tech/tags/measuring.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

KeyASRBench measures whether ASR transcripts preserve selected whole phrases, including names, products, technical terms, versions, and relational phrases--a dimension that conventional sequence error rates can miss. Its bilingual, 10-hour model-conditioned challenge set contains 1,501 clips and 4,723 keyword occurrences. Across five locally inferred checkpoints and two official API systems, language-specific MER/WER ranges from 5.31% to 12.91%, while occurrence-based Keyword Error Rate (KER) is higher, at 13.61% to 33.81%. A disjoint 30-minute model-blind control has KER of 10.65% to 27.31%, suggesting selected-phrase errors also occur beyond the challenge-set selection.

### Source excerpt

arXiv:2611.10852v1 Announce Type: cross Abstract: Public automatic speech recognition (ASR) benchmarks increasingly report low sequence error rates, yet production systems still use hotword lists, contextual biasing, retrieval, and post-correction. This motivates a complementary question: does a transcript preserve selected names, products, technical terms, versions, and relational phrases? We present KeyASRBench, a bilingual pilot that operationalizes occurrence-based Keyword Error Rate (KER) over frozen whole-phrase annotations. The 10-hour model-conditioned challenge set contains 1,501 Chinese and English clips with 4,723 keyword occurrences. Across five locally inferred checkpoints and two official API systems, language-specific MER/WER ranges from 5.31% to 12.91%, while KER ranges from 13.61% to 33.81%. A disjoint 30-minute model-blind control yields KER of 10.65%-27.31%. The control confirms that selected-phrase errors extend beyond challenge-set selection.

## Text Grouping Changes Completion-Time Comparisons in Long-Form Simultaneous Speech Translation

DevFeed: [Text Grouping Changes Completion-Time Comparisons in Long-Form Simultaneous Speech Translation](<https://devfeed.tech/articles/text-grouping-changes-completion-time-comparisons-in-long-form-simultaneous-speech-translation-126284.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12501>)

Author: Hieu Minh Truong, Dung D. Le

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [machine-translation](<https://devfeed.tech/topics/machine-translation.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [asr](<https://devfeed.tech/topics/asr.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [availability](<https://devfeed.tech/tags/availability.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [change](<https://devfeed.tech/tags/change.md>), [changes](<https://devfeed.tech/tags/changes.md>), [compare](<https://devfeed.tech/tags/compare.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [completion](<https://devfeed.tech/tags/completion.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [effects](<https://devfeed.tech/tags/effects.md>), [english](<https://devfeed.tech/tags/english.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [gap](<https://devfeed.tech/tags/gap.md>), [history](<https://devfeed.tech/tags/history.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-translation](<https://devfeed.tech/tags/speech-translation.md>), [text](<https://devfeed.tech/tags/text.md>), [time](<https://devfeed.tech/tags/time.md>), [translation](<https://devfeed.tech/tags/translation.md>)

### AI overview

Grouping translation updates before synthesis changed StreamAtt's measured completion lead, with opposite effects across synthesizers: four-second grouping increased the mean lead from 106.6 to 229.1 seconds with XTTS-v2, but reduced it from 169.5 to 146.5 seconds with Piper. Both produced shorter audio; differences in playback idle accounted for the opposing results. The directions recurred on a separately selected twenty-talk panel. In continuing CosyVoice 3 sessions, grouping left paired audio unchanged and delayed completion on 19 of 20 blocks. The findings indicate that speech-timing comparisons should specify how text is grouped for synthesis as well as which synthesizer is used. Recognition and automatic word timing describe outputs but do not establish translation accuracy or intelligibility.

### Source excerpt

arXiv:2611.12501v1 Announce Type: cross Abstract: Does grouping translation updates before speech synthesis change the measured completion gap between two translators? We compare StreamAtt with its fixed-audio-history baseline on ten English-German talks. For each translation, we keep the intended text and its availability times fixed and vary how updates are grouped for synthesis. Four-second grouping increases StreamAtt's mean completion lead from 106.6 to 229.1 seconds with XTTS-v2, but reduces it from 169.5 to 146.5 seconds with Piper. Both produce shorter audio; different changes in playback idle account for the opposite effects. These directions recur on a separately selected twenty-talk panel. In continuing CosyVoice 3 sessions, grouping instead leaves paired audio unchanged and delays completion on nineteen of twenty blocks. A speech-timing comparison therefore needs to specify how text enters synthesis, not just which synthesizer is used. Recognition and automatic word timing describe the outputs, but do not establish translation accuracy or intelligibility.

## Firefox 160 Available With Speech-To-Text, MKV Files With FLAC / MP3 / PCM Audio

DevFeed: [Firefox 160 Available With Speech-To-Text, MKV Files With FLAC / MP3 / PCM Audio](<https://devfeed.tech/articles/firefox-160-available-with-speech-to-text-mkv-files-with-flac-mp3-pcm-audio-124250.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/Firefox-160-Released>)

Author: Michael Larabel

Published: 2026-11-09T23:12:33Z

Content type: release

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [Firefox](<https://devfeed.tech/topics/firefox.md>), [asr](<https://devfeed.tech/topics/asr.md>), [releases](<https://devfeed.tech/topics/releases.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [audio](<https://devfeed.tech/tags/audio.md>), [browser](<https://devfeed.tech/tags/browser.md>), [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [device](<https://devfeed.tech/tags/device.md>), [domain-name](<https://devfeed.tech/tags/domain-name.md>), [firefox](<https://devfeed.tech/tags/firefox.md>), [flac](<https://devfeed.tech/tags/flac.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [mp3](<https://devfeed.tech/tags/mp3.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [pcm](<https://devfeed.tech/tags/pcm.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [picture-in-picture](<https://devfeed.tech/tags/picture-in-picture.md>), [playback](<https://devfeed.tech/tags/playback.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [real-time-speech-to-text](<https://devfeed.tech/tags/real-time-speech-to-text.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>)

### AI overview

Firefox 160 adds playback support for Matroska (MKV) files with FLAC, MP3, or PCM audio, including MP3 support despite an earlier request noting it was not on the feature plans due to low usage. Its Web Speech API speech recognition enables on-device, real-time speech-to-text in 40 languages. Picture-in-Picture gains speech playback speed settings from 0.25x to 2.0x, and the address bar increases contrast between the domain name and the rest of the address. Release binaries are available from ftp.mozilla.org.

### Source excerpt

The Firefox 160.0 release builds are now available as the latest bi-weekly stable update to Mozilla's web browser...

## Underdog shows off a 32MB speech model in new demo

DevFeed: [Underdog shows off a 32MB speech model in new demo](<https://devfeed.tech/articles/underdog-shows-off-a-32mb-speech-model-in-new-demo-121061.md>)

Original publisher: [Read original article](<https://techcrunch.com/2026/11/06/underdog-shows-off-a-32mb-speech-model-in-new-demo/>)

Author: Ivan Mehta

Published: 2026-11-06T13:02:16Z

Content type: news

Language: en

Sources: [TechCrunch | AI News & Artificial Intelligence](<https://devfeed.tech/sources/techcrunch-ai-news-artificial-intelligence.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [addition](<https://devfeed.tech/tags/addition.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-companies](<https://devfeed.tech/tags/ai-companies.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [app](<https://devfeed.tech/tags/app.md>), [based](<https://devfeed.tech/tags/based.md>), [best](<https://devfeed.tech/tags/best.md>), [can](<https://devfeed.tech/tags/can.md>), [demo](<https://devfeed.tech/tags/demo.md>), [generate](<https://devfeed.tech/tags/generate.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [model](<https://devfeed.tech/tags/model.md>), [music](<https://devfeed.tech/tags/music.md>), [shows](<https://devfeed.tech/tags/shows.md>), [size](<https://devfeed.tech/tags/size.md>), [small](<https://devfeed.tech/tags/small.md>), [speech](<https://devfeed.tech/tags/speech.md>), [underdog](<https://devfeed.tech/tags/underdog.md>)

### AI overview

Underdog's 32 MB Woof model powers a browser demo that speaks text typed on a virtual typewriter and generates music through a tuner. The company says Woof has 31 million parameters and can run on most devices without an internet connection; the model is available to download. The article cautions that a model this small is unlikely to replace larger ones, while suggesting it could support local assistants or personalized podcasts. It also notes Underdog's separate 167 MB Bark transcription and dictation model for edge devices.

### Source excerpt

Despite its small size, Underdog's new model can generate both speech and music.

## Murmur: Efficient Long-Context ASR Inference with Hierarchical Chunking

DevFeed: [Murmur: Efficient Long-Context ASR Inference with Hierarchical Chunking](<https://devfeed.tech/articles/murmur-efficient-long-context-asr-inference-with-hierarchical-chunking-120043.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.01483>)

Author: Wei-Tzu Lee, Keisuke Kamahori, Baris Kasikci

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Model Optimizer](<https://devfeed.tech/topics/model-optimizer.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [speculative-decoding](<https://devfeed.tech/tags/speculative-decoding.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

Murmur uses hierarchical chunking to speed long-form ASR while preserving long-context verification. Its analysis finds that chunks of about five minutes match single-pass accuracy while avoiding repetition loops. The system decodes target chunks in parallel, using a fast ASR model on 30-second windows to draft transcripts and simple format rules to draft timestamps and speaker tags; the long-context model verifies every token through lossless, structure-aware speculative decoding. Across two long-context ASR models and four English and Mandarin meeting datasets, Murmur reduces end-to-end latency, including diarization, by 2.5x on average (1.9-2.9x) versus single-pass decoding and lowers tcpWER by an average of 1.9 points.

### Source excerpt

arXiv:2606.01483v2 Announce Type: replace-cross Abstract: Long-form automatic speech recognition (ASR) requires both high accuracy and low inference latency, but existing systems fall short of handling the trade-off between the two. Chunk-based approaches process audio in parallel windows for low latency, but lose cross-chunk context and need brittle heuristics to align speakers and timestamps at boundaries. Long-context ASR models transcribe everything in a single pass for better accuracy, but run several times slower, and a single repetition loop can corrupt the entire output. In this paper, we first analyze how context length affects the accuracy of long-context ASR models and find that an intermediate chunk size of about five minutes matches single-pass accuracy while avoiding repetition loops. Building on this, we propose Murmur, an inference system for long-form ASR based on hierarchical chunking. The long-context model decodes intermediate target chunks in parallel, and a fast short-window ASR on 30 s draft windows accelerates each chunk with lossless, structure-aware speculative decoding. The short-window ASR drafts the transcript, simple format rules draft the timestamps and speaker tags, and the long-context model verifies every token. On two long-context ASR models and four meeting datasets in English and Mandarin, Murmur cuts end-to-end latency per recording by 2.5x on average (1.9-2.9x) relative to single-pass decoding, including diarization, while lowering tcpWER by 1.9 points on average. Our code is available at https://github.com/uw-syfi/Murmur.

## Gabor Gabor: E-Gaborformer, a Strictly Causal Speech Recognizer with Gabor-Dictionary Convolutions and Delta-Rule Memory

DevFeed: [Gabor Gabor: E-Gaborformer, a Strictly Causal Speech Recognizer with Gabor-Dictionary Convolutions and Delta-Rule Memory](<https://devfeed.tech/articles/gabor-gabor-e-gaborformer-a-strictly-causal-speech-recognizer-with-gabor-dictionary-convolutions-and-delta-rule-memory-120702.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.08092>)

Author: Nir Krakowski, Tamir Cohen, Ohad Zohar, Chen Zarfati

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>)

Tags: [3](<https://devfeed.tech/tags/3.md>), [asr](<https://devfeed.tech/tags/asr.md>), [attention](<https://devfeed.tech/tags/attention.md>), [combines](<https://devfeed.tech/tags/combines.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [delta](<https://devfeed.tech/tags/delta.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [future](<https://devfeed.tech/tags/future.md>), [lookahead](<https://devfeed.tech/tags/lookahead.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [production](<https://devfeed.tech/tags/production.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [real-time-voice-agents](<https://devfeed.tech/tags/real-time-voice-agents.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [scale](<https://devfeed.tech/tags/scale.md>), [speech](<https://devfeed.tech/tags/speech.md>), [state](<https://devfeed.tech/tags/state.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [temporal](<https://devfeed.tech/tags/temporal.md>)

### AI overview

E-Gaborformer is a streaming speech-recognition encoder designed to avoid future-audio dependence and keep per-call memory constant. It combines Gated DeltaNet-2 recurrent layers with sliding-window attention in a 3:1 ratio and constructs temporal convolutions from a fixed dictionary of causal Gabor atoms. The abstract reports about 19 MB of streaming state per production call, including the prediction network, compared with over 1 GB after ten minutes for a same-size causal-attention encoder. In a controlled LibriSpeech study, it reached target accuracy up to 18% sooner and matched or beat E-Branchformer under the same front end and recipe. A 0.6B-parameter Hebrew-English transducer using the encoder had 28-31% lower English WER than same-size NVIDIA Nemotron 3.5 ASR Streaming on voice-agent traffic, and up to 20% lower Hebrew WER than the authors' prior system. The same weights supported strictly causal and 480 ms-lookahead decoding without reported accuracy loss; a two-class RNN-T lattice formulation reportedly trained 2.6 times faster with exact loss and gradients.

### Source excerpt

arXiv:2611.08092v1 Announce Type: new Abstract: Real-time voice agents need a speech recognizer that never waits for future audio and whose memory per call does not grow with the length of the call. We present E-Gaborformer, a strictly causal encoder that combines Gated DeltaNet-2 recurrent layers with sliding-window attention in a 3:1 ratio, and builds every temporal convolution from a fixed dictionary of causal Gabor atoms. Its streaming state is constant, about 19 MB per call at production scale including the prediction network, where a causal-attention encoder of the same size accumulates over 1 GB after ten minutes. In a controlled causal study on LibriSpeech, E-Gaborformer leaves the initial plateau earlier, reaches a given accuracy up to 18% sooner, and matches or beats an E-Branchformer trained with the identical front end and recipe. At production scale, a 0.6B-parameter Hebrew and English streaming transducer built on E-Gaborformer has 28 to 31% lower word error rate (WER) than NVIDIA's Nemotron 3.5 ASR Streaming, a model of the same size, on English voice-agent traffic, and up to 20% lower Hebrew WER than our previous production system. The same weights serve strictly causal and 480 ms-lookahead decoding with no loss in accuracy, and a two-class formulation of the RNN-T lattice trains the transducer 2.6x faster with exact loss and gradients.

## Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription

DevFeed: [Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription](<https://devfeed.tech/articles/confucius4-r2t2-a-stable-prefix-streaming-asr-model-for-real-real-time-transcription-118414.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06043>)

Author: Daniel Huang, Qi He, Shaoxiong Duan, Yingjie Li, Qiao Cheng, Yitao Duan

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [low-latency](<https://devfeed.tech/tags/low-latency.md>), [model-weights](<https://devfeed.tech/tags/model-weights.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [stable](<https://devfeed.tech/tags/stable.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [streaming-speech-recognition](<https://devfeed.tech/tags/streaming-speech-recognition.md>), [transcription](<https://devfeed.tech/tags/transcription.md>)

### AI overview

Confucius4-R2T2 converts Qwen3-ASR-1.7B into a streaming recognizer that keeps previously emitted text unchanged. It is post-trained on 1,500 hours of Chinese and English audio, with the pretrained audio encoder and projector frozen; learning from Longest Stable Prefix labels enables streaming without architectural changes. The abstract reports streaming quality close to the base model's full-context quality, low emission lag, and gains beyond the post-training languages. The authors also report proving that LSP is monotone under the prefix order, consistent with full-audio output, and maximal among labels equal to a whole earlier output. Model weights and code are described as open source.

### Source excerpt

arXiv:2611.06043v1 Announce Type: cross Abstract: Streaming speech recognition is increasingly consumed by machines, such as translators and voice agents, that act on each token as it appears. A token that is later revised costs wasted work or a wrong action. In this work, we present Confucius4-R2T2 (short for Real Real-Time Transcription), a stable-prefix streaming ASR model that never modifies its earlier output while maintaining high accuracy and low latency. R2T2 is obtained by post-training Qwen3-ASR-1.7B on 1,500 hours of Chinese and English audio while freezing its pre-trained audio encoder and projector. The adaptation is based on the concept of Longest Stable Prefix (LSP), a new notion that we introduce in this work. We prove that LSP is monotone under the prefix order, consistent with the full-audio output, and maximal among labels that equal a whole earlier output. Learning from LSP labels converts a full-context model into a stable-prefix streaming one without any architectural change. Extensive evaluations show that R2T2 brings streaming recognition close to the base model's full-context quality while delivering stable text with low emission lag. Its gains extend beyond the post-training languages, demonstrating a practical path from full-context ASR to stable streaming. We open-source the model weights and code at https://huggingface.co/netease-youdao/Confucius4-R2T2 and https://github.com/netease-youdao/Confucius4-R2T2.

## Text-Only Domain Adaptation for Low-Resource Medical ASR: Shallow Fusion and Density Ratio Rescoring for Hausa

DevFeed: [Text-Only Domain Adaptation for Low-Resource Medical ASR: Shallow Fusion and Density Ratio Rescoring for Hausa](<https://devfeed.tech/articles/text-only-domain-adaptation-for-low-resource-medical-asr-shallow-fusion-and-density-ratio-rescoring-for-hausa-118371.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05744>)

Author: Oche David Ankeli, Hassan Adelani Luqman, Theodora Ngozi Egbunike, Joshua Gottschalk, Bonaventure F. P. Dossou

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [concept-learning](<https://devfeed.tech/topics/concept-learning.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [domain](<https://devfeed.tech/tags/domain.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [medical](<https://devfeed.tech/tags/medical.md>), [ratio](<https://devfeed.tech/tags/ratio.md>), [resource](<https://devfeed.tech/tags/resource.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [text](<https://devfeed.tech/tags/text.md>)

### AI overview

For Hausa medical ASR, decoding-time language-model fusion lowered word error rate (WER) with a fixed MMS-1B-FL102 acoustic model, without collecting domain-specific speech. In prompt-disjoint five-fold cross-validation on 250 recordings, Shallow Fusion with a medical 4-gram KenLM reduced WER from 29.56% to 23.55%; Density Ratio Approach rescoring reached 23.33%. Both beat greedy decoding significantly, but their difference was not significant. General-language-model-only Shallow Fusion achieved 23.96%, so the medical model's additional gain was modest, and the Medical Term Error Rate results did not establish a distinct medical-domain advantage.

### Source excerpt

arXiv:2611.05744v1 Announce Type: cross Abstract: Automatic speech recognition (ASR) systems often struggle with specialised medical vocabulary, especially in low-resource languages where domain-specific speech data are scarce. We investigate text-only domain adaptation for Hausa medical ASR using the fixed MMS-1B-FL102 acoustic model and two decoding-time language-model fusion methods: Shallow Fusion (SF) and the Density Ratio Approach (DRA). A 4-gram KenLM trained on 19,421 Hausa health sentences serves as the medical target model, while a 75,372-sentence general-domain model provides the source prior for DRA. Evaluation uses 250 recordings of 25 physician-reviewed prompts from 10 speakers across six northern Nigerian states. Under prompt-disjoint five-fold cross-validation, SF and DRA reduce WER from 29.56% to 23.55% and 23.33%, respectively. Both significantly outperform greedy decoding (p < 0.001), while their difference is not significant (p = 0.423). However, general-LM-only SF achieves 23.96% WER, indicating that the medical LM's additional improvement over a general LM is modest. Medical Term Error Rate also improves under fusion, but does not establish a distinct medical-domain advantage. These findings demonstrate that lightweight, text-only language-model fusion can improve Hausa medical ASR without additional domain-specific speech collection.

## AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech

DevFeed: [AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech](<https://devfeed.tech/articles/aegisvox-factorized-privacy-and-clinical-preservation-for-long-form-speech-117294.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.14643>)

Author: Praphul Singh, Akshat Agarwal, Ratul Ghosh, Aashna Kanuga, Devashish Khatwani, Anurag Dwarakanath, Sri Gadde, Krishnaram Kenthapadi

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [asr](<https://devfeed.tech/topics/asr.md>), [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [codec](<https://devfeed.tech/tags/codec.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [identifier](<https://devfeed.tech/tags/identifier.md>), [measurement](<https://devfeed.tech/tags/measurement.md>), [measurements](<https://devfeed.tech/tags/measurements.md>), [model](<https://devfeed.tech/tags/model.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [rendering](<https://devfeed.tech/tags/rendering.md>), [replacement](<https://devfeed.tech/tags/replacement.md>), [speaker](<https://devfeed.tech/tags/speaker.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [tradeoffs](<https://devfeed.tech/tags/tradeoffs.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [turns](<https://devfeed.tech/tags/turns.md>), [voice](<https://devfeed.tech/tags/voice.md>)

### AI overview

AegisVox assigns consistent surrogate voices and replaces identifier-bearing turns with document-consistent text, aiming to protect identity while retaining clinical speech cues. In a matched comparison across 390 public simulated or non-patient conversations (63.6 hours), text-to-speech realized 535 of 601 planned replacements, versus 453 for a codec-based editor. Yet similar aggregate clinical preservation masked more model-labeled contradictions, and conversation-wide speaker scores masked greater edited-speech linkage in a score-direction sensitivity analysis. Fresh transcription of original audio and an independent judge were used to examine measurement sensitivity. The findings show that better replacement realization can coexist with worse clinical or speaker-linkage measurements; the framework supports use-specific evaluation, not guarantees of identifier removal or clinical safety.

### Source excerpt

arXiv:2610.14643v2 Announce Type: replace Abstract: Clinical speech reveals identity through both spoken identifiers and voice, yet its pauses, delivery, and speaker roles can also carry useful clinical evidence. Protecting that speech therefore requires more than preserving its words. We present AegisVox, a factorized framework that assigns consistent surrogate voices and edits identifier-bearing turns using document-consistent replacements. Across 390 public simulated or non-patient conversations totaling 63.6 hours, a matched comparison of two speech renderers separates replacement realization from retained identifiers, speaker linkage, and clinical preservation. Text-to-speech rendering realizes 535/601 planned replacements versus 453/601 for the codec-based speech editor. However, similar aggregate clinical preservation conceals more model-labeled contradictions, and conversation-wide speaker scores conceal greater edited-speech linkage under a score-direction sensitivity analysis. Fresh transcription of original audio provides a reference for interpreting the clinical measurement chain; an independent judge tests sensitivity to model-based assessment. Better replacement realization can therefore coexist with worse clinical or speaker-linkage measurements. AegisVox supports use-specific evaluation of those tradeoffs, not a guarantee of identifier removal or clinical safety.

## Soniox STT v6 Adds Agent Text Input and Echo Filtering for Voice Agents

DevFeed: [Soniox STT v6 Adds Agent Text Input and Echo Filtering for Voice Agents](<https://devfeed.tech/articles/soniox-stt-v6-adds-agent-text-input-and-echo-filtering-for-voice-agents-115949.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/11/03/soniox-stt-v6-agent-text-input-echo-cancellation/>)

Author: Michal Sutter

Published: 2026-11-03T20:18:22Z

Content type: release

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>), [speech-to-speech](<https://devfeed.tech/topics/speech-to-speech.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-context](<https://devfeed.tech/tags/agent-context.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai-webinars](<https://devfeed.tech/tags/ai-webinars.md>), [api](<https://devfeed.tech/tags/api.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [audio-language-model](<https://devfeed.tech/tags/audio-language-model.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cancellation](<https://devfeed.tech/tags/cancellation.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [echo](<https://devfeed.tech/tags/echo.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [harness](<https://devfeed.tech/tags/harness.md>), [input](<https://devfeed.tech/tags/input.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [model](<https://devfeed.tech/tags/model.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [release](<https://devfeed.tech/tags/release.md>), [replies](<https://devfeed.tech/tags/replies.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [speech-to-text-api](<https://devfeed.tech/tags/speech-to-text-api.md>), [staff](<https://devfeed.tech/tags/staff.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [stt](<https://devfeed.tech/tags/stt.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>), [text](<https://devfeed.tech/tags/text.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

Soniox STT v6 can use a voice agent's streamed text to help recognize short caller replies and filter leaked agent speech. Agent text input is available only on streaming `stt-rt-v6`: send text over the caller-audio WebSocket, ideally 500 ms to 1 second before playback because messages have no timestamps. The feature complements acoustic echo cancellation, which should remain enabled, and still allows callers to interrupt. The file model `stt-async-v6` does not support agent text input. Soniox reports improvements but has published no v6 accuracy or latency results; the cited Pipecat figures are for v5. The excerpt lists prices of $0.12/hour for streaming and $0.10/hour for async, with agent text tokens billed separately.

### Source excerpt

Soniox has released Soniox STT v6, the latest generation of its speech-to-text API. Soniox STT v6 is a speech-to-text model that reads what a voice agent says while hearing the caller. It uses that text to recognize short replies and to drop the agent's own voice. The release ships as stt-rt-v6 for streaming and stt-async-v6 [...] The post Soniox STT v6 Adds Agent Text Input and Echo Filtering for Voice Agents appeared first on MarkTechPost.

[Next page](<https://devfeed.tech/topics/asr.md?cursor=WyIyMDI2LTExLTAzVDIwOjE4OjIyKzAwOjAwIiwgIjIxODJmNWU5LWNhZGYtNDhiMC05ZTNlLTU0NmU2YzFlMTM0NCJd>)