# Speech Processing

A computing discipline concerned with acquiring, analyzing, transforming, storing, and reproducing speech signals, including recognition and synthesis.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Reson8 releases Rumble-2-Flash speech model as a free open-weights download

DevFeed: [Reson8 releases Rumble-2-Flash speech model as a free open-weights download](<https://devfeed.tech/articles/reson8-releases-rumble-2-flash-speech-model-as-a-free-open-weights-download-140709.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/18/reson8-releases-rumble-2-flash-speech-model-as-a-free-open-weights-download/>)

Author: Duncan Riley

Published: 2026-11-18T14:00:57Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [releases](<https://devfeed.tech/topics/releases.md>), [Whisper](<https://devfeed.tech/topics/whisper.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-act](<https://devfeed.tech/tags/ai-act.md>), [amsterdam](<https://devfeed.tech/tags/amsterdam.md>), [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [balderton-capital](<https://devfeed.tech/tags/balderton-capital.md>), [developers](<https://devfeed.tech/tags/developers.md>), [download](<https://devfeed.tech/tags/download.md>), [european-union](<https://devfeed.tech/tags/european-union.md>), [free](<https://devfeed.tech/tags/free.md>), [general-data-protection-regulation](<https://devfeed.tech/tags/general-data-protection-regulation.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [iso-27001](<https://devfeed.tech/tags/iso-27001.md>), [james-wise](<https://devfeed.tech/tags/james-wise.md>), [jarno-verhagen](<https://devfeed.tech/tags/jarno-verhagen.md>), [medical-transcription](<https://devfeed.tech/tags/medical-transcription.md>), [model](<https://devfeed.tech/tags/model.md>), [netherlands](<https://devfeed.tech/tags/netherlands.md>), [news](<https://devfeed.tech/tags/news.md>), [np-hard-ventures](<https://devfeed.tech/tags/np-hard-ventures.md>), [open-asr-leaderboard](<https://devfeed.tech/tags/open-asr-leaderboard.md>), [open-weights](<https://devfeed.tech/tags/open-weights.md>), [openai](<https://devfeed.tech/tags/openai.md>), [pre-seed-funding](<https://devfeed.tech/tags/pre-seed-funding.md>), [raoul-ritter](<https://devfeed.tech/tags/raoul-ritter.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [reson8](<https://devfeed.tech/tags/reson8.md>), [resonate-labs](<https://devfeed.tech/tags/resonate-labs.md>), [rumble-2](<https://devfeed.tech/tags/rumble-2.md>), [rumble-2-flash](<https://devfeed.tech/tags/rumble-2-flash.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-models](<https://devfeed.tech/tags/speech-models.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [startup](<https://devfeed.tech/tags/startup.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [thomas-kluiters](<https://devfeed.tech/tags/thomas-kluiters.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>), [whisper-large-v3](<https://devfeed.tech/tags/whisper-large-v3.md>)

### AI overview

Reson8 has released Rumble-2-Flash as a free open-weights download under Apache 2.0. The 500-million-parameter model is designed for fast transcription of prerecorded audio and supports more than 40 languages. In Reson8's reported tests, it averaged 7.09% word error rate across seven languages and 17.6% across 10 lower-resource languages; Reson8 reports Whisper Large V3 at 39.82% on the latter test. These are company-reported results. Custom vocabulary support for up to 50,000 terms remains exclusive to the larger Rumble-2 model. Rumble-2-Flash is also available through Reson8's API, starting at EUR 0.01 per hour of audio.

### Source excerpt

Amsterdam-based speech recognition startup Resonate Labs B.V. today released its Rumble-2-Flash transcription model as a free download with open weights. Designed for fast transcription of prerecorded audio, the 500 million-parameter model supports more than 40 languages. Reson8 is pitching the release at developers and researchers who need speech recognition built for languages other than English. According [...] The post Reson8 releases Rumble-2-Flash speech model as a free open-weights download appeared first on SiliconANGLE.

## Event-Based Dynamic Lip Region Estimation During Speech

DevFeed: [Event-Based Dynamic Lip Region Estimation During Speech](<https://devfeed.tech/articles/event-based-dynamic-lip-region-estimation-during-speech-141843.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23111>)

Author: Hamze Hammami, Omar Eldardeer, Francesco Rea

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Video Analytics](<https://devfeed.tech/topics/video-analytics.md>)

Tags: [cameras](<https://devfeed.tech/tags/cameras.md>), [cnn](<https://devfeed.tech/tags/cnn.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [event](<https://devfeed.tech/tags/event.md>), [grid](<https://devfeed.tech/tags/grid.md>), [high-dynamic-range](<https://devfeed.tech/tags/high-dynamic-range.md>), [human-robot-interaction](<https://devfeed.tech/tags/human-robot-interaction.md>), [network](<https://devfeed.tech/tags/network.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [region](<https://devfeed.tech/tags/region.md>), [rnn](<https://devfeed.tech/tags/rnn.md>), [roi](<https://devfeed.tech/tags/roi.md>), [sensor](<https://devfeed.tech/tags/sensor.md>), [speaker](<https://devfeed.tech/tags/speaker.md>), [speakers](<https://devfeed.tech/tags/speakers.md>), [speech](<https://devfeed.tech/tags/speech.md>), [stateful](<https://devfeed.tech/tags/stateful.md>), [stateless](<https://devfeed.tech/tags/stateless.md>), [voxel](<https://devfeed.tech/tags/voxel.md>)

### AI overview

The method estimates a dense lip-region heatmap from full event-camera input, addressing the reliance of event-based speech systems on pre-cropped regions or mouth landmarks. It encodes events in polarity-separated voxel grids and uses a two-pathway continuous-time recurrent network: a stateless fast pathway and a stateful slow pathway, both using learned bounded time constants tied to elapsed time. On four unseen speakers it reaches 69.5% on lips and switches targets 2.1 times per second, compared with 8.9 and 11.0 switches for larger convolutional baselines. A spatial encoder improves precision but fails at a new face scale. On a real sensor despite training only on simulated events, the method scores 58.0% versus the CNN's 43.2%, with fewer than half as many target switches.

### Source excerpt

arXiv:2611.23111v1 Announce Type: cross Abstract: Event cameras offer high temporal resolution and low power consumption, making them promising for speech-related applications. These include automatic lip reading, audiovisual speech analysis, and human-robot interaction, particularly in challenging environments characterized by low illumination or high dynamic range. Event-based lip reading, speaker recognition, and voice activity detection, however, mostly operate on a cropped region of interest (ROI), by hand or by the dataset, and no event-based method supplies that ROI as a spatial region; existing event-based face analysis gives the mouth as landmarks, which presents a mismatch between application and localization. The present work predicts a dense heatmap over the lip region from the full event "frame". Events are encoded as a polarity-separated voxel grid and processed at every location by a two-pathway continuous-time recurrent network (CT-RNN) with learned, log-space bounded time constants acting on the real elapsed time of each bin: a stateless fast pathway and a stateful slow pathway, utilised for refining and anchoring the prediction. The model reaches 69.5% on lips on four unseen speakers and switches target only 2.1 times per second, against 8.9 and 11.0 for convolutional baselines with more parameters. Adding a spatial encoder improves precision but fails on a new face scale. On a real event sensor, trained only on simulated events, the proposed model reaches 58.0% on lips against 43.2% for the CNN, with fewer than half its target switches.

## A Japanese speech synthesis library that runs on an ESP32-S3 using Arduino

DevFeed: [A Japanese speech synthesis library that runs on an ESP32-S3 using Arduino](<https://devfeed.tech/articles/a-japanese-speech-synthesis-library-that-runs-on-an-esp32-s3-using-arduino-137900.md>)

Original publisher: [Read original article](<https://blog.adafruit.com/2026/11/17/a-japanese-speech-synthesis-library-that-runs-on-an-esp32-s3-using-arduino/>)

Author: Anne Barela

Published: 2026-11-17T16:18:08Z

Content type: article

Language: en

Sources: [Adafruit Industries](<https://devfeed.tech/sources/adafruit-industries-makers-hackers-artists-designers-and-engineers.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Library](<https://devfeed.tech/topics/library.md>), [ESP32-S3](<https://devfeed.tech/topics/esp32-s3.md>), [Arduino](<https://devfeed.tech/topics/arduino.md>)

Tags: [amplifier](<https://devfeed.tech/tags/amplifier.md>), [arduino](<https://devfeed.tech/tags/arduino.md>), [arduino-ide](<https://devfeed.tech/tags/arduino-ide.md>), [audio](<https://devfeed.tech/tags/audio.md>), [board](<https://devfeed.tech/tags/board.md>), [bus](<https://devfeed.tech/tags/bus.md>), [dac](<https://devfeed.tech/tags/dac.md>), [download](<https://devfeed.tech/tags/download.md>), [esp32](<https://devfeed.tech/tags/esp32.md>), [esp32-s3](<https://devfeed.tech/tags/esp32-s3.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [example](<https://devfeed.tech/tags/example.md>), [github](<https://devfeed.tech/tags/github.md>), [i2s](<https://devfeed.tech/tags/i2s.md>), [japanese](<https://devfeed.tech/tags/japanese.md>), [library](<https://devfeed.tech/tags/library.md>), [projects](<https://devfeed.tech/tags/projects.md>), [sketch](<https://devfeed.tech/tags/sketch.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-synthesis](<https://devfeed.tech/tags/speech-synthesis.md>), [text](<https://devfeed.tech/tags/text.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voicevox](<https://devfeed.tech/tags/voicevox.md>)

### AI overview

MakoTalk 2.1.2 synthesizes Japanese speech offline on an ESP32-S3: an Arduino sketch supplies kana, selects a voice dictionary, configures I2S pins, and calls mspeak() to play audio. The library installs from a ZIP in the Arduino IDE; its MinimalSpeak example speaks at startup and again after reset. The default Zundamon voice uses 123 sound segments. On a 4MB board with the Default partition, the app has about 1.2MB, too little for the roughly 1.56MB sample. The software terms allow personal study and non-commercial sharing with credit, but commercial use--including evaluation and prototyping--requires prior permission; the voice data has separate terms that request credit in distributions, writeups, and videos.

### Source excerpt

MakoTalk, from airuyoshii-ai, is a Japanese speech synthesis library that runs on an ESP32-S3. Text goes in as kana, the Japanese phonetic script, and audio comes out over I2S, the digital audio bus used by small DAC and amplifier boards. A sketch loads the library, picks a voice dictionary, sets the I2S pins, then calls [...]

## ProTTS: End-to-End Prominence-aware Natural and Controllable Text-to-Speech via Automatic Word Stress Discovery

DevFeed: [ProTTS: End-to-End Prominence-aware Natural and Controllable Text-to-Speech via Automatic Word Stress Discovery](<https://devfeed.tech/articles/protts-end-to-end-prominence-aware-natural-and-controllable-text-to-speech-via-automatic-word-stress-discovery-138945.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19832>)

Author: Sai Harshitha Aluru, Chiranjeevi Yarra

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [energy](<https://devfeed.tech/tags/energy.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [features](<https://devfeed.tech/tags/features.md>), [framework](<https://devfeed.tech/tags/framework.md>), [inference](<https://devfeed.tech/tags/inference.md>), [model](<https://devfeed.tech/tags/model.md>), [speech](<https://devfeed.tech/tags/speech.md>), [stress](<https://devfeed.tech/tags/stress.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [text](<https://devfeed.tech/tags/text.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [text-to-speech-tts](<https://devfeed.tech/tags/text-to-speech-tts.md>), [training](<https://devfeed.tech/tags/training.md>), [word](<https://devfeed.tech/tags/word.md>)

### AI overview

ProTTS combines automatic word-stress discovery and speech synthesis in one FastSpeech_2-based model. A clustering head learns stress from prosodic features without stress labels; the discovered stress guides predictors that adjust pitch, energy, and duration. At inference, users can supply a custom stress mask to control emphasis. The abstract reports a listener MOS of 3.96 versus 3.12 for the baseline and an 83.5% preference rate. Stress-detection F1 is 0.839/0.861 for German/Italian L2 speech on ISLE, compared with 0.70 for an unsupervised and 0.82 for a supervised baseline; zero-shot transfer to Tatoeba reaches 0.933 F1. These are abstract-level results across the stated datasets and benchmarks.

### Source excerpt

arXiv:2611.19832v1 Announce Type: cross Abstract: Natural speech conveys meaning not only through words, but also through the prominence on a few words. Yet most text-to-speech (TTS) systems remain prominence-unaware, existing prominence-aware TTS require external stress annotations, while unsupervised stress detectors operate in isolation, disconnected from synthesis. This creates a fundamental gap no existing system can automatically discover where stress should be placed, realize it as natural-sounding prosody, and offer user control over emphasis placement, all within a single model. To circumvent this, we propose ProTTS, a unified framework where stress detection emerges from the TTS model itself and, in turn, guides it to synthesize more natural and controllable speech. We incorporate a clustering head into the variance adaptor of FastSpeech_2 that discovers word stress directly from aggregated prosodic features. The discovered stress then drives emphasis variance predictors that modify pitch, energy, and duration for stressed words. The framework is trained end-to-end: TTS reconstruction and clustering losses jointly shape a stress-discriminative feature space without requiring stress labels at any stage. At inference, the same architecture also supports user-controllable emphasis by accepting a custom stress mask. We evaluate ProTTS on ISLE (non-native), LibriTTS, VCTK (native training), and three evaluation benchmarks (Tatoeba, TinyStress-15k, EmphAssess). Listeners prefer ProTTS over the baseline with a MOS of 3.96 vs. 3.12 and 83.5% preference rate. For stress detection, ProTTS achieves 0.839/0.861 F1 on ISLE (German/Italian L2), outperforming unsupervised (0.70) and supervised (0.82) baselines, and 0.933 F1 in zero-shot transfer to Tatoeba.

## Latency, Speech Recognition and Turn-Taking in Voice Agents for Spanish Phone Negotiation: An Externally Measured Comparison of Pipecat, LiveKit and GPT-Live

DevFeed: [Latency, Speech Recognition and Turn-Taking in Voice Agents for Spanish Phone Negotiation: An Externally Measured Comparison of Pipecat, LiveKit and GPT-Live](<https://devfeed.tech/articles/latency-speech-recognition-and-turn-taking-in-voice-agents-for-spanish-phone-negotiation-an-externally-measured-comparison-of-pipecat-livekit-and-gpt-live-137399.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19118>)

Author: Jovan Popoca Moran, Eduardo de Avila-Armenta, Ra\\'ul Alejandro Aquino-Ibarra, Jorge Octavio Nicol\\'as-D\\'iaz, Umberto Leon-Dominguez

Published: 2026-11-17T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [livekit](<https://devfeed.tech/topics/livekit.md>), [asr](<https://devfeed.tech/topics/asr.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [gpt-live](<https://devfeed.tech/tags/gpt-live.md>), [latency](<https://devfeed.tech/tags/latency.md>), [livekit](<https://devfeed.tech/tags/livekit.md>), [negotiation](<https://devfeed.tech/tags/negotiation.md>), [phone](<https://devfeed.tech/tags/phone.md>), [pipecat](<https://devfeed.tech/tags/pipecat.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

In simulated Spanish freight-rate calls, GPT-Live had the lowest median listener-measured response latency (2,444.8 ms), but its 171.2 ms advantage over Pipecat was not statistically conclusive; LiveKit measured 3,090.0 ms. Tool calls added 0.9-1.1 seconds across all three architectures, and framework-reported latency understated listener wait. GPT-Live also had lower word and amount error rates than the two cascades. With default settings, Pipecat and LiveKit often treated an overlapping "ok" as the caller's answer, sometimes recording agreement; no such case occurred for GPT-Live in the reported calls. The study recommends measuring latency from the listener's side and adding safeguards for amounts and backchannels in cascades. Results come from 77 calls per architecture using recordings from one speaker without a telephone network, so they do not establish performance in live phone negotiations.

### Source excerpt

arXiv:2611.19118v1 Announce Type: cross Abstract: A voice agent that negotiates by phone must answer quickly, understand spoken amounts and cope with a caller who talks over it. We compared three architectures for such an agent, negotiating freight rates in Spanish: two cascades (speech recognition, language model and speech synthesis) orchestrated by Pipecat and by LiveKit Agents, and GPT-Live, an OpenAI speech model that listens and speaks without separate stages. A program played the carrier with the same 47 recordings of one speaker for all agents, without the telephone network, in 77 calls per architecture over five negotiation scenarios, and measured latency on the listener's side with a validated instrument. Median latency to the content response was 2,616.0 ms (Pipecat), 3,090.0 ms (LiveKit) and 2,444.8 ms (GPT-Live); the confidence interval of GPT-Live's 171.2 ms advantage over Pipecat includes zero. According to its own timestamps, GPT-Live began to answer 800 ms after the end of the line, close to the 0.8 s published by OpenAI; most of the remaining wait, about 1,367 ms and nearly constant, is the audio's round trip through OpenAI's service. A tool call added 0.9-1.1 s in every architecture, more than any difference between them, and the frameworks' self-reported latency fell short of the listener's wait by 9% (Pipecat) and 22% (LiveKit). Word error rates were low (6.03, 4.93 and 1.47%), but the error on amounts was 13.01% and 12.61% in the cascades, against 2.86% in GPT-Live. With default settings, Pipecat and LiveKit stopped at almost every backchannel and often took it as the carrier's answer: an "ok" spoken over the agent ended in a registered agreement in 3 of 15 Pipecat calls, 2 of 15 LiveKit calls and no GPT-Live call (descriptive counts, not tested). Latency should therefore be measured on the listener's side, and a cascade needs safeguards for spoken amounts and backchannels.

## Bland Launches Tofu-1, a Full-Duplex Speech-to-Speech Model for Calls

DevFeed: [Bland Launches Tofu-1, a Full-Duplex Speech-to-Speech Model for Calls](<https://devfeed.tech/articles/bland-launches-tofu-1-a-full-duplex-speech-to-speech-model-for-calls-134510.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/11/15/bland-tofu-1-full-duplex-speech-to-speech-model/>)

Author: Michal Sutter

Published: 2026-11-15T10:06:21Z

Content type: news

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai-shorts](<https://devfeed.tech/tags/ai-shorts.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [audio-language-model](<https://devfeed.tech/tags/audio-language-model.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [full](<https://devfeed.tech/tags/full.md>), [harness](<https://devfeed.tech/tags/harness.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [model](<https://devfeed.tech/tags/model.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [phone](<https://devfeed.tech/tags/phone.md>), [preview](<https://devfeed.tech/tags/preview.md>), [research](<https://devfeed.tech/tags/research.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-speech](<https://devfeed.tech/tags/speech-to-speech.md>), [staff](<https://devfeed.tech/tags/staff.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

Bland's Tofu-1 is an enterprise research-preview model that processes and generates call audio directly, without an intervening transcript. Bland says it can track speakers, separate voices from background sounds, detect cues such as hesitation or urgency, and handle interruptions while speaking. The excerpt also describes scene generation, but says Bland's TV Test walkthrough is illustrative rather than recorded model output. Access is limited to some enterprises; banking, healthcare, insurance, and government are named as target sectors. Bland has not published benchmarks, model size, latency, or pricing, and the reported "over 92%" screening result is self-reported. Competitor figures cited in the article use differing FDB benchmark versions and are not directly comparable.

### Source excerpt

Bland has released Tofu-1, a full-duplex speech-to-speech model for noisy, high-stakes phone calls. It is in enterprise research preview, with no published benchmarks, size or pricing yet. The post Bland Launches Tofu-1, a Full-Duplex Speech-to-Speech Model for Calls appeared first on MarkTechPost.

## Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot

DevFeed: [Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot](<https://devfeed.tech/articles/cost-optimizing-assemblyai-speech-to-text-inference-on-aws-with-amazon-ec2-spot-132652.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/cost-optimizing-assemblyai-speech-to-text-inference-on-aws-with-amazon-ec2-spot/>)

Author: Belair Albahagwi

Published: 2026-11-13T19:35:35Z

Content type: news

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [cloud cost management](<https://devfeed.tech/topics/cloud-cost-management.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [amazon-sqs](<https://devfeed.tech/tags/amazon-sqs.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [assemblyai](<https://devfeed.tech/tags/assemblyai.md>), [aws](<https://devfeed.tech/tags/aws.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cost-savings](<https://devfeed.tech/tags/cost-savings.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [ec2](<https://devfeed.tech/tags/ec2.md>), [ecs](<https://devfeed.tech/tags/ecs.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [regions](<https://devfeed.tech/tags/regions.md>), [retry](<https://devfeed.tech/tags/retry.md>), [scale](<https://devfeed.tech/tags/scale.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [spot](<https://devfeed.tech/tags/spot.md>), [sqs](<https://devfeed.tech/tags/sqs.md>)

### AI overview

AssemblyAI reports 70% lower inference compute costs than On-Demand pricing using GPU Spot Instances, while maintaining SLA commitments. Its Amazon ECS clusters combine an always-on On-Demand baseline with diversified Spot capacity across g4dn, g5, and g6 GPU families and Availability Zones. Amazon SQS decouples job submission from execution: tasks finish work within Spot's two-minute interruption notice or return unfinished jobs to the queue for another worker to retry. Custom images with model artifacts and containers take about two minutes to create, reducing recovery time. Independent deployments in each AWS Region keep audio within required geographic boundaries. The article says compute demand grew nearly fourfold year over year; interruptions may add processing time, and exhausted retries can be routed to a dead-letter queue for investigation.

### Source excerpt

Learn how AssemblyAI runs cost-optimized speech-to-text inference at scale on AWS, using Amazon ECS, Amazon EC2 Spot GPU instances, and an Amazon SQS-based retry architecture to achieve 70% cost savings while maintaining SLA commitments across European Regions.

## Speech Tokenizers are Vulnerable: Transferable Semantic Attack and Robust Tokenizer

DevFeed: [Speech Tokenizers are Vulnerable: Transferable Semantic Attack and Robust Tokenizer](<https://devfeed.tech/articles/speech-tokenizers-are-vulnerable-transferable-semantic-attack-and-robust-tokenizer-132067.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14929>)

Author: Zhisheng Zhang, Yifan Mi, Yixuan Zhou, Haiyun Li, Shengbo Cai, Dongfu Song, Ziang Xu, Jie Hao, Zhiyong Wu

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [asr](<https://devfeed.tech/tags/asr.md>), [attack](<https://devfeed.tech/tags/attack.md>), [audio](<https://devfeed.tech/tags/audio.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [based](<https://devfeed.tech/tags/based.md>), [black](<https://devfeed.tech/tags/black.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [box](<https://devfeed.tech/tags/box.md>), [building](<https://devfeed.tech/tags/building.md>), [can](<https://devfeed.tech/tags/can.md>), [cause](<https://devfeed.tech/tags/cause.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [robust](<https://devfeed.tech/tags/robust.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [speech](<https://devfeed.tech/tags/speech.md>), [tokenizers](<https://devfeed.tech/tags/tokenizers.md>)

### AI overview

Speech tokenizers can be disrupted by small perturbations, degrading downstream ASR and large audio-language model performance. The proposed T-SemAttack jointly shifts representations across an ensemble of semantic encoders while preserving perceptual quality. The authors report substantial drift in deep representations and directional alignment between surrogate-model input gradients, which they suggest may help explain transferability. ROSETok combines noisy training with robust semantic distillation to target reconstruction fidelity and downstream robustness. Experiments across multiple datasets, nine open-source and black-box ASR systems, and four LALMs report strong transferable attacks and robustness for ROSETok.

### Source excerpt

arXiv:2611.14929v1 Announce Type: cross Abstract: Speech tokenizers serve as speech front ends, encoding input speech signals into representations for downstream models, such as automatic speech recognition (ASR) systems and large audio-language models (LALMs). In this paper, we find that speech tokenizers are fragile at the semantic level: small perturbations can disrupt their semantic encoding process and consequently cause severe degradation in downstream ASR and LALM performance. To study this vulnerability, we propose T-SemAttack, a transferable semantic attack based on a semantic encoder ensemble. By jointly drifting the representation spaces of multiple semantic encoders, T-SemAttack effectively disrupts the semantic content of speech while preserving perceptual quality, thereby inducing error features. We further analyze layer-wise representation drift and cross-model gradient alignment. Our findings show that perturbations can induce substantial shifts in deep semantic representations and that the input gradients of different surrogate encoders exhibit directional alignment. These observations suggest that shared adversarially sensitive directions in the input space may be a key factor underlying cross-model attack transferability. Building on these, we introduce ROSETok, a robust speech tokenizer that combines noisy training with robust semantic distillation to improve reconstruction fidelity and downstream SpeechLLM robustness. Extensive experiments on multiple datasets, 9 open-source and black-box ASR systems, and 4 LALMs demonstrate that T-SemAttack achieves strong transferable attack performance, while the proposed Robust Speech Tokenizer exhibits robustness under high-fidelity reconstruction and downstream tasks. Our code and demo are available at https://wxzyd123.github.io/t-semattack.

## Top 6 AI Text to Speech Tools in 2026: Features, Strengths and Limits

DevFeed: [Top 6 AI Text to Speech Tools in 2026: Features, Strengths and Limits](<https://devfeed.tech/articles/top-6-ai-text-to-speech-tools-in-2026-features-strengths-and-limits-130705.md>)

Original publisher: [Read original article](<https://www.sitepoint.com/top-6-ai-text-to-speech-tools-in-2026-features-strengths-and-limits/>)

Author: Div John

Published: 2026-11-11T13:59:35Z

Content type: comparison

Language: en

Sources: [SitePoint](<https://devfeed.tech/sources/sitepoint.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [developer-tools](<https://devfeed.tech/tags/developer-tools.md>), [features](<https://devfeed.tech/tags/features.md>), [limits](<https://devfeed.tech/tags/limits.md>), [reading](<https://devfeed.tech/tags/reading.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text](<https://devfeed.tech/tags/text.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [tools](<https://devfeed.tech/tags/tools.md>), [web](<https://devfeed.tech/tags/web.md>)

### AI overview

There is no single best text-to-speech tool; the choice depends on whether you need expressive voiceovers, document listening, licensed corporate narration, or developer integration. OpenArt combines voice models with video and music tools and supports bracketed performance cues, but limits scripts to 10,000 characters. ElevenLabs offers expressive and multilingual models plus an API, with model-specific language support and credit costs to check. Murf AI provides studio controls and pronunciation libraries; its free plan lacks downloads and commercial rights. Speechify focuses on reading documents aloud across apps and formats. WellSaid Labs emphasizes consent-based, commercially licensed voices and shared team pronunciation controls, with sales-quoted pricing. Amazon Polly is API-focused, with SSML, custom lexicons, MP3/OGG output, and usage-based pricing. The excerpt advises checking current plans and licenses, testing with your own script, correcting names and numbers, and listening to the output before publishing.

### Source excerpt

null Continue reading Top 6 AI Text to Speech Tools in 2026: Features, Strengths and Limits on SitePoint.

## TransferBreaker: Breaking Adversarial Transferability in Fine-Tuned Speech Recognition

DevFeed: [TransferBreaker: Breaking Adversarial Transferability in Fine-Tuned Speech Recognition](<https://devfeed.tech/articles/transferbreaker-breaking-adversarial-transferability-in-fine-tuned-speech-recognition-128584.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.09109>)

Author: Mojtaba Nafez, Aref Mousavi, Mohammad Ebrahim Mahdavi, Mobina Poulaei, Kiarash Kiani Feriz, Mohammad Hossein Rohban

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [invariance](<https://devfeed.tech/tags/invariance.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [performance](<https://devfeed.tech/tags/performance.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [safety](<https://devfeed.tech/tags/safety.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

Adversarial audio perturbations crafted against a public pretrained ASR model can transfer to fine-tuned target models, undermining the protection expected from black-box deployment. TransferBreaker combines Base Adversarial Fine-Tuning, which limits training to perturbations effective on the base model; Latent Jacobian Regularization, which suppresses adversarially sensitive latent directions; and HybridGrad-AFT, which interpolates perturbations from base and target gradients to address adaptive attacks. The abstract reports evaluation across three languages and four large ASR models, with adversarial word error rate reduced from 92.6 to 27.8. It does not specify the evaluation setup or variation across models and languages.

### Source excerpt

arXiv:2610.09109v2 Announce Type: replace Abstract: Many organizations fine-tune publicly available pretrained Automatic Speech Recognition (ASR) models and deploy them in black-box settings, assuming limited access provides protection. We show this assumption is fragile: adversarial perturbations crafted on the public base model transfer effectively to fine-tuned target models, severely degrading performance and posing concerns for safety-critical applications. We propose TransferBreaker, a unified fine-tuning framework that suppresses adversarial transfer by integrating Base Adversarial Fine-Tuning, which restricts adversarial training to base-effective perturbations; Latent Jacobian Regularization, which enforces latent-space invariance by suppressing adversarially sensitive directions; and HybridGrad-AFT, which improves robustness against adaptive attacks by interpolating transferable perturbations from base and target gradients. We theoretically justify all components and evaluate TransferBreaker across three languages and four large ASR models, reducing adversarial WER from 92.6 to 27.8. Our code is publicly available at https://github.com/rohban-lab/TransferBreaker.

## Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts

DevFeed: [Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts](<https://devfeed.tech/articles/unified-synthesis-of-compositional-speech-and-sound-from-free-form-text-prompts-128118.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.28063>)

Author: Yuyue Wang, Xihua Wang, Xin Cheng, Yijing Chen, Ruihua Song

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Generative AI](<https://devfeed.tech/topics/generative-ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [coherent](<https://devfeed.tech/tags/coherent.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-mm](<https://devfeed.tech/tags/cs-mm.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [events](<https://devfeed.tech/tags/events.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [fast](<https://devfeed.tech/tags/fast.md>), [flexibility](<https://devfeed.tech/tags/flexibility.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generator](<https://devfeed.tech/tags/generator.md>), [independent](<https://devfeed.tech/tags/independent.md>), [intermediate](<https://devfeed.tech/tags/intermediate.md>), [large](<https://devfeed.tech/tags/large.md>), [speech](<https://devfeed.tech/tags/speech.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [text](<https://devfeed.tech/tags/text.md>)

### AI overview

PlanAudio generates speech and sound together directly from free-form text, without first decomposing or rewriting the prompt. It uses a single autoregressive framework that predicts a continuous audio layout before synthesizing acoustic tokens. The authors also introduce a training corpus and the FUAG benchmark, covering speech, sound, and joint generation. In reported experiments, PlanAudio outperforms pipeline and unified baselines on joint speech-and-sound generation and remains competitive with specialist models for speech or sound alone. The abstract says audio layout planning performs better than textual planning, acoustic planning, and direct generation; it provides no numerical results or evaluation details.

### Source excerpt

arXiv:2605.28063v2 Announce Type: replace-cross Abstract: Generating unified audio that naturally integrates speech and sounds from free-form text prompts remains challenging. For example, the prompt ``A fast drum roll ends with a cymbal crash, after which a man shouts: `Come on!'.'' describes several events that should appear in order and form a coherent recording rather than independent tracks that lack coherent temporal and semantic interactions. Current methods broadly follow two paradigms: pipeline methods couple separate text-to-sound and text-to-speech models, while unified models rely on a single generator that still requires structured, multi-field prompt inputs. Both paradigms require upstream prompt decomposition or rewriting, limiting free-form flexibility and risking cascading errors. To enable direct generation from free-form prompts within a single model, we introduce PlanAudio, an end-to-end framework that maps unconstrained free-form text to unified audio without intermediate prompt preprocessing. To address the challenges of direct free-form generation, PlanAudio integrates latent planning into a single autoregressive framework, predicting continuous audio layout before acoustic token synthesis. We construct a large-scale training corpus and introduce Free-Form Unified Audio Generation Benchmark (FUAG) for evaluation, both spanning speech, sound, and joint speech-and-sound generation. Experiments show that a single PlanAudio outperforms existing pipeline and unified baselines on speech-and-sound generation, while remaining competitive with specialist models on speech and sound generation. Further analysis validates the effectiveness of audio layout planning over textual planning, acoustic planning, and direct generation. Sample Available: https://planaudio.github.io/PlanAudio2026/.

## Nepali-English Code-Mixed Speech Recognition: Data Preparation and Adaptation of a Speech Foundation Model

DevFeed: [Nepali-English Code-Mixed Speech Recognition: Data Preparation and Adaptation of a Speech Foundation Model](<https://devfeed.tech/articles/nepali-english-code-mixed-speech-recognition-data-preparation-and-adaptation-of-a-speech-foundation-model-128321.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13512>)

Author: Jeevan Neupane, Janardan Bhetwal, Kshitiz Paudel, Anup Aryal, Bipin Bashyal

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Whisper](<https://devfeed.tech/topics/whisper.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>)

### AI overview

Full fine-tuning of SraVaani on a 172.8-hour, six-domain Nepali speech corpus reduced test WER from 48.6% to 17.1%; a 12-epoch run reached 16.3%, close to fine-tuned Whisper large-v3-turbo at 16.4%. To handle the model tokenizer's lack of support for digits and common symbols, the study generates Nepali and English number readings and uses zero-shot ASR output to choose a plausible form. Freezing the encoder worsened WER to 25.5%, and excluding numeric clips raised WER on numeric test speech from 14.3% to 21.3%. FLEURS evaluation with original and mixed-script references indicates transcription conventions account for some, but not all, out-of-domain performance loss.

### Source excerpt

arXiv:2611.13512v1 Announce Type: new Abstract: Nepali speakers regularly mix English words into conversation, but multilingual speech recognisers often struggle with language switches and the writing script used for English words. We study how to adapt SraVaani, a 449M-parameter hybrid TDT-CTC model, using a 172.8-hour corpus covering six Nepali speech domains. Its pretrained tokenizer does not support digits and several common symbols. Rather than replace it or discard these recordings, we generate Nepali and English readings for numbers and use zero-shot ASR output to select a plausible form. Full fine-tuning reduces test word error rate (WER) from 48.6% to 17.1%; a 12-epoch run reaches 16.3%, compared with 16.4% for fine-tuned Whisper large-v3-turbo. Freezing the encoder increases WER to 25.5%, while excluding numeric clips raises WER on numeric test speech from 14.3% to 21.3%. We also evaluate on FLEURS using both its original references and mixed-script alternatives. The comparison shows that transcription conventions explain part, but not all, of the out-of-domain performance loss.

## Speech Processing Pioneer Bishnu Saroop Atal Dies at 93

DevFeed: [Speech Processing Pioneer Bishnu Saroop Atal Dies at 93](<https://devfeed.tech/articles/speech-processing-pioneer-bishnu-saroop-atal-dies-at-93-127672.md>)

Original publisher: [Read original article](<https://spectrum.ieee.org/speech-processing-pioneer-in-memoriam>)

Author: Amanda Davis

Published: 2026-11-10T19:00:02Z

Content type: news

Language: en

Sources: [IEEE Spectrum](<https://devfeed.tech/sources/ieee-spectrum.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [communications](<https://devfeed.tech/topics/communications.md>)

Tags: [2013](<https://devfeed.tech/tags/2013.md>), [acoustics](<https://devfeed.tech/tags/acoustics.md>), [at-t](<https://devfeed.tech/tags/at-t.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [bell-labs](<https://devfeed.tech/tags/bell-labs.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [careers](<https://devfeed.tech/tags/careers.md>), [cellular](<https://devfeed.tech/tags/cellular.md>), [cellular-networks](<https://devfeed.tech/tags/cellular-networks.md>), [coding](<https://devfeed.tech/tags/coding.md>), [communications](<https://devfeed.tech/tags/communications.md>), [computing](<https://devfeed.tech/tags/computing.md>), [contributions](<https://devfeed.tech/tags/contributions.md>), [digital](<https://devfeed.tech/tags/digital.md>), [digital-communications](<https://devfeed.tech/tags/digital-communications.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [electrical-engineering](<https://devfeed.tech/tags/electrical-engineering.md>), [ieee-member-news](<https://devfeed.tech/tags/ieee-member-news.md>), [in-memoriam](<https://devfeed.tech/tags/in-memoriam.md>), [obits](<https://devfeed.tech/tags/obits.md>), [obituaries](<https://devfeed.tech/tags/obituaries.md>), [pioneer](<https://devfeed.tech/tags/pioneer.md>), [processing](<https://devfeed.tech/tags/processing.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-processing](<https://devfeed.tech/tags/speech-processing.md>), [the-institute](<https://devfeed.tech/tags/the-institute.md>), [type-ti](<https://devfeed.tech/tags/type-ti.md>)

### AI overview

Bishnu Saroop Atal's work helped make high-quality voice transmission more bandwidth-efficient. At Bell Labs, he helped develop linear predictive coding (LPC), widely used for low-bit-rate speech encoding, and multipulse LPC, which improved compressed speech quality. With Manfred R. Schroeder, he developed code-excited linear prediction (CELP), which produced high-quality speech at very low bit rates. CELP became a foundational speech-coding technique; its bandwidth savings helped cellular networks carry more simultaneous calls. Atal later led Bell's acoustics and speech research departments, continued research at AT&T and the University of Washington, and received the 2013 IEEE Jack S. Kilby Signal Processing Medal for contributions to speech coding.

### Source excerpt

Bishnu Saroop Atal Acoustics and signal processing pioneer Life Fellow, 93; died 19 August Atal was an acoustics and signal processing researcher at Bell Labs in Murray Hill, N.J., and later at AT&T's Shannon Laboratory, in Florham Park, N.J. His pioneering work on voice-coding techniques became the foundation for efficient, high-quality voice transmission over cellular networks. He received the 2013 IEEE Jack S. Kilby Signal Processing Medal for "fundamental and pioneering contributions to speech coding." He joined Bell Labs in 1961 as a communications researcher, and his early work there focused on speaker identification technology. During the 1960s he helped develop linear predictive coding, a powerful method for analyzing and compressing speech. LPC is now the most widely used method for encoding speech at a low bit rate. Atal developed increasingly sophisticated low-bit-rate speech-coding techniques. They included multipulse LPC, which improved the quality of compressed speech. Bishnu Saroop AtalNokia Bell Labs He also worked with his colleague Manfred R. Schroeder to develop code-excited linear prediction, a method for producing high-quality speech at very low bit rates. CELP became a foundational technique in modern speech coding, letting digital communications systems carry voice using less bandwidth. The extra efficiency expanded cellular capacity, allowing wireless networks to handle more simultaneous calls. Later in his career, he led the acoustics and speech research departments at Bell. The company named him a Fellow in 1994. When AT&T Bell Laboratories split into separate research organizations in 1996, Atal continued his career at the AT&T Florham Park headquarters, and he was named a Fellow the following year. He retired in 2002 and became an affiliate professor of electrical engineering at the University of Washington, in Seattle. There he continued his research in speech processing, speech recognition, and high-dimensional computing. Atal was elect

## Representational Alignment between Deep Neural Networks and Human Brain in Speech Processing under Audiovisual Noise

DevFeed: [Representational Alignment between Deep Neural Networks and Human Brain in Speech Processing under Audiovisual Noise](<https://devfeed.tech/articles/representational-alignment-between-deep-neural-networks-and-human-brain-in-speech-processing-under-audiovisual-noise-126255.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12291>)

Author: Cheng Luo, Jian Zhu, Zhoujian Sun, Yunyi Qi, Yu Qi, Wenhui Sun, Wenyuan Yu

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [after](<https://devfeed.tech/tags/after.md>), [align](<https://devfeed.tech/tags/align.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [applying](<https://devfeed.tech/tags/applying.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [asr](<https://devfeed.tech/tags/asr.md>), [brain](<https://devfeed.tech/tags/brain.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-neural-networks](<https://devfeed.tech/tags/deep-neural-networks.md>), [human-brain](<https://devfeed.tech/tags/human-brain.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural](<https://devfeed.tech/tags/neural.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [noise](<https://devfeed.tech/tags/noise.md>), [processing](<https://devfeed.tech/tags/processing.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-processing](<https://devfeed.tech/tags/speech-processing.md>), [under](<https://devfeed.tech/tags/under.md>)

### AI overview

The study reports that a pretrained automatic speech recognition model's shallow-layer embeddings align more closely with brain responses to acoustic features, while deeper-layer embeddings align more closely with responses to linguistic features. Researchers used neural encoding models for speech and noise envelopes, word onset, and surprisal, then applied representational similarity analysis to compare those responses with model embeddings before and after fine-tuning on audiovisual noisy data. Fine-tuning strengthened the reported alignment: shallow layers showed improved noise processing and deeper layers more refined linguistic representations. These findings suggest the model can develop brain-like patterns of speech processing in noise.

### Source excerpt

arXiv:2611.12291v1 Announce Type: new Abstract: Speech comprehension in the human brain is an incremental process that begins with acoustic processing and advances to linguistic processing. While recent studies have revealed that the hierarchy of deep neural networks (DNNs) correlates with the ascending auditory pathway, the exact nature of this DNN-brain alignment remains underexplored. In this study, we investigate how DNN representations align with the brain's acoustic-to-linguistic processing. Specifically, we employed neural encoding models to simulate neural responses to acoustic (i.e., speech and noise envelope) and linguistic features (i.e., word onset and surprisal). By applying representational similarity analysis (RSA), we quantified the similarity between these neural responses and the DNN embeddings generated by a pre-trained automatic speech recognition (ASR) model, both before and after fine-tuning on audiovisual noisy data. Our results demonstrate significant DNN-brain alignment: embeddings from shallow layers exhibit higher similarity to neural responses associated with acoustic features, while those from deeper layers align more closely with neural responses related to linguistic features. Importantly, the fine-tuning process enhances this alignment by improving noise processing in shallower layers and refining linguistic representations in deeper layers. These results suggest that fine-tuned DNN models can naturally develop human-like processing patterns in noisy environments, highlighting a functional alignment between the human brain and DNNs in speech representation.

## BARS: Beat-Adaptive Rap Synthesis via Flow Matching

DevFeed: [BARS: Beat-Adaptive Rap Synthesis via Flow Matching](<https://devfeed.tech/articles/bars-beat-adaptive-rap-synthesis-via-flow-matching-125506.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12061>)

Author: Taemin Cho, Hector Martel, Enyan Koh

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [audio](<https://devfeed.tech/tags/audio.md>), [coherent](<https://devfeed.tech/tags/coherent.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-mm](<https://devfeed.tech/tags/cs-mm.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generation](<https://devfeed.tech/tags/generation.md>), [masking](<https://devfeed.tech/tags/masking.md>), [rap](<https://devfeed.tech/tags/rap.md>), [rope](<https://devfeed.tech/tags/rope.md>), [speech](<https://devfeed.tech/tags/speech.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [tempo](<https://devfeed.tech/tags/tempo.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

BARS generates rap vocals with Flow Matching, conditioned on an instrumental beat, lyrics, and a short reference voice for zero-shot timbre control. Unlike the prior autoregressive systems described, it allows tempo and output length to vary independently of the beat, without explicit duration prediction or phoneme-level alignment. Lyrics are encoded with pretrained ByT5 and a jointly trained lightweight Transformer adapter, supporting slang, abbreviations, and numerals. Asymmetric Scaled RoPE in cross-attention is designed to improve lyric-audio sequence alignment under strong beat conditioning, which can otherwise lead to skipped or repeated tokens. Span masking enables lyric-level regeneration while preserving surrounding vocal style. The abstract reports high lyric fidelity, rhythmically coherent flow, and tempo control across varied styles and speeds.

### Source excerpt

arXiv:2611.12061v1 Announce Type: cross Abstract: We present BARS (Beat-Adaptive Rap Synthesis), a non-autoregressive rapping voice generation model based on Flow Matching, conditioned on a beat track (instrumental backing), lyrics, and a short reference voice for zero-shot timbre control. Unlike prior autoregressive rap synthesis, where rap pace and output duration are strictly determined by the beat, BARS enables independent control over the rap tempo, allowing flexible-length generation without explicit duration prediction or phoneme-level alignment. We encode lyrics using a pretrained ByT5 encoder with a lightweight Transformer adapter trained jointly with the model, without relying on speech-pretrained text encoders, enabling natural handling of non-standard orthography (e.g., slang, abbreviations) and numerals common in rap lyrics. To address alignment instabilities, such as token skipping or repetition, under strong beat conditioning, we propose Asymmetric Scaled RoPE for cross-attention, which introduces a positional structure to facilitate sequential lyric-audio alignment learning. Furthermore, the model supports partial re-generation via span masking, enabling lyric-level editing while preserving the surrounding vocal style. Experiments demonstrate that the proposed model achieves high lyric fidelity, natural rap flow that is rhythmically coherent with the beat, and robust tempo control across a wide range of rapping styles and speeds.

## Ninada: Sub-Clause Split-Graph VITS Streaming and Capability-Routed 24 kHz Speech Synthesis Across Twelve Scripts

DevFeed: [Ninada: Sub-Clause Split-Graph VITS Streaming and Capability-Routed 24 kHz Speech Synthesis Across Twelve Scripts](<https://devfeed.tech/articles/ninada-sub-clause-split-graph-vits-streaming-and-capability-routed-24-khz-speech-synthesis-across-twelve-scripts-127221.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10035>)

Author: Hemanth HM

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [audio](<https://devfeed.tech/tags/audio.md>), [bottlenecks](<https://devfeed.tech/tags/bottlenecks.md>), [browser](<https://devfeed.tech/tags/browser.md>), [capability](<https://devfeed.tech/tags/capability.md>), [checkpoint](<https://devfeed.tech/tags/checkpoint.md>), [client](<https://devfeed.tech/tags/client.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [covering](<https://devfeed.tech/tags/covering.md>), [cpus](<https://devfeed.tech/tags/cpus.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [dependency](<https://devfeed.tech/tags/dependency.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [desktop](<https://devfeed.tech/tags/desktop.md>), [graph](<https://devfeed.tech/tags/graph.md>), [scripts](<https://devfeed.tech/tags/scripts.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-synthesis](<https://devfeed.tech/tags/speech-synthesis.md>), [split](<https://devfeed.tech/tags/split.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>)

### AI overview

Ninada reduces VITS speech synthesis startup latency by splitting an ONNX model in memory into prior, flow, and windowed vocoder sessions that stream with 12-frame halos and raised-cosine crossfades. Its 24 kHz system covers English, Classical Sanskrit, and ten Indian languages. Capability routing uses HiFi-GAN specialists for desktop/WebGPU and smaller distilled iSTFTNet specialists for mobile WASM, with a zero-dependency Klatt fallback. On 480 held-out studio utterances, it reports 3.22 dB 0-8 kHz MCD overall (3.09 dB for mobile iSTFTNet) and 6.07 dB error at 8-12 kHz. Reported audible startup is 89-155 ms on desktop WebGPU; on a physical Google Pixel 10 Pro XL, streaming is 538-649 ms with no underruns at 0.75-0.76x real-time factor.

### Source excerpt

arXiv:2611.10035v1 Announce Type: new Abstract: End-to-end VITS text-to-speech models synthesize natural speech in a single forward pass, yet standard ONNX exports execute monolithically over an entire clause, delaying Time-To-First-Audio (TTFA) to several seconds and spiking activation memory. Client-side deployment across Indic scripts faces four further bottlenecks: 138.8 MB float32 checkpoints per language strain browser caches, mobile WebGPU stalls for 6-14 s compiling convolution shaders while still running slower than real time ($1.44\times$ RTF), small WebAssembly pthread pools stall at barriers on big.LITTLE mobile CPUs, and mobile Web Audio pipelines drop frames or mute output under AudioWorklet contention and iOS WKWebView rules. We present Ninada, an open-source 24 kHz TTS system covering English, Classical Sanskrit, and ten Indian languages (sa, en, hi, kn, mr, bn, ta, te, ml, gu, pa, or) across a capability-routed 48-checkpoint fleet: 24 studio HiFi-GAN v1 specialists (27.9 MB W8A32+FP16 ONNX) for desktop/WebGPU and 24 distilled c8c8_384 iSTFTNet specialists (21.7 MB ONNX, $2.64\times$ fewer vocoder MACs via an exact NOLA dual-window ConvTranspose1d inverse STFT with zero complex ops) for mobile WASM, plus a 0.0 MB zero-dependency Klatt fallback. At runtime, Ninada splits a single ONNX file in memory in $< 15$ ms into prior, flow, and windowed vocoder sub-sessions streaming 12-frame halos with raised-cosine crossfades. On $N = 480$ held-out 24 kHz studio utterances, Ninada achieves 3.22 dB 0-8 kHz MCD (3.09 dB mobile iSTFTNet; 1.33 dB 24-voice distillation MCD, vs. 5.17 dB for the 1,526.8 MB 16 kHz Meta MMS fleet) and 6.07 dB 8-12 kHz error, delivering 89-155 ms audible startup on desktop WebGPU and 538-649 ms zero-underrun streaming at 0.75-0.76x RTF on a physical Google Pixel 10 Pro XL.

## Text Grouping Changes Completion-Time Comparisons in Long-Form Simultaneous Speech Translation

DevFeed: [Text Grouping Changes Completion-Time Comparisons in Long-Form Simultaneous Speech Translation](<https://devfeed.tech/articles/text-grouping-changes-completion-time-comparisons-in-long-form-simultaneous-speech-translation-126284.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12501>)

Author: Hieu Minh Truong, Dung D. Le

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [machine-translation](<https://devfeed.tech/topics/machine-translation.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [asr](<https://devfeed.tech/topics/asr.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [availability](<https://devfeed.tech/tags/availability.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [change](<https://devfeed.tech/tags/change.md>), [changes](<https://devfeed.tech/tags/changes.md>), [compare](<https://devfeed.tech/tags/compare.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [completion](<https://devfeed.tech/tags/completion.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [effects](<https://devfeed.tech/tags/effects.md>), [english](<https://devfeed.tech/tags/english.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [gap](<https://devfeed.tech/tags/gap.md>), [history](<https://devfeed.tech/tags/history.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-translation](<https://devfeed.tech/tags/speech-translation.md>), [text](<https://devfeed.tech/tags/text.md>), [time](<https://devfeed.tech/tags/time.md>), [translation](<https://devfeed.tech/tags/translation.md>)

### AI overview

Grouping translation updates before synthesis changed StreamAtt's measured completion lead, with opposite effects across synthesizers: four-second grouping increased the mean lead from 106.6 to 229.1 seconds with XTTS-v2, but reduced it from 169.5 to 146.5 seconds with Piper. Both produced shorter audio; differences in playback idle accounted for the opposing results. The directions recurred on a separately selected twenty-talk panel. In continuing CosyVoice 3 sessions, grouping left paired audio unchanged and delayed completion on 19 of 20 blocks. The findings indicate that speech-timing comparisons should specify how text is grouped for synthesis as well as which synthesizer is used. Recognition and automatic word timing describe outputs but do not establish translation accuracy or intelligibility.

### Source excerpt

arXiv:2611.12501v1 Announce Type: cross Abstract: Does grouping translation updates before speech synthesis change the measured completion gap between two translators? We compare StreamAtt with its fixed-audio-history baseline on ten English-German talks. For each translation, we keep the intended text and its availability times fixed and vary how updates are grouped for synthesis. Four-second grouping increases StreamAtt's mean completion lead from 106.6 to 229.1 seconds with XTTS-v2, but reduces it from 169.5 to 146.5 seconds with Piper. Both produce shorter audio; different changes in playback idle account for the opposite effects. These directions recur on a separately selected twenty-talk panel. In continuing CosyVoice 3 sessions, grouping instead leaves paired audio unchanged and delays completion on nineteen of twenty blocks. A speech-timing comparison therefore needs to specify how text enters synthesis, not just which synthesizer is used. Recognition and automatic word timing describe the outputs, but do not establish translation accuracy or intelligibility.

## Underdog shows off a 32MB speech model in new demo

DevFeed: [Underdog shows off a 32MB speech model in new demo](<https://devfeed.tech/articles/underdog-shows-off-a-32mb-speech-model-in-new-demo-121061.md>)

Original publisher: [Read original article](<https://techcrunch.com/2026/11/06/underdog-shows-off-a-32mb-speech-model-in-new-demo/>)

Author: Ivan Mehta

Published: 2026-11-06T13:02:16Z

Content type: news

Language: en

Sources: [TechCrunch | AI News & Artificial Intelligence](<https://devfeed.tech/sources/techcrunch-ai-news-artificial-intelligence.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [addition](<https://devfeed.tech/tags/addition.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-companies](<https://devfeed.tech/tags/ai-companies.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [app](<https://devfeed.tech/tags/app.md>), [based](<https://devfeed.tech/tags/based.md>), [best](<https://devfeed.tech/tags/best.md>), [can](<https://devfeed.tech/tags/can.md>), [demo](<https://devfeed.tech/tags/demo.md>), [generate](<https://devfeed.tech/tags/generate.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [model](<https://devfeed.tech/tags/model.md>), [music](<https://devfeed.tech/tags/music.md>), [shows](<https://devfeed.tech/tags/shows.md>), [size](<https://devfeed.tech/tags/size.md>), [small](<https://devfeed.tech/tags/small.md>), [speech](<https://devfeed.tech/tags/speech.md>), [underdog](<https://devfeed.tech/tags/underdog.md>)

### AI overview

Underdog's 32 MB Woof model powers a browser demo that speaks text typed on a virtual typewriter and generates music through a tuner. The company says Woof has 31 million parameters and can run on most devices without an internet connection; the model is available to download. The article cautions that a model this small is unlikely to replace larger ones, while suggesting it could support local assistants or personalized podcasts. It also notes Underdog's separate 167 MB Bark transcription and dictation model for edge devices.

### Source excerpt

Despite its small size, Underdog's new model can generate both speech and music.

## Gabor Gabor: E-Gaborformer, a Strictly Causal Speech Recognizer with Gabor-Dictionary Convolutions and Delta-Rule Memory

DevFeed: [Gabor Gabor: E-Gaborformer, a Strictly Causal Speech Recognizer with Gabor-Dictionary Convolutions and Delta-Rule Memory](<https://devfeed.tech/articles/gabor-gabor-e-gaborformer-a-strictly-causal-speech-recognizer-with-gabor-dictionary-convolutions-and-delta-rule-memory-120702.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.08092>)

Author: Nir Krakowski, Tamir Cohen, Ohad Zohar, Chen Zarfati

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>)

Tags: [3](<https://devfeed.tech/tags/3.md>), [asr](<https://devfeed.tech/tags/asr.md>), [attention](<https://devfeed.tech/tags/attention.md>), [combines](<https://devfeed.tech/tags/combines.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [delta](<https://devfeed.tech/tags/delta.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [future](<https://devfeed.tech/tags/future.md>), [lookahead](<https://devfeed.tech/tags/lookahead.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [production](<https://devfeed.tech/tags/production.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [real-time-voice-agents](<https://devfeed.tech/tags/real-time-voice-agents.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [scale](<https://devfeed.tech/tags/scale.md>), [speech](<https://devfeed.tech/tags/speech.md>), [state](<https://devfeed.tech/tags/state.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [temporal](<https://devfeed.tech/tags/temporal.md>)

### AI overview

E-Gaborformer is a streaming speech-recognition encoder designed to avoid future-audio dependence and keep per-call memory constant. It combines Gated DeltaNet-2 recurrent layers with sliding-window attention in a 3:1 ratio and constructs temporal convolutions from a fixed dictionary of causal Gabor atoms. The abstract reports about 19 MB of streaming state per production call, including the prediction network, compared with over 1 GB after ten minutes for a same-size causal-attention encoder. In a controlled LibriSpeech study, it reached target accuracy up to 18% sooner and matched or beat E-Branchformer under the same front end and recipe. A 0.6B-parameter Hebrew-English transducer using the encoder had 28-31% lower English WER than same-size NVIDIA Nemotron 3.5 ASR Streaming on voice-agent traffic, and up to 20% lower Hebrew WER than the authors' prior system. The same weights supported strictly causal and 480 ms-lookahead decoding without reported accuracy loss; a two-class RNN-T lattice formulation reportedly trained 2.6 times faster with exact loss and gradients.

### Source excerpt

arXiv:2611.08092v1 Announce Type: new Abstract: Real-time voice agents need a speech recognizer that never waits for future audio and whose memory per call does not grow with the length of the call. We present E-Gaborformer, a strictly causal encoder that combines Gated DeltaNet-2 recurrent layers with sliding-window attention in a 3:1 ratio, and builds every temporal convolution from a fixed dictionary of causal Gabor atoms. Its streaming state is constant, about 19 MB per call at production scale including the prediction network, where a causal-attention encoder of the same size accumulates over 1 GB after ten minutes. In a controlled causal study on LibriSpeech, E-Gaborformer leaves the initial plateau earlier, reaches a given accuracy up to 18% sooner, and matches or beats an E-Branchformer trained with the identical front end and recipe. At production scale, a 0.6B-parameter Hebrew and English streaming transducer built on E-Gaborformer has 28 to 31% lower word error rate (WER) than NVIDIA's Nemotron 3.5 ASR Streaming, a model of the same size, on English voice-agent traffic, and up to 20% lower Hebrew WER than our previous production system. The same weights serve strictly causal and 480 ms-lookahead decoding with no loss in accuracy, and a two-class formulation of the RNN-T lattice trains the transducer 2.6x faster with exact loss and gradients.

## Audio8-TTS-Realistic-Air: Edge0's 0.6B Multilingual Text-to-Speech Model Explained

DevFeed: [Audio8-TTS-Realistic-Air: Edge0's 0.6B Multilingual Text-to-Speech Model Explained](<https://devfeed.tech/articles/audio8-tts-realistic-air-edge0-s-0-6b-multilingual-text-to-speech-model-explained-116234.md>)

Original publisher: [Read original article](<https://hackernoon.com/audio8-tts-realistic-air-edge0s-06b-multilingual-text-to-speech-model-explained?source=rss>)

Author: aimodels44

Published: 2026-11-04T08:54:59Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [ai](<https://devfeed.tech/tags/ai.md>), [api](<https://devfeed.tech/tags/api.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [audio8-vs-preview](<https://devfeed.tech/tags/audio8-vs-preview.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [business](<https://devfeed.tech/tags/business.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [content-creation](<https://devfeed.tech/tags/content-creation.md>), [cryptocurrency](<https://devfeed.tech/tags/cryptocurrency.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [features](<https://devfeed.tech/tags/features.md>), [model](<https://devfeed.tech/tags/model.md>), [multilingual-text-to-speech](<https://devfeed.tech/tags/multilingual-text-to-speech.md>), [specifications](<https://devfeed.tech/tags/specifications.md>), [speech](<https://devfeed.tech/tags/speech.md>), [technical](<https://devfeed.tech/tags/technical.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [text-to-speech-ai](<https://devfeed.tech/tags/text-to-speech-ai.md>), [text-to-speech-model](<https://devfeed.tech/tags/text-to-speech-model.md>), [tts](<https://devfeed.tech/tags/tts.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voice-cloning](<https://devfeed.tech/tags/voice-cloning.md>)

### AI overview

Audio8-TTS-Realistic-Air is Edge0's 0.6B multilingual speech model. It generates speech from text alone or clones a voice using clean reference audio and an exact transcript, and lists 23 languages--12 more than the earlier Preview release. Reported comparisons are mixed: on Seed-TTS, Realistic Air has a 1.25% error rate versus Preview's 1.23% and higher speaker similarity (0.704 versus 0.682); on the shared eight-language CV3 subset, its error rate is lower (4.56% versus 4.80%). Evaluations used different pipelines, so the figures do not establish a strictly matched ranking, and these metrics do not directly measure perceived realism. The model can be run through Transformers or a non-streaming, OpenAI-compatible SGLang Omni speech API. Transformers loading requires custom code via `trust_remote_code=True`; Python 3.10+ and a CUDA-capable GPU are recommended. Throughput and streaming performance were not benchmarked.

### Source excerpt

Learn how Audio8-TTS-Realistic-Air works, including its features, technical specifications, benchmarks, limitations, and voice cloning capabilities.

## Removing Speech, Keeping Activities: Speech Removal for Privacy-Preserving Acoustic Sensing in Assisted Living

DevFeed: [Removing Speech, Keeping Activities: Speech Removal for Privacy-Preserving Acoustic Sensing in Assisted Living](<https://devfeed.tech/articles/removing-speech-keeping-activities-speech-removal-for-privacy-preserving-acoustic-sensing-in-assisted-living-114001.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.02376>)

Author: Pavlos Nicolaou, Christos Efstratiou

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [fourier analysis](<https://devfeed.tech/topics/fourier-analysis.md>), [fourier transform](<https://devfeed.tech/topics/fourier-transform.md>), [Whisper](<https://devfeed.tech/topics/whisper.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Differential Privacy](<https://devfeed.tech/topics/differential-privacy.md>)

Tags: [acoustic-sensing](<https://devfeed.tech/tags/acoustic-sensing.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [audio](<https://devfeed.tech/tags/audio.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

The paper presents a U-Net front end that masks the short-time Fourier transform to suppress speech while preserving sounds for activity recognition. It is trained on synthetic mixtures of public sounds and speech, avoiding the need to record speech in target homes. At a speech level 5 dB above background, activity-recognition macro F1 after removal is 0.574 and 0.661 on two datasets, versus 0.233 and 0.178 without processing; the method outperforms speech enhancement, speech separation, and detector-gated muting across tested speech levels. Whisper recovers 5.2% of spoken words after removal, versus 89.0% unprocessed, while voice-activity-detected speech on real home recordings drops from 8.5% to 2.7%. Speech and speaker identity are reduced, not eliminated.

### Source excerpt

arXiv:2609.02376v2 Announce Type: replace-cross Abstract: Acoustic sensing can monitor the daily activities of older adults without cameras or wearables, yet the possibility of capturing private conversations remains a barrier to its deployment. The engineering application addressed in this paper is privacy-preserving acoustic monitoring of daily activities in assisted living. Our contribution in artificial intelligence is a speech-removal front end: a U-Net that masks the short-time Fourier transform of ambient audio and returns a waveform in which speech is suppressed and the sounds needed for activity recognition are kept. The model is trained only on synthetic mixtures of public sounds and speech, so no speech has to be recorded in the target homes. We evaluate it with group-disjoint cross-validation on environmental and domestic sound datasets mixed with reverberant speech, on speech from unseen corpora and on real home recordings, and measure privacy with three voice activity detectors, Whisper speech recognition and a speaker-verification attack. With speech 5 dB above the background, the macro-averaged F1 score of activity recognition is 0.574 and 0.661 after removal on the two datasets, compared with 0.233 and 0.178 without processing, and higher than that of speech enhancement, speech separation and detector-gated muting at every speech level. Whisper recovers 5.2% of the spoken words, compared with 89.0% without processing and a chance floor of 2.2%. On the real recordings, the speech proportion estimated by a voice activity detector falls from 8.5% to 2.7%. The method reduces, but does not eliminate, spoken content and speaker identity.

## C3: Speech-Guided KV Retrieval for Long-Context SpeechLLM Inference

DevFeed: [C3: Speech-Guided KV Retrieval for Long-Context SpeechLLM Inference](<https://devfeed.tech/articles/c3-speech-guided-kv-retrieval-for-long-context-speechllm-inference-114365.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.00333>)

Author: Dancheng Liu, Markus M\\"uller, Rupak Vignesh Swaminathan, Jing Liu, Srikanth Ronanki, Aram Galstyan

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [information retrieval](<https://devfeed.tech/topics/information-retrieval.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [audio](<https://devfeed.tech/tags/audio.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [budget](<https://devfeed.tech/tags/budget.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [errors](<https://devfeed.tech/tags/errors.md>), [generation](<https://devfeed.tech/tags/generation.md>), [index](<https://devfeed.tech/tags/index.md>), [inference](<https://devfeed.tech/tags/inference.md>), [key](<https://devfeed.tech/tags/key.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-bandwidth](<https://devfeed.tech/tags/memory-bandwidth.md>), [memory-scaling](<https://devfeed.tech/tags/memory-scaling.md>), [merge](<https://devfeed.tech/tags/merge.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [speech](<https://devfeed.tech/tags/speech.md>)

### AI overview

C³ targets the KV-cache costs of long-form SpeechLLM generation, where audio's higher encoding rate makes cache memory, bandwidth, and decoding access costly. The authors report that retrieval errors can compound into generation drift and identify uneven attention concentration across query heads, non-contiguous audio support, and continuity across decoding steps as relevant patterns. C³ indexes audio KVs hierarchically, gives query heads sharing a KV head a common retrieval budget, and adjusts retrieval resolution with paired split-merge updates. Across recognition, translation, and summarization on four SpeechLLMs, it reports the strongest overall performance among retrieval methods at comparable final-KV access budgets, with larger gains under tight budgets, alongside improved decoding efficiency and accelerator-memory scaling.

### Source excerpt

arXiv:2611.00333v1 Announce Type: new Abstract: Speech language models encode audio at a much higher rate than text, causing the KV cache to dominate the memory, bandwidth, and decoding cost of long-form generation. KV retrieval can reduce per-step access by attending to only a query-dependent subset of the cache, but existing methods are designed primarily for text and do not account for the structure of audio attention. We first show that retrieval errors may compound autoregressively into generation drift. We then analyze full audio attention during long-form generation and identify three properties relevant to retrieval: unequal attention concentration across query heads, non-contiguous audio support, and temporal continuity across decoding steps. Guided by these observations, we propose $C^3$(Cluster-Compete-Compete), which organizes audio KVs in a hierarchical key-space index, lets query heads sharing a KV head compete for a common retrieval budget, and adaptively reallocates retrieval resolution through paired split-merge updates. Across long-form speech recognition, translation, and summarization on four SpeechLLMs, $C^3$ achieves the strongest overall performance among retrieval methods under comparable final-KV access budgets, with particularly large gains in the tight-budget regime, while improving long-context decoding efficiency and accelerator-memory scaling.

## Variational decomposition autoencoding improves disentanglement of latent representations

DevFeed: [Variational decomposition autoencoding improves disentanglement of latent representations](<https://devfeed.tech/articles/variational-decomposition-autoencoding-improves-disentanglement-of-latent-representations-111366.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2601.06844>)

Author: Ioannis N. Ziogas, Aamna Al Shehhi, Ahsan H. Khandoker, Leontios J. Hadjileontiadis

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [VAE](<https://devfeed.tech/topics/vae.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>)

Tags: [architectures](<https://devfeed.tech/tags/architectures.md>), [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [biomedical](<https://devfeed.tech/tags/biomedical.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generative](<https://devfeed.tech/tags/generative.md>), [human-computer-interaction](<https://devfeed.tech/tags/human-computer-interaction.md>), [learning](<https://devfeed.tech/tags/learning.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural](<https://devfeed.tech/tags/neural.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [signal](<https://devfeed.tech/tags/signal.md>), [signal-processing](<https://devfeed.tech/tags/signal-processing.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [structure](<https://devfeed.tech/tags/structure.md>), [temporal](<https://devfeed.tech/tags/temporal.md>)

### AI overview

Variational decomposition autoencoding (VDA) builds signal-decomposition structure into variational representation learning. Its DecVAEs are encoder-only neural networks that autoencode distributions into multiple latent subspaces aligned with time-frequency characteristics. On simulated data and three public datasets covering speech recognition, dysarthric speech evaluation, and emotional speech classification, DecVAEs reportedly improve disentanglement quality and robustness over VAE-based methods. Subspaces become factor-specific when generative factors have distinct spectral signatures, so the evidence supports potential use for signals with that property; broader applications are proposed rather than established here.

### Source excerpt

arXiv:2601.06844v2 Announce Type: replace Abstract: Understanding the structure of complex, nonstationary, high-dimensional time-evolving signals is a central challenge in scientific data analysis. In many domains, such as speech and biomedical signal processing, the ability to learn disentangled and interpretable representations is critical for uncovering latent generative mechanisms. Traditional approaches to unsupervised representation learning, including variational autoencoders (VAEs), often struggle to capture the temporal and spectral diversity inherent in such data. Here we introduce variational decomposition autoencoding (VDA), a framework that incorporates a strong structural bias toward signal decomposition into variational representation learning. VDA is instantiated through variational decomposition autoencoders (DecVAEs), i.e., encoder-only neural networks that perform latent autoencoding at the level of distributions to learn multiple latent subspaces aligned with time-frequency characteristics. We evaluate DecVAEs on simulated data and three publicly available scientific datasets, spanning speech recognition, dysarthric speech evaluation, and emotional speech classification. DecVAEs improve on VAE-based methods in disentanglement quality and robustness, and their latent subspaces become factor-specific when the generative factors have distinct spectral signatures. These findings position decomposition-aware architectures as potential tools for extracting structured representations from speech, and potentially from other dynamic signals with factors exhibiting distinct spectral signatures, with applications in clinical diagnostics, human-computer interaction, and adaptive neurotechnologies.

## Incongruence-Aware Evaluation of Multimodal Speech Emotion Recognition in Audio Large Language Models

DevFeed: [Incongruence-Aware Evaluation of Multimodal Speech Emotion Recognition in Audio Large Language Models](<https://devfeed.tech/articles/incongruence-aware-evaluation-of-multimodal-speech-emotion-recognition-in-audio-large-language-models-111985.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.40883>)

Author: Fengji Jing, Guy Laban

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [code](<https://devfeed.tech/tags/code.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-mm](<https://devfeed.tech/tags/cs-mm.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [data](<https://devfeed.tech/tags/data.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [emotion](<https://devfeed.tech/tags/emotion.md>), [error](<https://devfeed.tech/tags/error.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large](<https://devfeed.tech/tags/large.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [speech](<https://devfeed.tech/tags/speech.md>)

### AI overview

The study reports that disagreement between spoken words and vocal delivery predicts speech emotion recognition errors for audio-capable large language models across acted, conversational, and spontaneous speech. It maps text and audio to a shared valence-arousal-dominance space and uses a threshold-free incongruence score based on a two-component Gaussian mixture, validated against human judgments. Across eNTERFACE'05, MELD, and CMU-MOSEI, greater incongruence raises error odds when audio is included, even after controlling for ground-truth emotion and feature-space distance. Supervised classifiers using the same features show the opposite pattern, suggesting the challenge may lie in how the models combine evidence. Text-only effects are inconsistent and can reverse when the transcript-reading model changes. The central result persists without the emotion lexicon; among six simpler alternatives, only this label-free score predicts errors in all three datasets.

### Source excerpt

arXiv:2610.40883v1 Announce Type: new Abstract: Voice-interactive large language models process both what a speaker says and how they say it, yet the two channels do not always agree. We ask whether that disagreement degrades speech emotion recognition. Mapping text and audio into a shared valence-arousal-dominance space, we convert the distance between them into a continuous, threshold-free incongruence score with a two-component Gaussian mixture, and validate the score against human judgement. We evaluate three audiocapable models on acted, conversational and spontaneous speech (eNTERFACE'05, MELD and CMU-MOSEI). With the model held constant across modalities, incongruence raises the odds of error whenever audio is present, in every corpus, after controlling for the ground-truth emotion and for distance in the feature space. Supervised classifiers trained on the same features behave in the opposite way, which suggests that the difficulty lies in how these models combine raw evidence rather than in the signal itself. Text-only effects are unreliable: swapping only the model that reads the transcript reverses their sign on one corpus. Rebuilding the score without the emotion lexicon reproduces the central result, and among six simpler alternatives it is the only label-free score that predicts error in all three corpora. Code, data tables and prompts are released.

[Next page](<https://devfeed.tech/topics/speech-processing.md?cursor=WyIyMDI2LTExLTAyVDA1OjAwOjAwKzAwOjAwIiwgIjQ1ZGI1YzU5LTcxZGEtNDJiNy1iNTRjLWVlZDg4NWI5OTkxNSJd>)