# voice ai

Technology that recognizes spoken audio as text and can synthesize text as spoken audio.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Rhythm-TTS: High-Fidelity Shortcut Latent Modeling and Probabilistic Rhythm Generation for Zero-Shot Text-to-Speech

DevFeed: [Rhythm-TTS: High-Fidelity Shortcut Latent Modeling and Probabilistic Rhythm Generation for Zero-Shot Text-to-Speech](<https://devfeed.tech/articles/rhythm-tts-high-fidelity-shortcut-latent-modeling-and-probabilistic-rhythm-generation-for-zero-shot-text-to-speech-145308.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2510.02848>)

Author: Hieu-Nghia Huynh-Nguyen, Ngoc-Son Nguyen, Long-Khanh Pham Huynh Nguyen Dang, Huy Tien Nguyen, Van Nguyen

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [audio](<https://devfeed.tech/tags/audio.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [benefit](<https://devfeed.tech/tags/benefit.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [changes](<https://devfeed.tech/tags/changes.md>), [code](<https://devfeed.tech/tags/code.md>), [codec](<https://devfeed.tech/tags/codec.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [generation](<https://devfeed.tech/tags/generation.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text](<https://devfeed.tech/tags/text.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [tts](<https://devfeed.tech/tags/tts.md>), [zero](<https://devfeed.tech/tags/zero.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

Rhythm-TTS aims to improve zero-shot speech fidelity and pacing diversity by combining discrete-token alignment with continuous acoustic generation. It treats predicted codec tokens as a semantic-acoustic prior, then uses flow matching to generate continuous speech latents directly, bypassing discrete-to-latent reconstruction at inference. Probabilistic Duration and Silence Generators sample phoneme durations and pauses before acoustic generation to vary rhythm across runs. The abstract reports improvements in fidelity and temporal naturalness on zero-shot TTS benchmarks, which evaluate intelligibility, naturalness, speaker similarity, efficiency, and rhythm diversity; it gives no numerical results in the supplied text.

### Source excerpt

arXiv:2510.02848v2 Announce Type: replace-cross Abstract: Zero-shot text-to-speech (TTS) systems are commonly built on either discrete codec tokens or continuous acoustic representations. Discrete-token models benefit from stable sequence modeling and explicit speech factorization, but they ultimately synthesize speech through codebook lookup and codeword composition, which can accumulate errors and impose a quality ceiling on naturalness and speaker similarity. Continuous-representation models can generate fine-grained speech latents directly, but often require heavier in-context modeling and less explicit alignment. This paper presents \textbf{Rhythm-TTS}, a shortcut zero-shot TTS framework that combines the factorizability of discrete codec tokens with the fidelity of continuous latent generation. Instead of using predicted codec tokens as the final acoustic representation, Rhythm-TTS uses them as a semantic-acoustic prior and directly generates holistic continuous speech latents with flow matching, bypassing the discrete-to-latent reconstruction path at inference time. We further address an underexplored dimension of speech naturalness: rhythm diversity across multiple synthesis runs. Motivated by the hypothesis that rhythm variation is largely governed by phoneme duration changes and the occurrence of silent pauses, we introduce probabilistic Duration and Silence Generators to sample dynamic pacing patterns before acoustic generation. Experiments on zero-shot TTS benchmarks evaluate intelligibility, naturalness, speaker similarity, efficiency, and run-to-run rhythm diversity, showing that the proposed shortcut latent modeling and probabilistic rhythm generation improve the fidelity and temporal naturalness of non-autoregressive zero-shot TTS. Code and audio samples are available at our demo page https://rhythm-tts.github.io.

## AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion

DevFeed: [AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion](<https://devfeed.tech/articles/audioturbo-fast-text-to-audio-generation-with-rectified-diffusion-145302.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2505.22106>)

Author: Junqi Zhao, Jinheng Zhao, Haohe Liu, Yun Chen, Lu Han, Haoyang Zhang, Xubo Liu, Mark Plumbley, Wenwu Wang

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Generative AI](<https://devfeed.tech/topics/generative-ai.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>)

Tags: [acceleration](<https://devfeed.tech/tags/acceleration.md>), [audio](<https://devfeed.tech/tags/audio.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [generation](<https://devfeed.tech/tags/generation.md>), [inference-speed](<https://devfeed.tech/tags/inference-speed.md>), [tta](<https://devfeed.tech/tags/tta.md>)

### AI overview

AudioTurbo accelerates text-to-audio generation by integrating rectified diffusion into pre-trained diffusion models. It learns first-order approximate ODE paths from deterministic noise-sample pairs produced by a pre-trained model, making use of those models rather than relying on a dedicated flow-based generative framework. The reported experiments find that AudioTurbo exceeds a flow-matching-based acceleration model in both generation quality and inference speed; the excerpt provides no quantitative results or further implementation details.

### Source excerpt

arXiv:2505.22106v2 Announce Type: replace-cross Abstract: Diffusion models have significantly improved the quality and diversity of text-to-audio (TTA) generation but are hindered by slow inference speed. Recent flow-based approaches have been explored for accelerating TTA generation. However, these approaches typically rely on dedicated flow-based generative frameworks and do not directly exploit existing pre-trained diffusion-based TTA models. To address this limitation while leveraging the strong generative capabilities of pre-trained diffusion models, we integrate rectified diffusion into these models to improve TTA generation efficiency. Specifically, we propose AudioTurbo, which learns first-order approximate ordinary differential equation (ODE) paths from deterministic noise-sample pairs generated by a pre-trained TTA model. Experiments show that our approach outperforms a flow-matching-based acceleration model in both generation quality and inference speed.

## ProTTS: End-to-End Prominence-aware Natural and Controllable Text-to-Speech via Automatic Word Stress Discovery

DevFeed: [ProTTS: End-to-End Prominence-aware Natural and Controllable Text-to-Speech via Automatic Word Stress Discovery](<https://devfeed.tech/articles/protts-end-to-end-prominence-aware-natural-and-controllable-text-to-speech-via-automatic-word-stress-discovery-138945.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19832>)

Author: Sai Harshitha Aluru, Chiranjeevi Yarra

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [energy](<https://devfeed.tech/tags/energy.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [features](<https://devfeed.tech/tags/features.md>), [framework](<https://devfeed.tech/tags/framework.md>), [inference](<https://devfeed.tech/tags/inference.md>), [model](<https://devfeed.tech/tags/model.md>), [speech](<https://devfeed.tech/tags/speech.md>), [stress](<https://devfeed.tech/tags/stress.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [text](<https://devfeed.tech/tags/text.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [text-to-speech-tts](<https://devfeed.tech/tags/text-to-speech-tts.md>), [training](<https://devfeed.tech/tags/training.md>), [word](<https://devfeed.tech/tags/word.md>)

### AI overview

ProTTS combines automatic word-stress discovery and speech synthesis in one FastSpeech_2-based model. A clustering head learns stress from prosodic features without stress labels; the discovered stress guides predictors that adjust pitch, energy, and duration. At inference, users can supply a custom stress mask to control emphasis. The abstract reports a listener MOS of 3.96 versus 3.12 for the baseline and an 83.5% preference rate. Stress-detection F1 is 0.839/0.861 for German/Italian L2 speech on ISLE, compared with 0.70 for an unsupervised and 0.82 for a supervised baseline; zero-shot transfer to Tatoeba reaches 0.933 F1. These are abstract-level results across the stated datasets and benchmarks.

### Source excerpt

arXiv:2611.19832v1 Announce Type: cross Abstract: Natural speech conveys meaning not only through words, but also through the prominence on a few words. Yet most text-to-speech (TTS) systems remain prominence-unaware, existing prominence-aware TTS require external stress annotations, while unsupervised stress detectors operate in isolation, disconnected from synthesis. This creates a fundamental gap no existing system can automatically discover where stress should be placed, realize it as natural-sounding prosody, and offer user control over emphasis placement, all within a single model. To circumvent this, we propose ProTTS, a unified framework where stress detection emerges from the TTS model itself and, in turn, guides it to synthesize more natural and controllable speech. We incorporate a clustering head into the variance adaptor of FastSpeech_2 that discovers word stress directly from aggregated prosodic features. The discovered stress then drives emphasis variance predictors that modify pitch, energy, and duration for stressed words. The framework is trained end-to-end: TTS reconstruction and clustering losses jointly shape a stress-discriminative feature space without requiring stress labels at any stage. At inference, the same architecture also supports user-controllable emphasis by accepting a custom stress mask. We evaluate ProTTS on ISLE (non-native), LibriTTS, VCTK (native training), and three evaluation benchmarks (Tatoeba, TinyStress-15k, EmphAssess). Listeners prefer ProTTS over the baseline with a MOS of 3.96 vs. 3.12 and 83.5% preference rate. For stress detection, ProTTS achieves 0.839/0.861 F1 on ISLE (German/Italian L2), outperforming unsupervised (0.70) and supervised (0.82) baselines, and 0.933 F1 in zero-shot transfer to Tatoeba.

## Robots That Talk Too Much and Don't Listen Enough

DevFeed: [Robots That Talk Too Much and Don't Listen Enough](<https://devfeed.tech/articles/robots-that-talk-too-much-and-don-t-listen-enough-132240.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16473>)

Author: Graham Wilcock, Kristiina Jokinen

Published: 2026-11-13T05:00:00Z

Content type: opinion

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [failures](<https://devfeed.tech/tags/failures.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [robots](<https://devfeed.tech/tags/robots.md>), [social-robots](<https://devfeed.tech/tags/social-robots.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

### AI overview

The paper argues that better speech recognition has improved spoken interaction with social robots, but turn-taking failures and other misunderstandings remain common. It reviews earlier approaches to speech recognition and turn-taking, then proposes multimodal bidirectional streaming as a way to reduce turn-taking failures. The excerpt does not specify implementation details or report comparative results.

### Source excerpt

arXiv:2611.16473v1 Announce Type: cross Abstract: Until recently, the biggest obstacles to natural spoken interaction with social robots were speech recognition errors and turn-taking failures. Advances in speech recognition have enabled much better spoken interaction, but turn-taking failures and other misunderstandings continue to be far too prevalent. This short paper looks back at earlier methods for improving speech recognition and turn-taking in interaction with social robots, and then looks at what can be done today, arguing that multimodal bidirectional streaming is needed to reduce the frequency of turn-taking failures.

## Top 6 AI Text to Speech Tools in 2026: Features, Strengths and Limits

DevFeed: [Top 6 AI Text to Speech Tools in 2026: Features, Strengths and Limits](<https://devfeed.tech/articles/top-6-ai-text-to-speech-tools-in-2026-features-strengths-and-limits-130705.md>)

Original publisher: [Read original article](<https://www.sitepoint.com/top-6-ai-text-to-speech-tools-in-2026-features-strengths-and-limits/>)

Author: Div John

Published: 2026-11-11T13:59:35Z

Content type: comparison

Language: en

Sources: [SitePoint](<https://devfeed.tech/sources/sitepoint.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [developer-tools](<https://devfeed.tech/tags/developer-tools.md>), [features](<https://devfeed.tech/tags/features.md>), [limits](<https://devfeed.tech/tags/limits.md>), [reading](<https://devfeed.tech/tags/reading.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text](<https://devfeed.tech/tags/text.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [tools](<https://devfeed.tech/tags/tools.md>), [web](<https://devfeed.tech/tags/web.md>)

### AI overview

There is no single best text-to-speech tool; the choice depends on whether you need expressive voiceovers, document listening, licensed corporate narration, or developer integration. OpenArt combines voice models with video and music tools and supports bracketed performance cues, but limits scripts to 10,000 characters. ElevenLabs offers expressive and multilingual models plus an API, with model-specific language support and credit costs to check. Murf AI provides studio controls and pronunciation libraries; its free plan lacks downloads and commercial rights. Speechify focuses on reading documents aloud across apps and formats. WellSaid Labs emphasizes consent-based, commercially licensed voices and shared team pronunciation controls, with sales-quoted pricing. Amazon Polly is API-focused, with SSML, custom lexicons, MP3/OGG output, and usage-based pricing. The excerpt advises checking current plans and licenses, testing with your own script, correcting names and numbers, and listening to the output before publishing.

### Source excerpt

null Continue reading Top 6 AI Text to Speech Tools in 2026: Features, Strengths and Limits on SitePoint.

## Fish Audio Releases Live 3, a Full-Duplex Voice Model That Reacts in 186 ms

DevFeed: [Fish Audio Releases Live 3, a Full-Duplex Voice Model That Reacts in 186 ms](<https://devfeed.tech/articles/fish-audio-releases-live-3-a-full-duplex-voice-model-that-reacts-in-186-ms-128465.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/11/10/fish-audio-live-3-full-duplex-voice-model/>)

Author: Asif Razzaq

Published: 2026-11-11T04:43:53Z

Content type: release

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-shorts](<https://devfeed.tech/tags/ai-shorts.md>), [api](<https://devfeed.tech/tags/api.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [audio](<https://devfeed.tech/tags/audio.md>), [audio-language-model](<https://devfeed.tech/tags/audio-language-model.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [beta](<https://devfeed.tech/tags/beta.md>), [browser](<https://devfeed.tech/tags/browser.md>), [call](<https://devfeed.tech/tags/call.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [english](<https://devfeed.tech/tags/english.md>), [fish-audio](<https://devfeed.tech/tags/fish-audio.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [launch](<https://devfeed.tech/tags/launch.md>), [model](<https://devfeed.tech/tags/model.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [releases](<https://devfeed.tech/tags/releases.md>), [staff](<https://devfeed.tech/tags/staff.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

Fish Audio Live 3 is a full-duplex voice model for real-time agents that listens while speaking and handles conversational timing within the speech model. The article describes it acknowledging pauses in a phone number, continuing after a backchannel such as "yeah," stopping for an interruption, and waiting through a correction such as "two pizzas... no, three." Short reactions may begin in as little as 186 ms; this is not a complete-reply latency measure. Fish Audio provides behavior demos but no benchmark scores or latency distribution, and the article reports English-only availability at launch. Developers can use the hosted Fish Agent Platform, Fish Audio API, or enterprise on-premises deployment. The platform is in public beta; enabling Live 3 fixes speech recognition to `fish_audio:transcribe-stream-1`.

### Source excerpt

Fish Audio released Live 3, a full-duplex voice model for voice agents. It listens while it speaks, separates "mm-hmm" from real interruptions, and can react in as little as 186 ms. English only at launch The post Fish Audio Releases Live 3, a Full-Duplex Voice Model That Reacts in 186 ms appeared first on MarkTechPost.

## Audio8-TTS-Realistic-Air: Edge0's 0.6B Multilingual Text-to-Speech Model Explained

DevFeed: [Audio8-TTS-Realistic-Air: Edge0's 0.6B Multilingual Text-to-Speech Model Explained](<https://devfeed.tech/articles/audio8-tts-realistic-air-edge0-s-0-6b-multilingual-text-to-speech-model-explained-116234.md>)

Original publisher: [Read original article](<https://hackernoon.com/audio8-tts-realistic-air-edge0s-06b-multilingual-text-to-speech-model-explained?source=rss>)

Author: aimodels44

Published: 2026-11-04T08:54:59Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [ai](<https://devfeed.tech/tags/ai.md>), [api](<https://devfeed.tech/tags/api.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [audio8-vs-preview](<https://devfeed.tech/tags/audio8-vs-preview.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [business](<https://devfeed.tech/tags/business.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [content-creation](<https://devfeed.tech/tags/content-creation.md>), [cryptocurrency](<https://devfeed.tech/tags/cryptocurrency.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [features](<https://devfeed.tech/tags/features.md>), [model](<https://devfeed.tech/tags/model.md>), [multilingual-text-to-speech](<https://devfeed.tech/tags/multilingual-text-to-speech.md>), [specifications](<https://devfeed.tech/tags/specifications.md>), [speech](<https://devfeed.tech/tags/speech.md>), [technical](<https://devfeed.tech/tags/technical.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [text-to-speech-ai](<https://devfeed.tech/tags/text-to-speech-ai.md>), [text-to-speech-model](<https://devfeed.tech/tags/text-to-speech-model.md>), [tts](<https://devfeed.tech/tags/tts.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voice-cloning](<https://devfeed.tech/tags/voice-cloning.md>)

### AI overview

Audio8-TTS-Realistic-Air is Edge0's 0.6B multilingual speech model. It generates speech from text alone or clones a voice using clean reference audio and an exact transcript, and lists 23 languages--12 more than the earlier Preview release. Reported comparisons are mixed: on Seed-TTS, Realistic Air has a 1.25% error rate versus Preview's 1.23% and higher speaker similarity (0.704 versus 0.682); on the shared eight-language CV3 subset, its error rate is lower (4.56% versus 4.80%). Evaluations used different pipelines, so the figures do not establish a strictly matched ranking, and these metrics do not directly measure perceived realism. The model can be run through Transformers or a non-streaming, OpenAI-compatible SGLang Omni speech API. Transformers loading requires custom code via `trust_remote_code=True`; Python 3.10+ and a CUDA-capable GPU are recommended. Throughput and streaming performance were not benchmarked.

### Source excerpt

Learn how Audio8-TTS-Realistic-Air works, including its features, technical specifications, benchmarks, limitations, and voice cloning capabilities.

## AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech

DevFeed: [AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech](<https://devfeed.tech/articles/aegisvox-factorized-privacy-and-clinical-preservation-for-long-form-speech-117294.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.14643>)

Author: Praphul Singh, Akshat Agarwal, Ratul Ghosh, Aashna Kanuga, Devashish Khatwani, Anurag Dwarakanath, Sri Gadde, Krishnaram Kenthapadi

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [asr](<https://devfeed.tech/topics/asr.md>), [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [codec](<https://devfeed.tech/tags/codec.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [identifier](<https://devfeed.tech/tags/identifier.md>), [measurement](<https://devfeed.tech/tags/measurement.md>), [measurements](<https://devfeed.tech/tags/measurements.md>), [model](<https://devfeed.tech/tags/model.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [rendering](<https://devfeed.tech/tags/rendering.md>), [replacement](<https://devfeed.tech/tags/replacement.md>), [speaker](<https://devfeed.tech/tags/speaker.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [tradeoffs](<https://devfeed.tech/tags/tradeoffs.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [turns](<https://devfeed.tech/tags/turns.md>), [voice](<https://devfeed.tech/tags/voice.md>)

### AI overview

AegisVox assigns consistent surrogate voices and replaces identifier-bearing turns with document-consistent text, aiming to protect identity while retaining clinical speech cues. In a matched comparison across 390 public simulated or non-patient conversations (63.6 hours), text-to-speech realized 535 of 601 planned replacements, versus 453 for a codec-based editor. Yet similar aggregate clinical preservation masked more model-labeled contradictions, and conversation-wide speaker scores masked greater edited-speech linkage in a score-direction sensitivity analysis. Fresh transcription of original audio and an independent judge were used to examine measurement sensitivity. The findings show that better replacement realization can coexist with worse clinical or speaker-linkage measurements; the framework supports use-specific evaluation, not guarantees of identifier removal or clinical safety.

### Source excerpt

arXiv:2610.14643v2 Announce Type: replace Abstract: Clinical speech reveals identity through both spoken identifiers and voice, yet its pauses, delivery, and speaker roles can also carry useful clinical evidence. Protecting that speech therefore requires more than preserving its words. We present AegisVox, a factorized framework that assigns consistent surrogate voices and edits identifier-bearing turns using document-consistent replacements. Across 390 public simulated or non-patient conversations totaling 63.6 hours, a matched comparison of two speech renderers separates replacement realization from retained identifiers, speaker linkage, and clinical preservation. Text-to-speech rendering realizes 535/601 planned replacements versus 453/601 for the codec-based speech editor. However, similar aggregate clinical preservation conceals more model-labeled contradictions, and conversation-wide speaker scores conceal greater edited-speech linkage under a score-direction sensitivity analysis. Fresh transcription of original audio provides a reference for interpreting the clinical measurement chain; an independent judge tests sensitivity to model-based assessment. Better replacement realization can therefore coexist with worse clinical or speaker-linkage measurements. AegisVox supports use-specific evaluation of those tradeoffs, not a guarantee of identifier removal or clinical safety.

## Audio Prompt Injection: What Voice Agents Hear That Text Guardrails Miss

DevFeed: [Audio Prompt Injection: What Voice Agents Hear That Text Guardrails Miss](<https://devfeed.tech/articles/audio-prompt-injection-what-voice-agents-hear-that-text-guardrails-miss-106704.md>)

Original publisher: [Read original article](<https://blog.checkpoint.com/ai-security/audio-guardrails-voice-agents/>)

Author: maciejd@checkpoint.com

Published: 2026-10-29T13:00:13Z

Content type: article

Language: en

Sources: [Check Point Blog](<https://devfeed.tech/sources/check-point-blog.md>)

Topics: [prompt injection](<https://devfeed.tech/topics/prompt-injection.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>)

Tags: [ai-security](<https://devfeed.tech/tags/ai-security.md>), [attack](<https://devfeed.tech/tags/attack.md>), [audio](<https://devfeed.tech/tags/audio.md>), [guardrails](<https://devfeed.tech/tags/guardrails.md>), [prompt-injection](<https://devfeed.tech/tags/prompt-injection.md>), [transcript](<https://devfeed.tech/tags/transcript.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>)

### AI overview

Voice agents that receive audio directly can be manipulated in ways transcript-only checks may miss: tampered sound can produce silence or garbled text for a transcriber while the model still hears the original instruction. The described audio guardrail first checks whether a recording is usable or manipulated, then transcribes it and scores the words for prompt attacks in more than 80 languages. The application receives the verdict and chooses a response, such as ending the turn, asking the caller to repeat, or transferring to a person. The article's rule is to screen audio when audio reaches the model, and transcripts when the model receives transcripts; this can vary by agent architecture. Checks take about a quarter second for clips under 30 seconds. The Early Access feature addresses prompt attacks, not voice cloning. Its self-hosted version keeps audio within the organization's infrastructure; its SaaS version is available for evaluations.

### Source excerpt

Key takeaways Only 24 percent of organizations can fully inspect AI traffic without impacting performance, according to Check Point's 2026 Cloud Security Report. ->Voice agents are easy to talk into things. Researchers got a leading realtime voice model to give harmful answers to about 70 percent of spoken requests by varying their speed, pitch and background noise ->A transcript check can be blinded. An attacker can speak the instruction and tamper with the recording, so the transcript shows garbled words or nothing at all while the model still receives the original audio ->Audio guardrails screen the sound itself. They check [...] The post Audio Prompt Injection: What Voice Agents Hear That Text Guardrails Miss appeared first on Check Point Blog.

## Voice AI in vogue: Nvidia releases new open transcription model

DevFeed: [Voice AI in vogue: Nvidia releases new open transcription model](<https://devfeed.tech/articles/voice-ai-in-vogue-nvidia-releases-new-open-transcription-model-104587.md>)

Original publisher: [Read original article](<https://techcrunch.com/2026/10/28/voice-ai-in-vogue-nvidia-releases-new-open-transcription-model/>)

Author: Julie Bort

Published: 2026-10-28T15:00:00Z

Content type: news

Language: en

Sources: [TechCrunch | AI News & Artificial Intelligence](<https://devfeed.tech/sources/techcrunch-ai-news-artificial-intelligence.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [NeMo](<https://devfeed.tech/topics/nemo.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [audio](<https://devfeed.tech/tags/audio.md>), [business](<https://devfeed.tech/tags/business.md>), [developers](<https://devfeed.tech/tags/developers.md>), [dgx-spark](<https://devfeed.tech/tags/dgx-spark.md>), [elevenlabs](<https://devfeed.tech/tags/elevenlabs.md>), [free](<https://devfeed.tech/tags/free.md>), [in-brief](<https://devfeed.tech/tags/in-brief.md>), [languages](<https://devfeed.tech/tags/languages.md>), [medical](<https://devfeed.tech/tags/medical.md>), [model](<https://devfeed.tech/tags/model.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open](<https://devfeed.tech/tags/open.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [recordings](<https://devfeed.tech/tags/recordings.md>), [releases](<https://devfeed.tech/tags/releases.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [tc](<https://devfeed.tech/tags/tc.md>), [tools](<https://devfeed.tech/tags/tools.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [voice](<https://devfeed.tech/tags/voice.md>)

### AI overview

Nvidia's Nemotron 3.5 Transcribe is a free, open-weight model for recorded audio: it supports up to 40 languages, translates 31 languages to and from English, handles recordings up to an hour, and can distinguish as many as eight speakers. Nvidia positions it for meeting and medical transcription apps, with customization for industry jargon using the updated Granary dataset and tools such as NeMo Speech. The company says its 30-billion-parameter model can run on a single chip, including Jetson AGX Thor, DGX Spark, and Blackwell hardware, and claims benchmark wins; the excerpt gives no benchmark results. It does not support livestreams, unlike ElevenLabs' Realtime offering for instant speech in voice agents.

### Source excerpt

Nvidia just released a free speech-to-text model that can transcribe up to 40 languages, but it only works on recordings, not live audio, so it's a big step for meeting and medical transcription tools while still leaving real-time use to rivals like ElevenLabs.

## NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt

DevFeed: [NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt](<https://devfeed.tech/articles/nvidia-nemotron-3-5-transcribe-and-granary-v2-intelligent-multilingual-transcription-you-can-prompt-and-adapt-105893.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/nvidia/nemotron-transcribe-granary>)

Author: Maryam Motamedi; Piotr Żelasko; Kunal Dhawan; Nikolay Karpov; Nune Tadevosyan; Meline Mkrtchyan; Nithin Rao Koluguri; Francesco

Published: 2026-10-28T14:58:17Z

Content type: release

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>), [Internationalization (i18n)](<https://devfeed.tech/topics/i18n.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [applications](<https://devfeed.tech/tags/applications.md>), [audio](<https://devfeed.tech/tags/audio.md>), [data](<https://devfeed.tech/tags/data.md>), [data-pipeline](<https://devfeed.tech/tags/data-pipeline.md>), [domain](<https://devfeed.tech/tags/domain.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [format](<https://devfeed.tech/tags/format.md>), [granary-v2](<https://devfeed.tech/tags/granary-v2.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [model](<https://devfeed.tech/tags/model.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [nemotron-3-5](<https://devfeed.tech/tags/nemotron-3-5.md>), [nemotron-3-5-transcribe](<https://devfeed.tech/tags/nemotron-3-5-transcribe.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-nemotron](<https://devfeed.tech/tags/nvidia-nemotron.md>), [open](<https://devfeed.tech/tags/open.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [v2](<https://devfeed.tech/tags/v2.md>)

### AI overview

NVIDIA Nemotron 3.5 Transcribe combines multilingual transcription, speaker attribution, translation, and audio Q&A in one promptable open-weight checkpoint. It supports transcription in 40 languages, translation between English and 31 languages, up to eight speakers, and recordings up to an hour; language support varies by task, and audio Q&A is trained in English. It is designed for recorded audio, with vLLM serving available and quantized local deployment described as forthcoming. Granary v2 provides multilingual manifests and a reusable curation pipeline, while NeMo Speech tools support fine-tuning. In an Indian medical ASR case study, fine-tuning reduced held-out WER from 22.44% to 12.17% versus the best-prompt baseline, with a slight LibriSpeech test-clean WER increase from 1.54% to 1.69%; a medical specialist still led on exact brand-name matches. The source recommends evaluating domain entities and general-speech retention.

### Source excerpt

# NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt ## One open model for multilingual transcription, speaker attribution, translation, and audio Q&A, with top leaderboard results, plus the open Granary v2 data pipeline and fine-tuning recipes to adapt it. A transcript can capture every spoken word and still leave users or applications wondering who said what, how to translate it, or how to format domain-specific jargon.

## From Voiceprints to User Entities: Online Speaker-Memory Construction for Multi-User Spoken Agents

DevFeed: [From Voiceprints to User Entities: Online Speaker-Memory Construction for Multi-User Spoken Agents](<https://devfeed.tech/articles/from-voiceprints-to-user-entities-online-speaker-memory-construction-for-multi-user-spoken-agents-103180.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.36713>)

Author: Yucong Zhang, Cancan Li, Bo Xiao, Ke Zeng, Ming Li

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [identity](<https://devfeed.tech/tags/identity.md>), [memory](<https://devfeed.tech/tags/memory.md>), [online](<https://devfeed.tech/tags/online.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [speaker](<https://devfeed.tech/tags/speaker.md>), [user](<https://devfeed.tech/tags/user.md>)

### AI overview

EvoSpeakerMem treats voiceprints as evolving evidence about persistent user entities, routing speech causally and selectively updating acoustic state before partitioning memory writes and retrieval by identity through a replaceable text-memory backend. The training-free layer is evaluated with LoCoMo-ID, a benchmark featuring hidden speaker identities, speaker-sensitive queries, simulated recording shifts, and four-speaker confusion tests. Gated updates improve identity-gated QA over fixed representations by 3.86 percentage points in clean conditions and 11.62 under shifts, with no wrong-user authorizations; the advantage persists in the four-speaker test. Separately, longitudinal SMIIP-TV results show genuine-probe rejection rising from 29.8% to 55.6% over about a year at a frozen near-enrollment threshold. The authors caution that less fragmentation and better QA can still accompany incorrect cross-user links, so QA alone misses the fragmentation-contamination trade-off.

### Source excerpt

arXiv:2610.36713v1 Announce Type: cross Abstract: Personalized spoken agents must identify speakers before accessing user-specific memory, yet memory benchmarks typically assume correctly attributed histories. We argue that voiceprints are observations of identity, not identities themselves, and formulate online speaker-memory construction to maintain persistent user entities as acoustic evidence evolves. We introduce EvoSpeakerMem, a training-free layer for causal speaker routing, selective acoustic-state updates, and identity-partitioned memory writes and retrieval through a replaceable text-memory backend. We also construct LoCoMo-ID, a spoken-memory benchmark with hidden speaker identities, speaker-sensitive queries, simulated recording-condition shifts, and four-speaker identity-confusion tests. On longitudinal SMIIP-TV, genuine-probe rejection under a frozen near-enrollment threshold rises from 29.8% to 55.6% over approximately one year. On LoCoMo-ID speaker-sensitive queries, gated updating improves identity-gated QA over fixed representations by 3.86 and 11.62 percentage points under clean and shifted conditions, respectively, with no wrong-user authorizations. The advantage persists in the four-speaker test. Yet reducing fragmentation and improving QA can coexist with incorrect cross-user links, revealing a fragmentation--contamination trade-off that QA alone cannot capture.

## Better speech synthesis for audiobooks

DevFeed: [Better speech synthesis for audiobooks](<https://devfeed.tech/articles/better-speech-synthesis-for-audiobooks-97855.md>)

Original publisher: [Read original article](<https://www.amazon.science/blog/better-speech-synthesis-for-audiobooks>)

Author: Arent van Korlaar

Published: 2026-10-23T15:20:02Z

Content type: article

Language: en

Sources: [Amazon Science homepage](<https://devfeed.tech/sources/amazon-science-homepage.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [adapting](<https://devfeed.tech/tags/adapting.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [all](<https://devfeed.tech/tags/all.md>), [allows](<https://devfeed.tech/tags/allows.md>), [also](<https://devfeed.tech/tags/also.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [annotations](<https://devfeed.tech/tags/annotations.md>), [applications](<https://devfeed.tech/tags/applications.md>), [approach](<https://devfeed.tech/tags/approach.md>), [audio](<https://devfeed.tech/tags/audio.md>), [audiobook](<https://devfeed.tech/tags/audiobook.md>), [better](<https://devfeed.tech/tags/better.md>), [character-voices](<https://devfeed.tech/tags/character-voices.md>), [conversational-ai](<https://devfeed.tech/tags/conversational-ai.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-synthesis](<https://devfeed.tech/tags/speech-synthesis.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [text](<https://devfeed.tech/tags/text.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [text-to-speech-tts](<https://devfeed.tech/tags/text-to-speech-tts.md>), [voices](<https://devfeed.tech/tags/voices.md>)

### AI overview

Audible's controllable text-to-speech pipeline aims to make audiobook dialogue expressive while keeping character voices distinct and consistent. BookLLM reads the book to identify characters, assign profiles from a curated voice catalogue, attribute passages to speakers, and write delivery captions for dialogue. A controllable TTS model combines those profiles and captions to synthesize speech. Its training data pairs audio with varied natural-language descriptions, expanded through human annotation, expert-model analysis, and a speech captioner that supports iterative synthetic-data generation. In tests on 25 English and Spanish books, BookLLM identified speakers with 97% and 94% accuracy, respectively. Dialogue differentiation improved by 3.7 points on a MUSHRA test, and listening experience improved significantly for eight of ten voices across English and Spanish tests. Minor characters use the default narrator voice.

### Source excerpt

"Controllable text to speech" makes speech more expressive, character voices more distinct and consistent.

## Microsoft's new voice AI can start working out what you want before you finish speaking

DevFeed: [Microsoft's new voice AI can start working out what you want before you finish speaking](<https://devfeed.tech/articles/microsoft-s-new-voice-ai-can-start-working-out-what-you-want-before-you-finish-speaking-97496.md>)

Original publisher: [Read original article](<https://thenewstack.io/microsoft-streaming-speech-models/>)

Author: Adrian Bridgwater

Published: 2026-10-23T12:00:00Z

Content type: news

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Microsoft](<https://devfeed.tech/topics/microsoft.md>), [Azure](<https://devfeed.tech/topics/azure.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

Microsoft has introduced three voice models in Microsoft Foundry: MAI-Transcribe-2-Streaming returns early speech hypotheses within the low hundreds of milliseconds and refines them into a stable transcript; MAI-Voice-2.1 provides expressive multilingual speech; and MAI-Voice-2.1-Flash targets responsive, high-volume applications, balancing naturalness, speed, and cost. The article says early transcription may let agents identify requests or prepare tool calls before callers finish, and reports Microsoft's claim that the streaming model ranks first for partial and final transcript accuracy on the Artificial Analysis leaderboard. Phonely's practitioners welcome faster speech input and output but caution that latency alone is insufficient: faster replies can cause audible interruptions unless turn-taking improves too. They also stress that reliable agents need operational learning and that critical requirements, such as collecting required information before a refund, should be enforced in code rather than left to the model.

### Source excerpt

"I think you said, 'representative.'" It's the fastest way to bypass automated speech recognition chatbots that fail to route human The post Microsoft's new voice AI can start working out what you want before you finish speaking appeared first on The New Stack.

## Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions

DevFeed: [Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions](<https://devfeed.tech/articles/beyond-naturalness-probing-automated-text-to-speech-evaluators-on-linguistically-grounded-dimensions-97959.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.09930>)

Author: Oluwanifemi Bamgbose, Jash Shah, Simon Rosen, Lindsay Devon Brin, Hoang H Nguyen, Anke Koelzer, Rachel Hansen, Tara Bogavelli, Fanny Riols

Published: 2026-10-23T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [human review](<https://devfeed.tech/topics/human-review.md>)

Tags: [annotation](<https://devfeed.tech/tags/annotation.md>), [audio](<https://devfeed.tech/tags/audio.md>), [automated](<https://devfeed.tech/tags/automated.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [detection](<https://devfeed.tech/tags/detection.md>), [dimension](<https://devfeed.tech/tags/dimension.md>), [dimensions](<https://devfeed.tech/tags/dimensions.md>), [errors](<https://devfeed.tech/tags/errors.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [llm](<https://devfeed.tech/tags/llm.md>), [model](<https://devfeed.tech/tags/model.md>), [opinion](<https://devfeed.tech/tags/opinion.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>)

### AI overview

The study benchmarks four MOS predictors and four Audio-LLM judges against 860 utterances annotated by trained linguists across 10 perceptual dimensions of speech. MOS predictors are more sensitive to acoustic properties at prosodic and paralinguistic levels; Audio-LLM judges selectively detect errors, favor word-level errors, and vary with prompting. Neither group reliably captures a broad range of linguistically structured, human-perceptible errors. The authors release the dataset, annotation schema, and evaluation code to support targeted TTS evaluation.

### Source excerpt

arXiv:2608.09930v2 Announce Type: replace-cross Abstract: Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Model (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive. We deconstruct "naturalness" into a linguistically grounded annotation schema spanning 10 distinct perceptual dimensions, and use it to construct the first dimension-level meta-evaluation benchmark for TTS, comprising 860 utterances annotated by trained linguist raters. Results from benchmarking four MOS predictors and four Audio-LLM judges reveal that MOS predictors are more sensitive to acoustic signal-level properties at the prosodic and paralinguistic levels, while Audio-LLM judges show selective, prompt-dependent detection that favors word-level errors and does not generalize across all dimensions. Neither class reliably captures a breadth of linguistically structured, human-perceptible speech errors. Our dataset, annotation schema, and evaluation code are publicly released to support more targeted and interpretable TTS evaluation.

## Mistral AI Releases Voxtral 3: Speech-to-Text, TTS and a Voice Agent SDK

DevFeed: [Mistral AI Releases Voxtral 3: Speech-to-Text, TTS and a Voice Agent SDK](<https://devfeed.tech/articles/mistral-ai-releases-voxtral-3-speech-to-text-tts-and-a-voice-agent-sdk-91855.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/10/20/mistral-voxtral-3-speech-models-voice-agent-sdk/>)

Author: Asif Razzaq

Published: 2026-10-21T06:03:12Z

Content type: news

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [asr](<https://devfeed.tech/topics/asr.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>), [livekit](<https://devfeed.tech/topics/livekit.md>), [SDK](<https://devfeed.tech/topics/sdk.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Whisper](<https://devfeed.tech/topics/whisper.md>), [machine-translation](<https://devfeed.tech/topics/machine-translation.md>), [context window](<https://devfeed.tech/topics/context-window.md>)

Tags: [agent-sdk](<https://devfeed.tech/tags/agent-sdk.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai-shorts](<https://devfeed.tech/tags/ai-shorts.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [asr](<https://devfeed.tech/tags/asr.md>), [audio-language-model](<https://devfeed.tech/tags/audio-language-model.md>), [diarization](<https://devfeed.tech/tags/diarization.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [for-devs](<https://devfeed.tech/tags/for-devs.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [livekit](<https://devfeed.tech/tags/livekit.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mistral-ai](<https://devfeed.tech/tags/mistral-ai.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [releases](<https://devfeed.tech/tags/releases.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [staff](<https://devfeed.tech/tags/staff.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>), [tts](<https://devfeed.tech/tags/tts.md>), [uncategorized](<https://devfeed.tech/tags/uncategorized.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

Voxtral 3 bundles three transcription models, Voxtral Speech 3 for text-to-speech, and a LiveKit-based voice-agent SDK; the models are in public preview on Mistral's API. Small Transcribe 3 scores 2.1% AA-WER (#3 on Artificial Analysis) and 18.4% cpWER for diarization, while Mini is a 4B, lower-cost batch option. All transcription models support diarization and context biasing; batch transcription covers 50 languages. The SDK streams speech recognition, LLM reasoning and tool calls, then speech generation, supports caller interruption and SIP or WebRTC sessions, and allows swapping its component models. Realtime transcription trades accuracy for latency: at 240 ms it scores 7.4% WER, versus 4.2% for Gemini 3.5 Realtime Transcribe. Listed prices range from $0.0035-$0.008 per transcription minute; TTS is $13 per million characters. The article reports that GPU requirements are undisclosed and no open weights were announced.

### Source excerpt

Mistral AI released Voxtral 3, a family of 3 transcription models, 1 text-to-speech model and a LiveKit-based voice agent SDK. Voxtral Small Transcribe 3 scores 2.1% AA-WER, ranking #3 on Artificial Analysis. The post Mistral AI Releases Voxtral 3: Speech-to-Text, TTS and a Voice Agent SDK appeared first on MarkTechPost.

## Steering intonational tunes in text-to-speech synthesis

DevFeed: [Steering intonational tunes in text-to-speech synthesis](<https://devfeed.tech/articles/steering-intonational-tunes-in-text-to-speech-synthesis-94283.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.27403>)

Author: Juan Pablo Vigneaux, Jennifer Cole

Published: 2026-10-21T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>)

Tags: [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [speech](<https://devfeed.tech/tags/speech.md>), [steering](<https://devfeed.tech/tags/steering.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>)

### AI overview

In CSM and S2 text-to-speech systems, the transformer backbone's hidden state carries English nuclear intonation through the depth model; the first token level carries little or none. Prompt control mostly fails except for a final question mark. Steering with vectors derived from human imitations of specified tunes moves most recordings toward a broadly falling or rising melody, including across speakers, with a small predicted-naturalness cost. Vectors derived from punctuation can also shift intonation, but pushing toward assertions degrades voice quality. Edge tones shift more coherently than pitch accents, and steering cannot select a precise tune.

### Source excerpt

arXiv:2610.27403v1 Announce Type: cross Abstract: Text-to-speech systems built on residual vector quantization often generate audio in two stages: a transformer backbone emits the first level of each frame's token stack, and a smaller depth model generates the rest autoregressively. We ask which components of such a model are responsible for prosodic intonation, more precisely for generating the English nuclear tune (for instance, the falling pitch melody that distinguishes an assertion from a question's rising melody), and whether interventions on those components can be used to control intonation. Prompt-based control mostly fails; only a final question mark is a reliable lever. Interchange interventions in CSM and S2 locate the tune: the backbone's hidden state carries it to the waveform through the depth model, and the first level of the stack carries little or none of it. Steering the backbone state with a difference-of-means vector computed on new material generated with a final period or question mark moves the tune, but the push toward an assertion degrades the voice. Steering vectors computed instead from the backbone states associated with human imitations of theoretically-specified tunes, whose text carries no punctuation, and applied inside the generation loop, move most recordings to the other condition in both systems (even when fitted without the speaker being steered), at a small cost in predicted naturalness. These imitation-derived vectors are aligned with the one obtained from the two punctuated conditions far beyond a shuffled-label null, so the backbone state of a punctuated prompt already represents intonation. The steered recordings broadly become falling or rising, according to the intent, but the edge tone moves more coherently than the pitch accent, and the target tune cannot be selected precisely beyond this broad tendency.

## TTS-Q: Speaker-Aware Post-Training Quantization of Text-to-Speech Diffusion Transformers

DevFeed: [TTS-Q: Speaker-Aware Post-Training Quantization of Text-to-Speech Diffusion Transformers](<https://devfeed.tech/articles/tts-q-speaker-aware-post-training-quantization-of-text-to-speech-diffusion-transformers-94183.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.12474>)

Author: Xiaoqin Feng, Yuqi Li, Weilun Feng, Yi-Cheng Lin, Kai Li, Xianglong Wang, Kuo Yang, Chuanguang Yang, Yingli Tian, Wei Li, Zhongwei Cheng, Lin Chen

Published: 2026-10-21T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [speaker](<https://devfeed.tech/tags/speaker.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [tts](<https://devfeed.tech/tags/tts.md>)

### AI overview

The abstract reports that 4-bit weight-and-activation quantization preserves TTS intelligibility fairly well but degrades speaker timbre. In tests on 1B- and 3.5B-parameter diffusion transformers, both weight and activation quantization contribute to the timbre loss; only activation errors show timestep structure that a precision schedule can target. TTS-Q selects difficult items from task-matched calibration candidates and assigns higher precision to late-step feed-forward activations within a balanced precision budget. The authors report the strongest timbre recovery at both scales without reducing intelligibility. A separately calibrated W4A4 reference reduces peak GPU memory; the excerpt gives no numerical memory or quality results.

### Source excerpt

arXiv:2610.12474v3 Announce Type: replace-cross Abstract: Zero-shot text-to-speech (TTS) can clone a voice from seconds of audio. Yet the quality comes from billion-parameter diffusion transformers whose size and multi-step sampling put them out of reach of the on-device, assistive, and real-time applications that need them most. Post-training quantization (PTQ) shrinks such models without retraining, but it has been built for language and vision, not speech. At 4 bits, a strong quantizer leaves intelligibility close to full precision but consistently erodes speaker timbre, so error rates stay low while the voice drifts from the target speaker. We ask why quantization hurts speech, where the damage lives, and whether it can be repaired without retraining. To find out, we run the first 4-bit weight-and-activation (W4A4) PTQ study of a TTS diffusion transformer at 1B and 3.5B. The timbre cost comes from both the weight and the activation quantization, but only the activation part has the timestep structure a precision schedule can exploit. We therefore introduce TTS-Q, which selects hard-to-quantize items from task-matched calibration candidates and applies a budget-balanced mixed-precision schedule that assigns higher precision to late-step feed-forward activations. TTS-Q recovers timbre best at both scales, at no cost to intelligibility. Separately, the deployed calibrated-W4A4 reference reduces peak memory on GPU, providing a practical reference for resource-constrained TTS deployment.

## NEW GUIDE: CircuitPython Text to Speech

DevFeed: [NEW GUIDE: CircuitPython Text to Speech](<https://devfeed.tech/articles/new-guide-circuitpython-text-to-speech-90198.md>)

Original publisher: [Read original article](<https://blog.adafruit.com/2026/10/19/new-guide-circuitpython-text-to-speech/>)

Author: Anne Barela

Published: 2026-10-19T15:18:19Z

Content type: tutorial

Language: en

Sources: [Adafruit Industries](<https://devfeed.tech/sources/adafruit-industries-makers-hackers-artists-designers-and-engineers.md>)

Topics: [CircuitPython](<https://devfeed.tech/topics/circuitpython.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>)

Tags: [adafruit-learning-system](<https://devfeed.tech/tags/adafruit-learning-system.md>), [arduino](<https://devfeed.tech/tags/arduino.md>), [circuitpython](<https://devfeed.tech/tags/circuitpython.md>), [feather](<https://devfeed.tech/tags/feather.md>), [fruit-jam](<https://devfeed.tech/tags/fruit-jam.md>), [guide](<https://devfeed.tech/tags/guide.md>), [microcontrollers](<https://devfeed.tech/tags/microcontrollers.md>), [psram](<https://devfeed.tech/tags/psram.md>), [python](<https://devfeed.tech/tags/python.md>), [python-on-microcontrollers](<https://devfeed.tech/tags/python-on-microcontrollers.md>), [software](<https://devfeed.tech/tags/software.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-synthesis](<https://devfeed.tech/tags/speech-synthesis.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>)

### AI overview

This article introduces CircuitPython 11 support for speech synthesis on microcontrollers. It names Moonshine Klatt for average-capability chips and PicoTTS for more powerful microcontrollers with PSRAM, noting that model capability depends on processor performance and available memory.

### Source excerpt

Text to speech has been a favorite since the introduction of the TI Speak & Spell. There are many environments where feedback via speech would be a great feature. Up until recently, using microcontrollers for speech was too slow. But not any more, thanks to CircuitPython 11! The capability of a speech model depends on [...]

## Fish Audio Releases Drama 3, a 35B Open-Weight MoE Text-to-Speech Model

DevFeed: [Fish Audio Releases Drama 3, a 35B Open-Weight MoE Text-to-Speech Model](<https://devfeed.tech/articles/fish-audio-releases-drama-3-a-35b-open-weight-moe-text-to-speech-model-87311.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/10/14/fish-audio-drama-3-open-weight-tts-model/>)

Author: Asif Razzaq

Published: 2026-10-14T22:52:17Z

Content type: news

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [mixtral](<https://devfeed.tech/topics/mixtral.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Generative AI](<https://devfeed.tech/topics/generative-ai.md>), [tgi](<https://devfeed.tech/topics/tgi.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-shorts](<https://devfeed.tech/tags/ai-shorts.md>), [amd](<https://devfeed.tech/tags/amd.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [audio-language-model](<https://devfeed.tech/tags/audio-language-model.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cost](<https://devfeed.tech/tags/cost.md>), [descript](<https://devfeed.tech/tags/descript.md>), [developers](<https://devfeed.tech/tags/developers.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [english](<https://devfeed.tech/tags/english.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [for-devs](<https://devfeed.tech/tags/for-devs.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [moe](<https://devfeed.tech/tags/moe.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [open-weights](<https://devfeed.tech/tags/open-weights.md>), [releases](<https://devfeed.tech/tags/releases.md>), [staff](<https://devfeed.tech/tags/staff.md>), [tech-news](<https://devfeed.tech/tags/tech-news.md>), [technology](<https://devfeed.tech/tags/technology.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

Fish Audio released Drama 3, a 35B mixture-of-experts text-to-speech model with about 3B active parameters, plain-language acting directions and support for 83 languages. Its open weights are available for self-hosting, which requires at least 48 GB of NVIDIA GPU memory in FP8; commercial use is free below USD 500,000 in annual revenue under the community license. Fish Audio reports a 93.2% score on Artificial Analysis Pronunciation Robustness, while exact sequences such as codes and emails scored 86.3%.

### Source excerpt

Fish Audio released Drama 3, a 35B mixture-of-experts text-to-speech model with about 3B active parameters. It takes plain-language direction, covers 83 languages and ships open weights under a revenue-capped community license. The post Fish Audio Releases Drama 3, a 35B Open-Weight MoE Text-to-Speech Model appeared first on MarkTechPost.

## Drama 3 now available on AI Gateway

DevFeed: [Drama 3 now available on AI Gateway](<https://devfeed.tech/articles/drama-3-now-available-on-ai-gateway-86360.md>)

Original publisher: [Read original article](<https://vercel.com/changelog/drama-3-now-available-on-ai-gateway>)

Author: Zachary Chen

Published: 2026-10-13T00:00:00Z

Content type: release

Language: en

Sources: [Vercel News](<https://devfeed.tech/sources/vercel-news.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [vercel ai sdk](<https://devfeed.tech/topics/vercel-ai-sdk.md>), [Claude](<https://devfeed.tech/topics/claude.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-gateway](<https://devfeed.tech/tags/ai-gateway.md>), [inference](<https://devfeed.tech/tags/inference.md>), [speech](<https://devfeed.tech/tags/speech.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>)

### AI overview

Vercel AI Gateway adds Fish Audio's Drama 3 text-to-speech model in preview. Drama 3 supports natural-language direction for emotion and pacing, sound and pause placement, delivery tags, and multi-speaker dialogue with separate voices. The article also describes AI Gateway features for model access, usage and cost tracking, retries, failover, reporting, and routing.

### Source excerpt

Drama 3 from Fish Audio is now available on AI Gateway, and 80% off for 30 days (through Nov 12). The text-to-speech model is currently in preview and lets you direct voice performances with natural-language instructions in your script, making it useful for voiceovers, character dialogue, and audio storytelling. Describe the emotion and pacing in square brackets, add sounds and pauses where they belong, or wrap selected words in tags such as <whisper> and <emphasis> to change their delivery. Directions can be written in a different language from the spoken text. Drama 3 also supports multi-speaker dialogue, with a separate voice for each character. Use fish-audio/drama-3 as the model name to get started: Try Drama 3 in the model playground. AI Gateway provides a unified API for calling models, tracking usage and cost, and configuring retries, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, Zero Data Retention support, budgets for API keys, routing rules, and more. AI Gateway reflects provider pricing with no markup and does not charge a platform fee on inference, including on Bring Your Own Key (BYOK) requests. Read more

## Voice AI startup Modulate raises $25M to bring audio-native models to more developers

DevFeed: [Voice AI startup Modulate raises $25M to bring audio-native models to more developers](<https://devfeed.tech/articles/voice-ai-startup-modulate-raises-25m-to-bring-audio-native-models-to-more-developers-61066.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/09/28/voice-ai-startup-modulate-raises-25m-to-bring-audio-native-models-to-more-developers/>)

Author: Duncan Riley

Published: 2026-09-28T14:30:51Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [livekit](<https://devfeed.tech/topics/livekit.md>), [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [open-llm-leaderboard](<https://devfeed.tech/topics/open-llm-leaderboard.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [audio-native-ai](<https://devfeed.tech/tags/audio-native-ai.md>), [carter-huffman](<https://devfeed.tech/tags/carter-huffman.md>), [deepfake](<https://devfeed.tech/tags/deepfake.md>), [developers](<https://devfeed.tech/tags/developers.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [ensemble-listening-model](<https://devfeed.tech/tags/ensemble-listening-model.md>), [funding](<https://devfeed.tech/tags/funding.md>), [future-ventures](<https://devfeed.tech/tags/future-ventures.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [hyperplane](<https://devfeed.tech/tags/hyperplane.md>), [lakestar](<https://devfeed.tech/tags/lakestar.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [massachusetts-institute-of-technology](<https://devfeed.tech/tags/massachusetts-institute-of-technology.md>), [mike-pappas](<https://devfeed.tech/tags/mike-pappas.md>), [modulate](<https://devfeed.tech/tags/modulate.md>), [news](<https://devfeed.tech/tags/news.md>), [open-asr-leaderboard](<https://devfeed.tech/tags/open-asr-leaderboard.md>), [speech-deepfake-arena](<https://devfeed.tech/tags/speech-deepfake-arena.md>), [steve-jurvetson](<https://devfeed.tech/tags/steve-jurvetson.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [trust-and-safety](<https://devfeed.tech/tags/trust-and-safety.md>), [velma](<https://devfeed.tech/tags/velma.md>), [velma-deepfake-detect](<https://devfeed.tech/tags/velma-deepfake-detect.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

Voice AI startup Modulate raised $25 million to expand access to its audio-native models, which analyze conversation signals such as emotion, intent, and possible deepfakes. Its Velma platform combines specialized models for uses including fraud detection, voice-agent monitoring, and moderation. The company plans new SDKs, APIs, industry-specific models, and hiring in research, engineering, and developer relations.

### Source excerpt

Voice artificial intelligence startup Modulate Inc. today announced it has raised $25 million in new funding to get its audio-native models in front of more developers. Modulate offers AI models that work on the raw audio of a conversation. Emotion, tone and intent all register as signals, as do signs that a voice is a [...] The post Voice AI startup Modulate raises $25M to bring audio-native models to more developers appeared first on SiliconANGLE.

## Deploying real-time personalized speech with Qwen3-TTS on Amazon SageMaker AI

DevFeed: [Deploying real-time personalized speech with Qwen3-TTS on Amazon SageMaker AI](<https://devfeed.tech/articles/deploying-real-time-personalized-speech-with-qwen3-tts-on-amazon-sagemaker-ai-60188.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/deploying-real-time-personalized-speech-with-qwen3-tts-on-amazon-sagemaker-ai/>)

Author: Suneesh T

Published: 2026-09-25T16:09:46Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Amazon SageMaker JumpStart](<https://devfeed.tech/topics/amazon-sagemaker-jumpstart.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [Amazon CloudWatch](<https://devfeed.tech/topics/amazon-cloudwatch.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [SDK](<https://devfeed.tech/topics/sdk.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon-cloudwatch](<https://devfeed.tech/tags/amazon-cloudwatch.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [amazon-sagemaker-jumpstart](<https://devfeed.tech/tags/amazon-sagemaker-jumpstart.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [speech](<https://devfeed.tech/tags/speech.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>), [tts](<https://devfeed.tech/tags/tts.md>), [voice](<https://devfeed.tech/tags/voice.md>)

### AI overview

A tutorial showing how to deploy the Qwen3-TTS-12Hz-1.7B-Base text-to-speech model from Amazon SageMaker JumpStart to a managed real-time endpoint and use it for voice cloning from a short reference recording. It also covers deployment configuration, endpoint monitoring, scaling, and invocation with the Amazon SageMaker Python SDK.

### Source excerpt

Deploy the publicly available Qwen3-TTS-12Hz-1.7B-Base text-to-speech model from Amazon SageMaker JumpStart to a fully managed, real-time endpoint, and clone a voice from a short reference clip. Cross-lingual cloning preserves the speaker's identity across languages.

## Pipecat Benchmarked 23 Real-Time STT Models for Voice Agents. There Isn't One Winner.

DevFeed: [Pipecat Benchmarked 23 Real-Time STT Models for Voice Agents. There Isn't One Winner.](<https://devfeed.tech/articles/pipecat-benchmarked-23-real-time-stt-models-for-voice-agents-there-isn-t-one-winner-59356.md>)

Original publisher: [Read original article](<https://hackernoon.com/pipecat-benchmarked-23-real-time-stt-models-for-voice-agents-there-isnt-one-winner?source=rss>)

Author: Speechmatics

Published: 2026-09-24T15:43:05Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [real-time](<https://devfeed.tech/topics/real-time.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [voice ai](<https://devfeed.tech/topics/voice-ai.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [ai-voice-agent](<https://devfeed.tech/tags/ai-voice-agent.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [good-company](<https://devfeed.tech/tags/good-company.md>), [latency](<https://devfeed.tech/tags/latency.md>), [models](<https://devfeed.tech/tags/models.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [pipecat](<https://devfeed.tech/tags/pipecat.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [speech-to-text-recognition](<https://devfeed.tech/tags/speech-to-text-recognition.md>), [stt-api-comparison](<https://devfeed.tech/tags/stt-api-comparison.md>), [stt-evaluation-metrics](<https://devfeed.tech/tags/stt-evaluation-metrics.md>), [voice-agent-stt](<https://devfeed.tech/tags/voice-agent-stt.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>), [voice-ai](<https://devfeed.tech/tags/voice-ai.md>)

### AI overview

An independent Pipecat benchmark compares 23 real-time speech-to-text models for voice agents using streaming latency and semantic accuracy. The results form a Pareto frontier rather than identifying one universal winner, and the benchmark's applicability depends on factors such as VAD behavior, wait periods, and evaluation data.

### Source excerpt

Pipecat benchmarked 23 real-time STT models for voice agents. See how latency and accuracy compare, and why the best model depends on your use case.

[Next page](<https://devfeed.tech/topics/voice-ai.md?cursor=WyIyMDI2LTA5LTI0VDE1OjQzOjA1KzAwOjAwIiwgIjJmOGFjYjVhLTVkYTYtNDExMS1iMTYwLWE2ZmEyMjIwZDk2NSJd>)