# NeuroCodec: EEG-Guided Latent-Domain Generative Model for High-Fidelity Target Speaker Extraction

DevFeed: [NeuroCodec: EEG-Guided Latent-Domain Generative Model for High-Fidelity Target Speaker Extraction](<https://devfeed.tech/articles/neurocodec-eeg-guided-latent-domain-generative-model-for-high-fidelity-target-speaker-extraction-94986.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.29815>)

Author: Julian Silva, Jaliya Kodithuwakku, Navindu Sirimanna, Shaveen Herath, Wageesha N. Manamperi, Joshua P. Kulasingham

Published: 2026-10-22T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformer](<https://devfeed.tech/topics/transformer.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>), [Generative Adversarial Network](<https://devfeed.tech/topics/generative-adversarial-network.md>), [fourier transform](<https://devfeed.tech/topics/fourier-transform.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [attention](<https://devfeed.tech/tags/attention.md>), [audio](<https://devfeed.tech/tags/audio.md>), [codec](<https://devfeed.tech/tags/codec.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [eeg](<https://devfeed.tech/tags/eeg.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [generative](<https://devfeed.tech/tags/generative.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [masking](<https://devfeed.tech/tags/masking.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [neural](<https://devfeed.tech/tags/neural.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [party](<https://devfeed.tech/tags/party.md>), [processing](<https://devfeed.tech/tags/processing.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [regions](<https://devfeed.tech/tags/regions.md>), [space](<https://devfeed.tech/tags/space.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [speaker](<https://devfeed.tech/tags/speaker.md>), [target](<https://devfeed.tech/tags/target.md>)

## AI overview

NeuroCodec uses EEG to guide target speaker extraction by generating speech in a neural audio codec's latent space, addressing fidelity limits of time-frequency masking in overlapping speech and narrowband audio. It models spatial EEG dependencies with graph convolutions and combines EEG and speech representations using attention and Mamba-based temporal modeling. A six-phase curriculum adds latent reconstruction, multi-scale spectral, and adversarial objectives. On the Cocktail Party and KUL datasets, it outperformed baselines on several perceptual metrics; in a 20-person listening study, its mean score was 82.24 versus 49.63 for M3ANet. The abstract reports 44.1 kHz reconstruction with spectral content up to 20 kHz and a causal streaming formulation for real-time processing.

## Source excerpt

arXiv:2610.29815v1 Announce Type: cross Abstract: Driven by recent advancements in hearing assistive devices, target speaker extraction (TSE) in multi-talker environments has drawn significant attention. To address this, hearing technologies identify attention using neural signals recorded via electroencephalography (EEG) to guide TSE, enabling neuro-steered auditory attention decoding. However, most existing techniques rely on time-frequency masking, which suffers from low fidelity in spectro-temporally overlapping regions and is often restricted to narrowband audio. In contrast, we introduce NeuroCodec, which, to the best of our knowledge, is the first generative EEG-guided TSE framework operating in the latent space of a neural audio codec. Specifically, NeuroCodec models spatial EEG dependencies via graph convolutions and fuses cross-modal EEG and speech representations through attention mechanisms and Mamba-based temporal modeling. We further introduce a six-phase training curriculum that progressively incorporates latent reconstruction, multi-scale spectral, and adversarial objectives to ensure stable, high-fidelity speech generation. Experiments on the public Cocktail Party and KUL datasets show NeuroCodec outperforms baselines, across several perceptual metrics. A listening experiment with 20 participants validated that NeuroCodec offers greater perceptual accuracy, achieving a mean score of 82.24 compared with 49.63 for M3ANet. Additionally, the proposed method is seen to enable 44.1 kHz reconstruction with spectral content extending up to 20 kHz, further supporting the perceptual results. Finally, we extend NeuroCodec with a causal streaming formulation suited for real-time processing, providing a basis for hearing assistive devices.