# Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4

DevFeed: [Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4](<https://devfeed.tech/articles/streaming-speech-recognition-on-a-microcontroller-a-65-7m-parameter-transducer-in-2-and-4-bits-on-the-esp32-p4-108719.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.38739>)

Author: Billal Boumedine

Published: 2026-10-30T04:00:00Z

Content type: news

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [ESP32-P4](<https://devfeed.tech/topics/esp32-p4.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [esp32-p4](<https://devfeed.tech/tags/esp32-p4.md>), [firmware](<https://devfeed.tech/tags/firmware.md>), [gptq](<https://devfeed.tech/tags/gptq.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [microcontroller](<https://devfeed.tech/tags/microcontroller.md>), [pie](<https://devfeed.tech/tags/pie.md>), [psram](<https://devfeed.tech/tags/psram.md>), [risc-v](<https://devfeed.tech/tags/risc-v.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-recognition](<https://devfeed.tech/tags/speech-recognition.md>), [speed](<https://devfeed.tech/tags/speed.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [streaming-speech-recognition](<https://devfeed.tech/tags/streaming-speech-recognition.md>)

## AI overview

The 65.7M-parameter Kroko streaming speech recognizer runs on the ESP32-P4 without a neural accelerator, processing each 1.28-second audio chunk in 0.90 seconds on both cores, including beam search. The C engine uses hand-written SIMD kernels and compresses 71 MB of int8 weights to 26.65 MB by quantizing encoder matrices to 2 or 4 bits with sensitivity-based groups, output correction, and GPTQ; the reported ablation found GPTQ did most of the work. LibriSpeech WER is 4.34% on test-clean and 10.26% on test-other, compared with 3.94% and 9.17% for the released model, with larger reported gaps on FLEURS and Common Voice. The authors attribute the accuracy gap to weight quantization, not integer arithmetic. A 16 MB target was not reached (the attempt stopped at 19.63 MB), and energy use was not measured.

## Source excerpt

arXiv:2610.38739v1 Announce Type: cross Abstract: We run Kroko, a 65.7M-parameter streaming Zipformer2 transducer for English, on the ESP32-P4, a microcontroller with two 400 MHz RISC-V cores, a 128-bit SIMD extension (PIE) and 32 MB of PSRAM, but no neural accelerator. Without any training, the encoder's 290 matrices are stored with 2 or 4 bits per weight in Hadamard-rotated groups: the width of each matrix comes from its measured sensitivity, and each matrix is first corrected toward the original model's output, then rounded with GPTQ and a search over the group norms. The released 71 MB of int8 weights become a 26.65 MB file. Activations are int16 between the matrix products and int8 into them. A C engine with hand-written PIE kernels processes each 1.28 s chunk of audio in 0.90 s on both cores, beam search included, and its PC build computes the same bits as the chip, so accuracy is measured on whole test sets. LibriSpeech WER is 4.34% on test-clean and 10.26% on test-other, against 3.94% and 9.17% for the released model; on FLEURS and Common Voice the gap grows to 2.3 and 1.1 points. A GPU simulator shows that the integer arithmetic costs nothing and that the whole gap comes from the weights, and an ablation on all of test-clean shows that GPTQ does most of the work. We also report the port's speed history, from 51.3 s to 0.62 s of encoder time per chunk, each design step measured again in one build, an attempt to reach 16 MB that stopped at 19.63 MB, and what did not work. We did not measure energy. The inference engine, the firmware and the 2/4-bit weights are available at https://github.com/BoumedineBillal/esp32-pie-asr.