# BARS: Beat-Adaptive Rap Synthesis via Flow Matching

DevFeed: [BARS: Beat-Adaptive Rap Synthesis via Flow Matching](<https://devfeed.tech/articles/bars-beat-adaptive-rap-synthesis-via-flow-matching-125506.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12061>)

Author: Taemin Cho, Hector Martel, Enyan Koh

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [audio](<https://devfeed.tech/tags/audio.md>), [coherent](<https://devfeed.tech/tags/coherent.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-mm](<https://devfeed.tech/tags/cs-mm.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generation](<https://devfeed.tech/tags/generation.md>), [masking](<https://devfeed.tech/tags/masking.md>), [rap](<https://devfeed.tech/tags/rap.md>), [rope](<https://devfeed.tech/tags/rope.md>), [speech](<https://devfeed.tech/tags/speech.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [tempo](<https://devfeed.tech/tags/tempo.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

## AI overview

BARS generates rap vocals with Flow Matching, conditioned on an instrumental beat, lyrics, and a short reference voice for zero-shot timbre control. Unlike the prior autoregressive systems described, it allows tempo and output length to vary independently of the beat, without explicit duration prediction or phoneme-level alignment. Lyrics are encoded with pretrained ByT5 and a jointly trained lightweight Transformer adapter, supporting slang, abbreviations, and numerals. Asymmetric Scaled RoPE in cross-attention is designed to improve lyric-audio sequence alignment under strong beat conditioning, which can otherwise lead to skipped or repeated tokens. Span masking enables lyric-level regeneration while preserving surrounding vocal style. The abstract reports high lyric fidelity, rhythmically coherent flow, and tempo control across varied styles and speeds.

## Source excerpt

arXiv:2611.12061v1 Announce Type: cross Abstract: We present BARS (Beat-Adaptive Rap Synthesis), a non-autoregressive rapping voice generation model based on Flow Matching, conditioned on a beat track (instrumental backing), lyrics, and a short reference voice for zero-shot timbre control. Unlike prior autoregressive rap synthesis, where rap pace and output duration are strictly determined by the beat, BARS enables independent control over the rap tempo, allowing flexible-length generation without explicit duration prediction or phoneme-level alignment. We encode lyrics using a pretrained ByT5 encoder with a lightweight Transformer adapter trained jointly with the model, without relying on speech-pretrained text encoders, enabling natural handling of non-standard orthography (e.g., slang, abbreviations) and numerals common in rap lyrics. To address alignment instabilities, such as token skipping or repetition, under strong beat conditioning, we propose Asymmetric Scaled RoPE for cross-attention, which introduces a positional structure to facilitate sequential lyric-audio alignment learning. Furthermore, the model supports partial re-generation via span masking, enabling lyric-level editing while preserving the surrounding vocal style. Experiments demonstrate that the proposed model achieves high lyric fidelity, natural rap flow that is rhythmically coherent with the beat, and robust tempo control across a wide range of rapping styles and speeds.