# Faster Assisted Generation with Dynamic Speculation

DevFeed: [Faster Assisted Generation with Dynamic Speculation](<https://devfeed.tech/articles/faster-assisted-generation-with-dynamic-speculation-7177.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/dynamic_speculation_lookahead>)

Author: Jonathan Mamou; Oren Pereg; Joao Gante; Lewis Tunstall; Daniel Korat; Nadav Timor; Moshe Wasserblat

Published: 2024-10-08T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI Inference](<https://devfeed.tech/topics/ai-inference.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [large-language-models](<https://devfeed.tech/topics/large-language-models.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [Latency](<https://devfeed.tech/topics/latency.md>)

Tags: [inference](<https://devfeed.tech/tags/inference.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [latency](<https://devfeed.tech/tags/latency.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [research](<https://devfeed.tech/tags/research.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

## AI overview

The article explains dynamic speculation lookahead for accelerating large language model inference through speculative decoding. A fast draft model generates candidate tokens, while a larger target model verifies them in parallel. It presents an oracle-based method that identifies the maximum number of draft tokens accepted at each iteration, aiming to reduce inference latency beyond static and heuristic schedules.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.