# Faster assisted generation support for Intel Gaudi

DevFeed: [Faster assisted generation support for Intel Gaudi](<https://devfeed.tech/articles/faster-assisted-generation-support-for-intel-gaudi-7108.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/assisted-generation-support-gaudi>)

Author: Haim Barad; Neha Raste; Tien Pei Chou

Published: 2024-06-04T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [intel](<https://devfeed.tech/topics/intel.md>), [text-generation](<https://devfeed.tech/topics/text-generation.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [optimum](<https://devfeed.tech/topics/optimum.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [ai](<https://devfeed.tech/tags/ai.md>), [diffusers](<https://devfeed.tech/tags/diffusers.md>), [gaudi](<https://devfeed.tech/tags/gaudi.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [inference](<https://devfeed.tech/tags/inference.md>), [intel](<https://devfeed.tech/tags/intel.md>), [latency](<https://devfeed.tech/tags/latency.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [optimum](<https://devfeed.tech/tags/optimum.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [techniques](<https://devfeed.tech/tags/techniques.md>), [text-generation](<https://devfeed.tech/tags/text-generation.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

## AI overview

The article explains how assisted decoding and speculative sampling were adapted and optimized for Intel Gaudi processors. Integrated into Optimum Habana, these techniques use draft and target models, KV caching, and quantized models to accelerate text generation while preserving the target model's sampling quality.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.