# Powerful ASR + diarization + speculative decoding with Hugging Face Inference Endpoints

DevFeed: [Powerful ASR + diarization + speculative decoding with Hugging Face Inference Endpoints](<https://devfeed.tech/articles/powerful-asr-diarization-speculative-decoding-with-hugging-face-inference-endpoints-7106.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/asr-diarization>)

Author: Sergei Petrov; Vaibhav Srivastav; Pedro Cuenca; Philipp Schmid

Published: 2024-05-01T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Whisper](<https://devfeed.tech/topics/whisper.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>)

Tags: [asr](<https://devfeed.tech/tags/asr.md>), [audio](<https://devfeed.tech/tags/audio.md>), [flash-attention-2](<https://devfeed.tech/tags/flash-attention-2.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [whisper](<https://devfeed.tech/tags/whisper.md>)

## AI overview

This article explains how to build a custom inference handler for Automatic Speech Recognition, speaker diarization, and speculative decoding on Hugging Face Inference Endpoints. It covers modular pipeline design, repository files, Pyannote-based diarization, PyTorch SDPA with Flash Attention 2, and constraints on speculative decoding such as batch size one and compatible decoder architectures.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.