# Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference

DevFeed: [Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference](<https://devfeed.tech/articles/introducing-multi-backends-trt-llm-vllm-support-for-text-generation-inference-7501.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/tgi-multi-backend>)

Author: Morgan Funtowicz; Hugo Larcher

Published: 2025-01-16T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [tgi](<https://devfeed.tech/topics/tgi.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Rust](<https://devfeed.tech/topics/rust.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [TensorRT-LLM](<https://devfeed.tech/topics/tensorrt-llm.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [Memory Safety](<https://devfeed.tech/topics/memory-safety.md>), [servers](<https://devfeed.tech/topics/servers.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [Python](<https://devfeed.tech/topics/python.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [backend](<https://devfeed.tech/tags/backend.md>), [backends](<https://devfeed.tech/tags/backends.md>), [community](<https://devfeed.tech/tags/community.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [http](<https://devfeed.tech/tags/http.md>), [inference](<https://devfeed.tech/tags/inference.md>), [jetstream](<https://devfeed.tech/tags/jetstream.md>), [llama-cpp](<https://devfeed.tech/tags/llama-cpp.md>), [memory-safety](<https://devfeed.tech/tags/memory-safety.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [performance](<https://devfeed.tech/tags/performance.md>), [production](<https://devfeed.tech/tags/production.md>), [python](<https://devfeed.tech/tags/python.md>), [rust](<https://devfeed.tech/tags/rust.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [tensorrt-llm](<https://devfeed.tech/tags/tensorrt-llm.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

## AI overview

Hugging Face introduces TGI Backends, a unified frontend architecture for integrating inference solutions such as vLLM, SGLang, llama.cpp, and TensorRT-LLM. The approach lets users switch backends based on model, hardware, and performance requirements while supporting production deployment across diverse accelerators. The article also describes TGI's Rust and Python components, including Rust-based HTTP and scheduling layers designed for memory safety and concurrency.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.