# tgi

Published articles for tgi.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## SyGra is a low-code/no-code framework for creating and transforming LLM and SLM datasets

DevFeed: [SyGra is a low-code/no-code framework for creating and transforming LLM and SLM datasets](<https://devfeed.tech/articles/sygra-the-one-stop-framework-for-building-data-for-llms-and-slms-7051.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ServiceNow-AI/sygra-data-gen-framework>)

Author: Bidyapati Pradhan; Vipul Mittal; Amit Kumar Saha; Surajit Dasgupta

Published: 2025-09-22T06:45:05Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [datasets](<https://devfeed.tech/topics/datasets.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Low code](<https://devfeed.tech/topics/low-code.md>), [dpo](<https://devfeed.tech/topics/dpo.md>), [Python](<https://devfeed.tech/topics/python.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [llms](<https://devfeed.tech/tags/llms.md>), [low-code](<https://devfeed.tech/tags/low-code.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [python](<https://devfeed.tech/tags/python.md>), [rag](<https://devfeed.tech/tags/rag.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

SyGra is presented as a low-code/no-code Python framework for creating, transforming, and aligning datasets for large and small language models. It supports use cases including Q&A generation, preference-pair creation, reasoning data, multilingual adaptation, quality filtering, and RAG-oriented datasets.

### Source excerpt

You start with a simple dataset, but the model fails on advanced reasoning tasks. How do you generate more complex datasets to strengthen performance? You already have a knowledge base, but it's not in Q&A format. How can you transform it into a usable question-answering dataset? You've prepared a supervised fine-tuning (SFT) dataset. But now you want to align your model using Direct Preference Optimization (DPO). How can you generate preference pairs?

## Introducing the Palmyra-mini family: Powerful, lightweight, and ready to reason!

DevFeed: [Introducing the Palmyra-mini family: Powerful, lightweight, and ready to reason!](<https://devfeed.tech/articles/introducing-the-palmyra-mini-family-powerful-lightweight-and-ready-to-reason-7060.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/Writer/announcing-palmyra-mini>)

Author: Rakshith; Tom Peres

Published: 2025-09-11T20:04:44Z

Content type: news

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [vllm](<https://devfeed.tech/topics/vllm.md>)

Tags: [announce](<https://devfeed.tech/tags/announce.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [inference](<https://devfeed.tech/tags/inference.md>), [models](<https://devfeed.tech/tags/models.md>), [open](<https://devfeed.tech/tags/open.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [release](<https://devfeed.tech/tags/release.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

WRITER announces three open Palmyra-mini models ranging from 1.5B to 1.7B parameters: a lightweight base model and two reasoning variants. The models target efficient inference across varied applications, with GGUF and MLX quantizations available. The article reports benchmark results, describes Chain of Thought training for the reasoning variants, and discusses inference-framework compatibility and reinforcement-learning trade-offs.

### Source excerpt

The team at WRITER is thrilled to announce the release of three new open models in the Palmyra-mini family. These models are designed to be powerful, lightweight, and highly performant for their size (1.5B to 1.7B), making them ideal for a wide range of applications with efficient inference. - palmyra-mini: A powerful, lightweight non-thinking base model. - palmyra-mini-thinking-a: A specialized variant optimized for complex reasoning and logic.

## The Transformers Library: standardizing model definitions

DevFeed: [The Transformers Library: standardizing model definitions](<https://devfeed.tech/articles/the-transformers-library-standardizing-model-definitions-7535.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/transformers-model-definition>)

Author: Lysandre; Arthur Zucker; Pedro Cuenca; Julien Chaumond

Published: 2025-05-15T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Transformer](<https://devfeed.tech/topics/transformer.md>), [LLMs](<https://devfeed.tech/topics/llms.md>), [interoperability](<https://devfeed.tech/topics/interoperability.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [MLX](<https://devfeed.tech/topics/mlx.md>)

Tags: [announcement](<https://devfeed.tech/tags/announcement.md>), [community](<https://devfeed.tech/tags/community.md>), [interoperability](<https://devfeed.tech/tags/interoperability.md>), [library](<https://devfeed.tech/tags/library.md>), [llama-cpp](<https://devfeed.tech/tags/llama-cpp.md>), [llms](<https://devfeed.tech/tags/llms.md>), [mlx](<https://devfeed.tech/tags/mlx.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [vlms](<https://devfeed.tech/tags/vlms.md>)

### AI overview

The article presents Transformers as a standard library for defining and supporting machine learning model architectures. It describes its broad ecosystem integrations, including training frameworks and inference engines, and highlights interoperability with vLLM, SGLang, TGI, llama.cpp, and MLX.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Welcome Llama 4 Maverick & Scout on Hugging Face

DevFeed: [Welcome Llama 4 Maverick & Scout on Hugging Face](<https://devfeed.tech/articles/welcome-llama-4-maverick-scout-on-hugging-face-7338.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/llama4-release>)

Author: ben burtenshaw; Vaibhav Srivastav; Pedro Cuenca; Clem 🤗; Rajat Arya; Jared Sulzdorf; Lysandre

Published: 2025-04-05T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [llama](<https://devfeed.tech/topics/llama.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [multimodal](<https://devfeed.tech/topics/multimodal.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>)

Tags: [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llama](<https://devfeed.tech/tags/llama.md>), [meta](<https://devfeed.tech/tags/meta.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

Hugging Face announces the release of Meta's Llama 4 Maverick and Scout, natively multimodal models built with an autoregressive Mixture-of-Experts architecture. The article describes their parameter sizes, expert counts, training data, language support, quantization options, licensing, and integration with the Hugging Face Hub, Transformers, TRL, and TGI.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Efficient Request Queueing - Optimizing LLM Performance

DevFeed: [Efficient Request Queueing - Optimizing LLM Performance](<https://devfeed.tech/articles/efficient-request-queueing-optimizing-llm-performance-7518.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/tngtech/llm-performance-request-queueing>)

Author: Benjamin Merkel

Published: 2025-04-02T13:33:53Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [backend](<https://devfeed.tech/tags/backend.md>), [batch](<https://devfeed.tech/tags/batch.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [performance](<https://devfeed.tech/tags/performance.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

The article explains how fair request queueing can improve LLM inference performance by batching requests while preventing one user from monopolizing a model backend.

### Source excerpt

A Blog post by TNG Technology Consulting GmbH on Hugging Face

## 🚀 Accelerating LLM Inference with TGI on Intel Gaudi

DevFeed: [🚀 Accelerating LLM Inference with TGI on Intel Gaudi](<https://devfeed.tech/articles/accelerating-llm-inference-with-tgi-on-intel-gaudi-7289.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/intel-gaudi-backend-for-tgi>)

Author: Baptiste Colle; Régis Pierrard; Ilyas Moutawwakil; Ella Charlaix; Ke Ding

Published: 2025-03-28T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [tgi](<https://devfeed.tech/topics/tgi.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [intel](<https://devfeed.tech/topics/intel.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [batching](<https://devfeed.tech/tags/batching.md>), [dell](<https://devfeed.tech/tags/dell.md>), [docker](<https://devfeed.tech/tags/docker.md>), [gaudi](<https://devfeed.tech/tags/gaudi.md>), [gemma](<https://devfeed.tech/tags/gemma.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [ibm](<https://devfeed.tech/tags/ibm.md>), [inference](<https://devfeed.tech/tags/inference.md>), [intel](<https://devfeed.tech/tags/intel.md>), [llm](<https://devfeed.tech/tags/llm.md>), [mixtral](<https://devfeed.tech/tags/mixtral.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [tgi](<https://devfeed.tech/tags/tgi.md>)

### AI overview

This article announces the integration of Intel Gaudi support into the main TGI codebase through TGI's multi-backend architecture. It describes deployment options across Gaudi hardware generations, production features such as dynamic batching and streamed responses, support for multiple LLMs, multi-card inference, vision-language models, FP8 precision, and an official Docker image.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference

DevFeed: [Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference](<https://devfeed.tech/articles/introducing-multi-backends-trt-llm-vllm-support-for-text-generation-inference-7501.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/tgi-multi-backend>)

Author: Morgan Funtowicz; Hugo Larcher

Published: 2025-01-16T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [tgi](<https://devfeed.tech/topics/tgi.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Rust](<https://devfeed.tech/topics/rust.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [TensorRT-LLM](<https://devfeed.tech/topics/tensorrt-llm.md>), [Concurrency](<https://devfeed.tech/topics/concurrency.md>), [Memory Safety](<https://devfeed.tech/topics/memory-safety.md>), [servers](<https://devfeed.tech/topics/servers.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [Python](<https://devfeed.tech/topics/python.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [backend](<https://devfeed.tech/tags/backend.md>), [backends](<https://devfeed.tech/tags/backends.md>), [community](<https://devfeed.tech/tags/community.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [http](<https://devfeed.tech/tags/http.md>), [inference](<https://devfeed.tech/tags/inference.md>), [jetstream](<https://devfeed.tech/tags/jetstream.md>), [llama-cpp](<https://devfeed.tech/tags/llama-cpp.md>), [memory-safety](<https://devfeed.tech/tags/memory-safety.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [performance](<https://devfeed.tech/tags/performance.md>), [production](<https://devfeed.tech/tags/production.md>), [python](<https://devfeed.tech/tags/python.md>), [rust](<https://devfeed.tech/tags/rust.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [tensorrt-llm](<https://devfeed.tech/tags/tensorrt-llm.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

Hugging Face introduces TGI Backends, a unified frontend architecture for integrating inference solutions such as vLLM, SGLang, llama.cpp, and TensorRT-LLM. The approach lets users switch backends based on model, hardware, and performance requirements while supporting production deployment across diverse accelerators. The article also describes TGI's Rust and Python components, including Rust-based HTTP and scheduling layers designed for memory safety and concurrency.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Introducing HUGS - Scale your AI with Open Models

DevFeed: [Introducing HUGS - Scale your AI with Open Models](<https://devfeed.tech/articles/introducing-hugs-scale-your-ai-with-open-models-7255.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/hugs>)

Author: Philipp Schmid; Jeff Boudier; Alvaro Bartolome; Simon Pagezy; Violette

Published: 2024-10-23T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Generative AI](<https://devfeed.tech/topics/generative-ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Microservices](<https://devfeed.tech/topics/microservices.md>), [API](<https://devfeed.tech/topics/api.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [accelerators](<https://devfeed.tech/tags/accelerators.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-accelerator](<https://devfeed.tech/tags/ai-accelerator.md>), [amd](<https://devfeed.tech/tags/amd.md>), [announcement](<https://devfeed.tech/tags/announcement.md>), [api](<https://devfeed.tech/tags/api.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-inferentia](<https://devfeed.tech/tags/aws-inferentia.md>), [azure](<https://devfeed.tech/tags/azure.md>), [digitalocean](<https://devfeed.tech/tags/digitalocean.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [model-deployment](<https://devfeed.tech/tags/model-deployment.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [performance](<https://devfeed.tech/tags/performance.md>), [technologies](<https://devfeed.tech/tags/technologies.md>), [text-generation](<https://devfeed.tech/tags/text-generation.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

Hugging Face introduces HUGS, optimized zero-configuration inference microservices for deploying open models in an organization's own infrastructure. Built on Hugging Face technologies including Text Generation Inference and Transformers, HUGS targets efficient, hardware-optimized deployment across NVIDIA and AMD GPUs, with AWS Inferentia and Google TPU support planned. It provides an OpenAI-compatible API and is designed to reduce deployment complexity and time for AI applications.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Llama can now see and run on your device - welcome Llama 3.2

DevFeed: [Llama can now see and run on your device - welcome Llama 3.2](<https://devfeed.tech/articles/llama-can-now-see-and-run-on-your-device-welcome-llama-3-2-7337.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/llama32>)

Author: merve; Philipp Schmid; Omar Sanseviero; Vaibhav Srivastav; Lewis Tunstall; Aritra Roy Gosthipaty; Pedro Cuenca

Published: 2024-09-25T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [llama](<https://devfeed.tech/topics/llama.md>), [multimodal](<https://devfeed.tech/topics/multimodal.md>), [Meta](<https://devfeed.tech/topics/meta.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [meta](<https://devfeed.tech/tags/meta.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

Meta's Llama 3.2 release introduces multimodal Vision models in 11B and 90B sizes, smaller text-only 1B and 3B models for on-device use, and vision-enabled Llama Guard 3. The article describes their capabilities, architecture, supported languages, inference examples, and integrations with Hugging Face Transformers, TGI, Inference Endpoints, Google Cloud, Amazon SageMaker, and DELL Enterprise Hub.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## TGI Multi-LoRA: Deploy Once, Serve 30 Models

DevFeed: [TGI Multi-LoRA: Deploy Once, Serve 30 Models](<https://devfeed.tech/articles/tgi-multi-lora-deploy-once-serve-30-models-7359.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/multi-lora-serving>)

Author: Derek Thomas; Diego Maniloff; David Holtz

Published: 2024-07-18T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [lora](<https://devfeed.tech/topics/lora.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [AI Inference](<https://devfeed.tech/topics/ai-inference.md>)

Tags: [cost](<https://devfeed.tech/tags/cost.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [guide](<https://devfeed.tech/tags/guide.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llm](<https://devfeed.tech/tags/llm.md>), [lora](<https://devfeed.tech/tags/lora.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [peft](<https://devfeed.tech/tags/peft.md>), [performance](<https://devfeed.tech/tags/performance.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [tgi](<https://devfeed.tech/tags/tgi.md>)

### AI overview

This article introduces TGI Multi-LoRA serving, a feature that allows organizations to deploy one base model and serve many specialized models. It explains how LoRA efficiently fine-tunes large pre-trained models by adding small adapter parameter sets, reducing storage and memory overhead while preserving model quality. The feature addresses the cost and operational complexity of deploying multiple fine-tuned Large Language Models.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Google Cloud TPUs made available to Hugging Face users

DevFeed: [Google Cloud TPUs made available to Hugging Face users](<https://devfeed.tech/articles/google-cloud-tpus-made-available-to-hugging-face-users-7524.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/tpu-inference-endpoints-spaces>)

Author: Simon Pagezy; Michelle Habonneau; Philipp Schmid; Alvaro Moran

Published: 2024-07-09T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Sovereign AI](<https://devfeed.tech/topics/sovereign-ai.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [spaces](<https://devfeed.tech/topics/spaces.md>), [optimum](<https://devfeed.tech/topics/optimum.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [gemma](<https://devfeed.tech/topics/gemma.md>), [llama](<https://devfeed.tech/topics/llama.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>)

Tags: [gcp](<https://devfeed.tech/tags/gcp.md>), [gemma](<https://devfeed.tech/tags/gemma.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llama](<https://devfeed.tech/tags/llama.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [optimum](<https://devfeed.tech/tags/optimum.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [performance](<https://devfeed.tech/tags/performance.md>), [spaces](<https://devfeed.tech/tags/spaces.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [tpu](<https://devfeed.tech/tags/tpu.md>)

### AI overview

Hugging Face announces that Google Cloud TPUs are available for Inference Endpoints and Spaces. Google TPU v5e configurations can deploy supported models through managed infrastructure, while Optimum TPU and Text Generation Inference help train and serve models on TPUs.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Benchmarking Text Generation Inference

DevFeed: [Benchmarking Text Generation Inference](<https://devfeed.tech/articles/benchmarking-text-generation-inference-7500.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/tgi-benchmarking>)

Author: Derek Thomas

Published: 2024-05-29T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [benchmarking](<https://devfeed.tech/topics/benchmarking.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [guide](<https://devfeed.tech/tags/guide.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rag](<https://devfeed.tech/tags/rag.md>), [techniques](<https://devfeed.tech/tags/techniques.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [watermarking](<https://devfeed.tech/tags/watermarking.md>)

### AI overview

This article explains how to benchmark and profile Text Generation Inference (TGI) for different large language model workloads and hardware configurations. It discusses serving optimizations and contrasts the context-window demands of retrieval-augmented generation with basic chat.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Deploy models on AWS Inferentia2 from Hugging Face

DevFeed: [Deploy models on AWS Inferentia2 from Hugging Face](<https://devfeed.tech/articles/deploy-models-on-aws-inferentia2-from-hugging-face-7285.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/inferentia-inference-endpoints>)

Author: Jeff Boudier; Philipp Schmid

Published: 2024-05-22T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [autoscaling](<https://devfeed.tech/topics/autoscaling.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [MLOps](<https://devfeed.tech/topics/mlops.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [mlops](<https://devfeed.tech/tags/mlops.md>), [model](<https://devfeed.tech/tags/model.md>), [optimum](<https://devfeed.tech/tags/optimum.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [tgi](<https://devfeed.tech/tags/tgi.md>)

### AI overview

Hugging Face announces support for deploying models on AWS Inferentia2 through Amazon SageMaker and Hugging Face Inference Endpoints. The update enables scalable inference for supported models, including Meta Llama 3, with multiple instance sizes, managed features, and autoscaling.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Hugging Face x LangChain : A new partner package

DevFeed: [Hugging Face x LangChain : A new partner package](<https://devfeed.tech/articles/hugging-face-x-langchain-a-new-partner-package-7306.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/langchain>)

Author: Joffrey THOMAS; Cyril KONDRATENKO; Erick Friis

Published: 2024-05-14T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [LangChain](<https://devfeed.tech/topics/langchain.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [LLMs](<https://devfeed.tech/topics/llms.md>), [tgi](<https://devfeed.tech/topics/tgi.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [community](<https://devfeed.tech/tags/community.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [langchain](<https://devfeed.tech/tags/langchain.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [rag](<https://devfeed.tech/tags/rag.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [tgi](<https://devfeed.tech/tags/tgi.md>)

### AI overview

This article introduces Hugging Face's partner package for LangChain. It explains how the package exposes Hugging Face pipelines, models, serverless inference endpoints, TGI deployments, chat templates, and embedding models, including integrations with RAG and agent use cases.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Ryght's Journey to Empower Healthcare and Life Sciences with Expert Support from Hugging Face

DevFeed: [Ryght's Journey to Empower Healthcare and Life Sciences with Expert Support from Hugging Face](<https://devfeed.tech/articles/ryght-s-journey-to-empower-healthcare-and-life-sciences-with-expert-support-from-hugging-face-7461.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ryght-case-study>)

Author: Andrew Reed; Johnny Crupi

Published: 2024-04-16T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [AI Platform](<https://devfeed.tech/topics/ai-platform.md>), [Generative AI](<https://devfeed.tech/topics/generative-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [Software as a service](<https://devfeed.tech/topics/saas.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [ai-platform](<https://devfeed.tech/tags/ai-platform.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [case-studies](<https://devfeed.tech/tags/case-studies.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [expert-support](<https://devfeed.tech/tags/expert-support.md>), [expert-support-program](<https://devfeed.tech/tags/expert-support-program.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [healthcare](<https://devfeed.tech/tags/healthcare.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [saas](<https://devfeed.tech/tags/saas.md>), [tgi](<https://devfeed.tech/tags/tgi.md>)

### AI overview

Ryght launched Ryght Preview, an enterprise-grade generative AI SaaS platform for healthcare and life sciences. The platform provides industry-specific AI copilots and custom solutions for analyzing and documenting complex data sources. Hugging Face supports the company through its Expert Support Program, including advisory meetings, technical workshops, and access to Text Generation Inference (TGI) and Text Embeddings Inference (TEI) services.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.