# inference-endpoints

Hugging Face managed service for deploying AI models to production on dedicated, autoscaling infrastructure.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Pay for Hugging Face Hub with your Google Cloud Account

DevFeed: [Pay for Hugging Face Hub with your Google Cloud Account](<https://devfeed.tech/articles/pay-for-hugging-face-hub-with-your-google-cloud-account-127705.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/hub-gcp-marketplace>)

Author: Zaïd Alhadad; Pierric Cistac; Simon Pagezy; Michelle Habonneau

Published: 2026-11-10T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [inference-providers](<https://devfeed.tech/topics/inference-providers.md>), [jobs](<https://devfeed.tech/topics/jobs.md>), [spaces](<https://devfeed.tech/topics/spaces.md>)

Tags: [audit-logs](<https://devfeed.tech/tags/audit-logs.md>), [billing](<https://devfeed.tech/tags/billing.md>), [compute](<https://devfeed.tech/tags/compute.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [google-cloud-marketplace](<https://devfeed.tech/tags/google-cloud-marketplace.md>), [guide](<https://devfeed.tech/tags/guide.md>), [hub](<https://devfeed.tech/tags/hub.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [inference-providers](<https://devfeed.tech/tags/inference-providers.md>), [jobs](<https://devfeed.tech/tags/jobs.md>), [linking](<https://devfeed.tech/tags/linking.md>), [marketplace](<https://devfeed.tech/tags/marketplace.md>), [models](<https://devfeed.tech/tags/models.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>)

### AI overview

Hugging Face organizations can route eligible Hub charges through a linked Google Cloud billing account by subscribing to the Hugging Face listing in Google Cloud Marketplace, choosing a project and billing account, then signing in to Hugging Face and selecting an organization where the user is an admin. The subscription requires Hugging Face review; while pending, existing billing continues. After approval, organizations with a credit card switch at the next billing period, while free-tier organizations without one usually switch immediately. The marketplace account replaces the credit card, so adding credits is unavailable. Team or Enterprise included credits apply before pay-as-you-go charges. Usage can be monitored in Hugging Face billing settings and Google Cloud Billing reports; linking and unlinking are recorded in organization audit logs.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Welcome Le Chonk: Mistral Large 4 on Hugging Face

DevFeed: [Welcome Le Chonk: Mistral Large 4 on Hugging Face](<https://devfeed.tech/articles/welcome-le-chonk-mistral-large-4-on-hugging-face-107127.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/mistral-large4>)

Author: merve; Alvaro Bartolome; Rémi Ouazan Reboul; Cyril Vallez; Quentin Gallouédec; Enrique Hernández Calabrés; Juan Julián; Xuan-Son Nguyen; Pedro Cuenca; ben burtenshaw

Published: 2026-10-29T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Generative AI](<https://devfeed.tech/topics/generative-ai.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Self Hosted AI Tools](<https://devfeed.tech/topics/self-hosted-ai-tools.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [peft](<https://devfeed.tech/topics/peft.md>), [lora](<https://devfeed.tech/topics/lora.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-workflows](<https://devfeed.tech/tags/agentic-workflows.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-servers](<https://devfeed.tech/tags/ai-servers.md>), [amd](<https://devfeed.tech/tags/amd.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [api](<https://devfeed.tech/tags/api.md>), [app](<https://devfeed.tech/tags/app.md>), [apple](<https://devfeed.tech/tags/apple.md>), [apple-silicon](<https://devfeed.tech/tags/apple-silicon.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [artificial-analysis](<https://devfeed.tech/tags/artificial-analysis.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [le-chonk](<https://devfeed.tech/tags/le-chonk.md>), [llamacpp](<https://devfeed.tech/tags/llamacpp.md>), [mistral](<https://devfeed.tech/tags/mistral.md>), [mistral-large-4](<https://devfeed.tech/tags/mistral-large-4.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

Mistral Large 4 ("Le Chonk") is an open weight multimodal model with about 1T total and 49B active parameters, a 1M token context window, and a sparse MoE decoder paired with a Pixtral vision encoder. The article emphasizes defensive cybersecurity: it reports 82% on vulnerability reproduction and patching and 93% on Cybench, while also noting strong refusal behavior on harmful cyber prompts. It describes support for Transformers, vLLM, and llama.cpp, hosted Inference Endpoints, and TRL fine tuning. Runtime and hardware needs vary by route: the vLLM NVFP4 checkpoint requires NVIDIA Blackwell GPUs, and llama.cpp's 2-bit quant can run on a single 512 GB Apple Silicon Ultra, while other listed quants require distributed inference. Reasoning effort supports low, medium, or high, defaults to high, and cannot be disabled.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Using the Hugging Face API from an AI Agent

DevFeed: [Using the Hugging Face API from an AI Agent](<https://devfeed.tech/articles/using-the-hugging-face-api-from-an-ai-agent-97560.md>)

Original publisher: [Read original article](<https://www.swytchcode.com/content/hugging-face-api-ai-agent>)

Author: Chaitrali Kakde

Published: 2026-10-23T13:32:34.359184Z

Content type: tutorial

Language: en

Sources: [Swytchcode](<https://devfeed.tech/sources/swytchcode.md>)

Topics: [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [personal access token](<https://devfeed.tech/topics/personal-access-token.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [API](<https://devfeed.tech/topics/api.md>), [API Governance](<https://devfeed.tech/topics/api-governance.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [API Logging](<https://devfeed.tech/topics/api-logging.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Claude Code](<https://devfeed.tech/topics/claude-code.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [account](<https://devfeed.tech/tags/account.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [hub](<https://devfeed.tech/tags/hub.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [lets](<https://devfeed.tech/tags/lets.md>), [model](<https://devfeed.tech/tags/model.md>), [scopes](<https://devfeed.tech/tags/scopes.md>), [token](<https://devfeed.tech/tags/token.md>), [using](<https://devfeed.tech/tags/using.md>), [vs](<https://devfeed.tech/tags/vs.md>), [write](<https://devfeed.tech/tags/write.md>)

### AI overview

Hugging Face access for agents covers three distinct operations: Hub reads, inference, and Hub writes. The article recommends a per-agent fine-grained token scoped to only the needed model or organization; broad read tokens can expose private and organizational repositories, while write tokens also permit edits and should not be given to agents. It describes storing credentials outside the project and allowlisting separate inference and read methods. For reliability, treat cold-model responses as retryable, validate results because an HTTP 200 can still contain an error or empty output, and cache inference by model and input hash to reduce repeated cost and quota use. It also recommends pinning approved models, limiting batch size, and reviewing model changes. Shared inference is cheaper for scheduled batch work but can have variable latency; a dedicated endpoint may suit user-facing requests. Sending customer text to shared inference raises a data-transfer review question.

### Source excerpt

"The Hugging Face API" is three different APIs, and a write token lets your agent edit your model cards. Token scopes, inference vs hub reads, and how to lock it down.

## Amazon SageMaker Inference: 2026 year-to-date launches in review

DevFeed: [Amazon SageMaker Inference: 2026 year-to-date launches in review](<https://devfeed.tech/articles/amazon-sagemaker-inference-2026-year-to-date-launches-in-review-55156.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-inference-2026-year-to-date-launches-in-review/>)

Author: Kareem Syed-Mohammed

Published: 2026-09-18T20:52:14Z

Content type: release

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Amazon SageMaker HyperPod](<https://devfeed.tech/topics/amazon-sagemaker-hyperpod.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>)

Tags: [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [latency](<https://devfeed.tech/tags/latency.md>), [optimization](<https://devfeed.tech/tags/optimization.md>)

### AI overview

Amazon SageMaker AI shipped 13 inference capabilities in 2026 across fully managed endpoints and SageMaker HyperPod Inference. The post reviews launches covering deployment, capacity, scaling, observability, benchmarking, and optimization for generative AI workloads.

### Source excerpt

Amazon SageMaker AI shipped 13 inference launches in year-to-date across two deployment paths: fully managed endpoints and Amazon SageMaker HyperPod Inference. This post reviews each launch, from inference recommendations and capacity-aware instance pools to tiered KV caching and disaggregated prefill and decode.

## Deploy Hugging Face models on Amazon SageMaker AI with coding agents

DevFeed: [Deploy Hugging Face models on Amazon SageMaker AI with coding agents](<https://devfeed.tech/articles/deploy-hugging-face-models-on-amazon-sagemaker-ai-with-coding-agents-55157.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/deploy-hugging-face-models-on-amazon-sagemaker-ai-with-coding-agents/>)

Author: Dario Salvati

Published: 2026-09-18T15:25:23Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Agent Skills](<https://devfeed.tech/topics/agent-skills.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [autoscaling](<https://devfeed.tech/topics/autoscaling.md>), [Amazon CloudWatch](<https://devfeed.tech/topics/amazon-cloudwatch.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>)

Tags: [agent-skills](<https://devfeed.tech/tags/agent-skills.md>), [amazon-cloudwatch](<https://devfeed.tech/tags/amazon-cloudwatch.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [claude](<https://devfeed.tech/tags/claude.md>), [claude-code](<https://devfeed.tech/tags/claude-code.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [technical-how-to](<https://devfeed.tech/tags/technical-how-to.md>)

### AI overview

A tutorial on using six open-source agent skills to deploy Hugging Face models on Amazon SageMaker AI. It covers selecting serving containers, configuring autoscaling and CloudWatch alarms, supporting multiple inference modes, and verifying teardown paths.

### Source excerpt

Deploy production-ready Hugging Face models on Amazon SageMaker AI using six open-source agent skills. Point a coding agent at a model and get back a real-time endpoint with the right serving container, autoscaling, Amazon CloudWatch alarms, and a verified teardown path.

## How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code

DevFeed: [How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code](<https://devfeed.tech/articles/how-hugging-face-inference-endpoints-jobs-and-buckets-power-search-on-papers-with-code-7447.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/pwc-search>)

Author: Niels Rogge

Published: 2026-08-21T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AI search](<https://devfeed.tech/topics/ai-search.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [jobs](<https://devfeed.tech/topics/jobs.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>)

Tags: [database](<https://devfeed.tech/tags/database.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [jobs](<https://devfeed.tech/tags/jobs.md>), [latency](<https://devfeed.tech/tags/latency.md>), [rag](<https://devfeed.tech/tags/rag.md>), [research](<https://devfeed.tech/tags/research.md>), [search](<https://devfeed.tech/tags/search.md>), [storage](<https://devfeed.tech/tags/storage.md>)

### AI overview

This article explains how Papers with Code uses hybrid search to find research papers through exact keyword matching and semantic vector search. The production system combines PostgreSQL full-text search, pgvector embeddings, reciprocal rank fusion, and Hugging Face Jobs, Storage Buckets, and Inference Endpoints.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Why Enterprise Platforms Need to Adapt for AI Coding, Agents, and Infrastructure Costs

DevFeed: [Why Enterprise Platforms Need to Adapt for AI Coding, Agents, and Infrastructure Costs](<https://devfeed.tech/articles/your-platform-was-built-for-a-different-era-and-ai-just-exposed-it-47961.md>)

Original publisher: [Read original article](<https://www.nextplatform.com/cloud/2026/08/06/sponsored-your-platform-was-built-for-a-different-era-and-ai-just-exposed-it/5284205>)

Author: Pankaj Gupta

Published: 2026-08-06T15:10:12Z

Content type: article

Language: en

Sources: [The Next Platform: In-depth coverage of high end computing](<https://devfeed.tech/sources/the-next-platform-in-depth-coverage-of-high-end-computing.md>)

Topics: [AI Platform](<https://devfeed.tech/topics/ai-platform.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Provisioning](<https://devfeed.tech/topics/provisioning.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [jobs](<https://devfeed.tech/topics/jobs.md>), [finops](<https://devfeed.tech/topics/finops.md>), [Model Context Protocol](<https://devfeed.tech/topics/model-context-protocol.md>), [Authentication](<https://devfeed.tech/topics/authentication.md>), [Authorization](<https://devfeed.tech/topics/authorization.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-assisted-development](<https://devfeed.tech/tags/ai-assisted-development.md>), [ai-coding](<https://devfeed.tech/tags/ai-coding.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [broadcom](<https://devfeed.tech/tags/broadcom.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [finops](<https://devfeed.tech/tags/finops.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [jobs](<https://devfeed.tech/tags/jobs.md>), [logging](<https://devfeed.tech/tags/logging.md>), [mcp](<https://devfeed.tech/tags/mcp.md>), [permissions](<https://devfeed.tech/tags/permissions.md>), [sponsored](<https://devfeed.tech/tags/sponsored.md>), [vmware](<https://devfeed.tech/tags/vmware.md>)

### AI overview

This sponsored article argues that enterprise internal platforms built for conventional containerized application development must adapt to AI workloads. It highlights increased code volume from AI coding assistants, platform requirements for AI agents, and rising costs from GPUs, inference endpoints, training jobs, tokens, and retries.

### Source excerpt

SPONSORED: Your Platform Was Built For A Different Era, And AI Just Exposed It

## Why AI Requires a New Approach to Platform Engineering

DevFeed: [Why AI Requires a New Approach to Platform Engineering](<https://devfeed.tech/articles/platform-engineering-2-0-your-platform-was-built-for-a-different-era-ai-just-exposed-it-47926.md>)

Original publisher: [Read original article](<https://www.devclass.com/ml-ai/2026/08/06/partner-content-platform-engineering-20-your-platform-was-built-for-a-different-era-ai-just-exposed-it/5283935>)

Author: Pankaj Gupta

Published: 2026-08-06T15:00:00Z

Content type: article

Language: en

Sources: [DevClass.com](<https://devfeed.tech/sources/devclass-com.md>)

Topics: [Platform Engineering](<https://devfeed.tech/topics/platform-engineering.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [finops](<https://devfeed.tech/topics/finops.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Authentication](<https://devfeed.tech/topics/authentication.md>), [Authorization](<https://devfeed.tech/topics/authorization.md>), [jobs](<https://devfeed.tech/topics/jobs.md>), [Model Context Protocol](<https://devfeed.tech/topics/model-context-protocol.md>), [Logging](<https://devfeed.tech/topics/logging.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-coding-assistants](<https://devfeed.tech/tags/ai-coding-assistants.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [finops](<https://devfeed.tech/tags/finops.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [jobs](<https://devfeed.tech/tags/jobs.md>), [logging](<https://devfeed.tech/tags/logging.md>), [ml-ai](<https://devfeed.tech/tags/ml-ai.md>), [permissions](<https://devfeed.tech/tags/permissions.md>), [platform](<https://devfeed.tech/tags/platform.md>), [platform-engineering](<https://devfeed.tech/tags/platform-engineering.md>)

### AI overview

This partner-content article argues that internal platforms designed for human developers and containerized applications must evolve for AI-assisted development and AI agents. It highlights requirements including GPU provisioning, agent authentication and permissions, MCP compatibility, audit logging, and cost controls for AI infrastructure and token usage.

### Source excerpt

The debate about whether enterprises need an internal platform is over. Google's 2025 DORA research ...

## Connect OpenSearch to private ML endpoints

DevFeed: [Connect OpenSearch to private ML endpoints](<https://devfeed.tech/articles/connect-opensearch-to-private-ml-endpoints-12786.md>)

Original publisher: [Read original article](<https://opensearch.org/blog/connect-opensearch-to-private-ml-endpoints/>)

Author: Nathalie Jonathan

Published: 2026-07-28T15:00:41Z

Content type: tutorial

Language: en

Sources: [OpenSearch](<https://devfeed.tech/sources/opensearch.md>)

Topics: [Amazon OpenSearch Service](<https://devfeed.tech/topics/amazon-opensearch-service.md>), [network security](<https://devfeed.tech/topics/network-security.md>), [Amazon VPC](<https://devfeed.tech/topics/amazon-vpc.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Security](<https://devfeed.tech/topics/security.md>), [AWS IAM](<https://devfeed.tech/topics/aws-iam.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Self-hosted](<https://devfeed.tech/topics/self-hosted.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [aws-identity-and-access-management-iam](<https://devfeed.tech/tags/aws-identity-and-access-management-iam.md>), [blog](<https://devfeed.tech/tags/blog.md>), [firewalls](<https://devfeed.tech/tags/firewalls.md>), [inference](<https://devfeed.tech/tags/inference.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [opensearch](<https://devfeed.tech/tags/opensearch.md>), [production](<https://devfeed.tech/tags/production.md>), [self-hosted](<https://devfeed.tech/tags/self-hosted.md>), [vpc](<https://devfeed.tech/tags/vpc.md>)

### AI overview

This guide explains how to connect OpenSearch and Amazon OpenSearch Service to machine learning models hosted on private infrastructure. It covers VPC-hosted endpoints, private SageMaker endpoints, internal API gateways, and self-hosted inference servers, with configuration steps for connectors, model registration, deployment, testing, and VPC egress.

### Source excerpt

Connect OpenSearch to ML models hosted on private infrastructure. Configure ML Commons connectors for VPC-hosted endpoints, private SageMaker models, and internal inference servers without exposing services to the public internet The post Connect OpenSearch to private ML endpoints appeared first on OpenSearch.

## Deploy and inference any model from HuggingFace

DevFeed: [Deploy and inference any model from HuggingFace](<https://devfeed.tech/articles/deploy-and-inference-any-model-from-huggingface-80171.md>)

Original publisher: [Read original article](<https://www.together.ai/blog/deploy-and-inference-any-model-from-huggingface>)

Author: Blaine Kasten

Published: 2026-05-08T00:00:00Z

Content type: tutorial

Language: en

Sources: [Together.ai](<https://devfeed.tech/sources/together-ai.md>)

Topics: [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [smolagents](<https://devfeed.tech/topics/smolagents.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>)

Tags: [deploy](<https://devfeed.tech/tags/deploy.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [huggingface](<https://devfeed.tech/tags/huggingface.md>), [inference](<https://devfeed.tech/tags/inference.md>), [model](<https://devfeed.tech/tags/model.md>)

### AI overview

The article walks through using Goose with Together's Dedicated Container Inference to deploy Netflix's void-model from Hugging Face in a GPU-backed container. It describes the agent's setup steps and asynchronous inference workflow, and explains how DCI provides managed infrastructure for running models.

### Source excerpt

Learn how to deploy any Hugging Face model in one session using Goose and Together's Dedicated Container Inference. Skip the setup complexity -- one prompt gets your model running in a production-grade GPU environment on release day.

## Using Elasticsearch Inference API along with Hugging Face models

DevFeed: [Using Elasticsearch Inference API along with Hugging Face models](<https://devfeed.tech/articles/using-elasticsearch-inference-api-along-with-hugging-face-models-78849.md>)

Original publisher: [Read original article](<https://www.elastic.co/search-labs/blog/hugging-face-elasticsearch-inference-api>)

Author: Jeffrey Rengifo

Published: 2026-03-23T00:00:00Z

Content type: tutorial

Language: en

Sources: [Elasticsearch Labs](<https://devfeed.tech/sources/elasticsearch-labs.md>)

Topics: [elasticsearch](<https://devfeed.tech/topics/elasticsearch.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [smolvlm](<https://devfeed.tech/topics/smolvlm.md>), [Google Chat](<https://devfeed.tech/topics/google-chat.md>)

Tags: [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai-recommendations](<https://devfeed.tech/tags/ai-recommendations.md>), [connect](<https://devfeed.tech/tags/connect.md>), [elasticsearch](<https://devfeed.tech/tags/elasticsearch.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [integrations](<https://devfeed.tech/tags/integrations.md>), [models](<https://devfeed.tech/tags/models.md>), [semantic-search](<https://devfeed.tech/tags/semantic-search.md>)

### AI overview

This tutorial shows how to connect Elasticsearch to Hugging Face inference endpoints and use SmolLM3-3B with semantic search to recommend multilingual blog articles. It walks through configuring the inference endpoint, indexing content, retrieving relevant articles, and generating recommendation text with chat completions.

### Source excerpt

Learn how to connect Elasticsearch to Hugging Face models using inference endpoints, and build a multilingual blog recommendation system with semantic search and chat completions.

## NVIDIA brings agents to life with DGX Spark and Reachy Mini

DevFeed: [NVIDIA brings agents to life with DGX Spark and Reachy Mini](<https://devfeed.tech/articles/nvidia-brings-agents-to-life-with-dgx-spark-and-reachy-mini-7371.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/nvidia-reachy-mini>)

Author: Jeff Boudier; Nader Khalil; Alec Fong

Published: 2026-01-05T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [DGX Spark](<https://devfeed.tech/topics/dgx-spark.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [reachy](<https://devfeed.tech/topics/reachy.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Orchestration](<https://devfeed.tech/topics/orchestration.md>), [Python](<https://devfeed.tech/topics/python.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [code](<https://devfeed.tech/tags/code.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [dgx-spark](<https://devfeed.tech/tags/dgx-spark.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [python](<https://devfeed.tech/tags/python.md>), [reachy](<https://devfeed.tech/tags/reachy.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [serverless](<https://devfeed.tech/tags/serverless.md>)

### AI overview

A step-by-step guide to building a desk-based AI companion with NVIDIA DGX Spark and Reachy Mini. It combines open reasoning and vision models, text-to-speech, Python, agent orchestration, and tool handlers, with options for local, cloud, and serverless deployment.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Building for an Open Future - our new partnership with Google Cloud

DevFeed: [Building for an Open Future - our new partnership with Google Cloud](<https://devfeed.tech/articles/building-for-an-open-future-our-new-partnership-with-google-cloud-7218.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/google-cloud>)

Author: Jeff Boudier; Simon Pagezy

Published: 2025-11-13T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Cloud Run](<https://devfeed.tech/topics/cloud-run.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [xet](<https://devfeed.tech/topics/xet.md>), [data](<https://devfeed.tech/topics/data.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>)

Tags: [announcement](<https://devfeed.tech/tags/announcement.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [google](<https://devfeed.tech/tags/google.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [networking](<https://devfeed.tech/tags/networking.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [storage](<https://devfeed.tech/tags/storage.md>), [vertex](<https://devfeed.tech/tags/vertex.md>), [vertex-ai](<https://devfeed.tech/tags/vertex-ai.md>), [xet](<https://devfeed.tech/tags/xet.md>)

### AI overview

Hugging Face and Google Cloud announce a strategic partnership focused on making open AI models easier to use, customize, deploy, and govern. The article describes integrations across Vertex AI, GKE AI/ML, Cloud Run GPUs, and other Google Cloud infrastructure, plus a planned CDN Gateway using Hugging Face Xet and Google Cloud storage and networking to accelerate model and dataset downloads and improve supply-chain robustness.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Blazingly fast whisper transcriptions with Inference Endpoints

DevFeed: [Blazingly fast whisper transcriptions with Inference Endpoints](<https://devfeed.tech/articles/blazingly-fast-whisper-transcriptions-with-inference-endpoints-7192.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/fast-whisper-endpoints>)

Author: Morgan Funtowicz; Freddy Boulton; Steven Zheng; Vaibhav Srivastav; Erik Kaunismäki; Michelle Habonneau

Published: 2025-05-13T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [GPU](<https://devfeed.tech/topics/gpu.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [guide](<https://devfeed.tech/tags/guide.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [whisper](<https://devfeed.tech/tags/whisper.md>)

### AI overview

Hugging Face introduces an optimized Whisper inference endpoint powered by vLLM. It targets newer NVIDIA GPUs and combines PyTorch compilation, CUDA graphs, and float8 KV-cache quantization to improve transcription inference speed and memory efficiency.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## The New and Fresh analytics in Inference Endpoints

DevFeed: [The New and Fresh analytics in Inference Endpoints](<https://devfeed.tech/articles/the-new-and-fresh-analytics-in-inference-endpoints-7181.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/endpoint-analytics>)

Author: Erik Kaunismäki; Thibault Goehringer; Remy; Corentin Regal; Michelle Habonneau

Published: 2025-03-21T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [dashboards](<https://devfeed.tech/topics/dashboards.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [debugging](<https://devfeed.tech/topics/debugging.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [real-time](<https://devfeed.tech/topics/real-time.md>)

Tags: [analytics](<https://devfeed.tech/tags/analytics.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [debug](<https://devfeed.tech/tags/debug.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [guide](<https://devfeed.tech/tags/guide.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [latency](<https://devfeed.tech/tags/latency.md>), [lifecycle](<https://devfeed.tech/tags/lifecycle.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [state](<https://devfeed.tech/tags/state.md>)

### AI overview

Hugging Face has refreshed the analytics dashboard for Inference Endpoints with real-time metrics, faster data loading, customizable time ranges, auto-refresh, and a replica lifecycle view. The updates help users monitor performance, investigate issues, and understand replica state transitions.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Remote VAEs for decoding with Inference Endpoints 🤗

DevFeed: [Remote VAEs for decoding with Inference Endpoints 🤗](<https://devfeed.tech/articles/remote-vaes-for-decoding-with-inference-endpoints-7456.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/remote_vae>)

Author: hlky; Sayak Paul

Published: 2025-02-24T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [VAE](<https://devfeed.tech/topics/vae.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [diffusers](<https://devfeed.tech/topics/diffusers.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Deadlock](<https://devfeed.tech/topics/deadlock.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [data](<https://devfeed.tech/tags/data.md>), [diffusers](<https://devfeed.tech/tags/diffusers.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [generate](<https://devfeed.tech/tags/generate.md>), [generation](<https://devfeed.tech/tags/generation.md>), [getting-started](<https://devfeed.tech/tags/getting-started.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hdd](<https://devfeed.tech/tags/hdd.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [nvme](<https://devfeed.tech/tags/nvme.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [queue](<https://devfeed.tech/tags/queue.md>), [remote](<https://devfeed.tech/tags/remote.md>), [tensors](<https://devfeed.tech/tags/tensors.md>), [time](<https://devfeed.tech/tags/time.md>), [vae](<https://devfeed.tech/tags/vae.md>)

### AI overview

This article presents an experimental approach for decoding latent-space diffusion outputs with remote VAEs hosted on Inference Endpoints. It explains how remote decoding can reduce consumer GPU memory pressure, avoid the quality loss associated with tiled decoding, and improve concurrency by queueing generation requests. The article includes setup and usage examples for random tensors and pipelines involving SD v1.5, Flux, and HunyuanVideo.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## 1 Billion Classifications

DevFeed: [1 Billion Classifications](<https://devfeed.tech/articles/1-billion-classifications-7132.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/billion-classifications>)

Author: Derek Thomas

Published: 2025-02-13T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [k6](<https://devfeed.tech/topics/k6.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Grafana](<https://devfeed.tech/topics/grafana.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>), [Software Testing](<https://devfeed.tech/topics/software-testing.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [JavaScript](<https://devfeed.tech/topics/javascript.md>), [.env](<https://devfeed.tech/topics/dotenv.md>)

Tags: [classification](<https://devfeed.tech/tags/classification.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [grafana](<https://devfeed.tech/tags/grafana.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [k6](<https://devfeed.tech/tags/k6.md>), [latency](<https://devfeed.tech/tags/latency.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [performance](<https://devfeed.tech/tags/performance.md>), [python](<https://devfeed.tech/tags/python.md>), [scale](<https://devfeed.tech/tags/scale.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

This article presents a methodology for calculating cost and latency when running large-scale classification and embedding workloads. It examines model architectures, hardware options, deployment, load testing, and inference servers, with a focus on processing 1 billion inputs and balancing batch-inference cost against latency.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## How to deploy and fine-tune DeepSeek models on AWS

DevFeed: [How to deploy and fine-tune DeepSeek models on AWS](<https://devfeed.tech/articles/how-to-deploy-and-fine-tune-deepseek-models-on-aws-7161.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/deepseek-r1-aws>)

Author: Simon Pagezy; Jeff Boudier; David Corvoysier

Published: 2025-01-30T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [deepseek](<https://devfeed.tech/topics/deepseek.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Amazon Bedrock](<https://devfeed.tech/topics/amazon-bedrock.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [autoscaling](<https://devfeed.tech/topics/autoscaling.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [aws](<https://devfeed.tech/tags/aws.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [llms](<https://devfeed.tech/tags/llms.md>), [models](<https://devfeed.tech/tags/models.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>)

### AI overview

This tutorial explains how to deploy and fine-tune DeepSeek-R1 and its distilled models on AWS using Hugging Face Inference Endpoints. It covers production deployment with dedicated compute, autoscaling, scale-to-zero, security, optimized hardware, and deployment options through Amazon Bedrock and Amazon SageMaker AI.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Bevygap: Autoscaling multiplayer Bevy game servers with Edgegap & Lightyear

DevFeed: [Bevygap: Autoscaling multiplayer Bevy game servers with Edgegap & Lightyear](<https://devfeed.tech/articles/bevygap-autoscaling-multiplayer-bevy-game-servers-with-edgegap-lightyear-25816.md>)

Original publisher: [Read original article](<https://www.metabrew.com/article/bevygap-bevy-multiplayer-with-edgegap-and-lightyear>)

Author: {"image"=\>"rj-headshot.jpg", "greetings"=\>nil}

Published: 2024-12-09T06:00:00Z

Content type: article

Language: en

Sources: [Richard Jones](<https://devfeed.tech/sources/richard-jones.md>)

Topics: [Bevy](<https://devfeed.tech/topics/bevy.md>), [autoscaling](<https://devfeed.tech/topics/autoscaling.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Docker](<https://devfeed.tech/topics/docker.md>), [GitHub Actions](<https://devfeed.tech/topics/github-actions.md>), [API](<https://devfeed.tech/topics/api.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [WebAssembly](<https://devfeed.tech/topics/web-assembly.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [bevy](<https://devfeed.tech/tags/bevy.md>), [bevygap](<https://devfeed.tech/tags/bevygap.md>), [docker](<https://devfeed.tech/tags/docker.md>), [edgegap](<https://devfeed.tech/tags/edgegap.md>), [game-servers](<https://devfeed.tech/tags/game-servers.md>), [gamedev](<https://devfeed.tech/tags/gamedev.md>), [github-actions](<https://devfeed.tech/tags/github-actions.md>), [latency](<https://devfeed.tech/tags/latency.md>), [multiplayer](<https://devfeed.tech/tags/multiplayer.md>), [netcode](<https://devfeed.tech/tags/netcode.md>), [rust](<https://devfeed.tech/tags/rust.md>), [wasm](<https://devfeed.tech/tags/wasm.md>)

### AI overview

This article describes adapting the Lightyear spaceships example into a server-authoritative multiplayer Bevy game and deploying its client and headless server with Docker. It explains using Edgegap to deploy game servers in suitable locations on demand and autoscale them in response to player demand, with supporting infrastructure including GitHub Actions and NATS.

### Source excerpt

If i'm going to write a server-authoritative multiplayer game, it would be nice to have a decent way to deploy it. Especially to deploy servers around the world in response to player demand, to autoscale and keep latency down.

## Investing in Performance: Fine-tune small models with LLM insights - a CFM case study

DevFeed: [Investing in Performance: Fine-tune small models with LLM insights - a CFM case study](<https://devfeed.tech/articles/investing-in-performance-fine-tune-small-models-with-llm-insights-a-cfm-case-study-7139.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/cfm-case-study>)

Author: Oussama Ahouzi; champonnois; Jérémy L'Hour; Pirashanth Ratnamogan; Bérengère Patault; Morgane Goibert

Published: 2024-12-03T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [argilla](<https://devfeed.tech/topics/argilla.md>), [llama](<https://devfeed.tech/topics/llama.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [Finance](<https://devfeed.tech/topics/finance.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Meta](<https://devfeed.tech/topics/meta.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Scalability](<https://devfeed.tech/topics/scalability.md>)

Tags: [argilla](<https://devfeed.tech/tags/argilla.md>), [case-studies](<https://devfeed.tech/tags/case-studies.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [expert-support](<https://devfeed.tech/tags/expert-support.md>), [expert-support-program](<https://devfeed.tech/tags/expert-support-program.md>), [financial-applications](<https://devfeed.tech/tags/financial-applications.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [llm](<https://devfeed.tech/tags/llm.md>), [models](<https://devfeed.tech/tags/models.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [scalability](<https://devfeed.tech/tags/scalability.md>)

### AI overview

A case study of Capital Fund Management's use of open-source LLMs to improve financial named entity recognition. It covers LLM-assisted labeling, fine-tuning smaller models with curated datasets, and deployment on Hugging Face Inference Endpoints to balance accuracy, cost, and scalability.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Deploying Speech-to-Speech on Hugging Face

DevFeed: [Deploying Speech-to-Speech on Hugging Face](<https://devfeed.tech/articles/deploying-speech-to-speech-on-hugging-face-7462.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/s2s_endpoint>)

Author: Andres Marafioti; Derek Thomas; Diego Maniloff; Eustache Le Bihan

Published: 2024-10-22T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Language models](<https://devfeed.tech/topics/language-models.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [docker](<https://devfeed.tech/tags/docker.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [speech-to-speech](<https://devfeed.tech/tags/speech-to-speech.md>), [text-to-speech](<https://devfeed.tech/tags/text-to-speech.md>)

### AI overview

A step-by-step guide to packaging and deploying Hugging Face's Speech-to-Speech pipeline on Inference Endpoints using a custom Docker image.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Llama can now see and run on your device - welcome Llama 3.2

DevFeed: [Llama can now see and run on your device - welcome Llama 3.2](<https://devfeed.tech/articles/llama-can-now-see-and-run-on-your-device-welcome-llama-3-2-7337.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/llama32>)

Author: merve; Philipp Schmid; Omar Sanseviero; Vaibhav Srivastav; Lewis Tunstall; Aritra Roy Gosthipaty; Pedro Cuenca

Published: 2024-09-25T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [llama](<https://devfeed.tech/topics/llama.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>), [Meta](<https://devfeed.tech/topics/meta.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [llama](<https://devfeed.tech/tags/llama.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [meta](<https://devfeed.tech/tags/meta.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

Meta's Llama 3.2 release introduces multimodal Vision models in 11B and 90B sizes, smaller text-only 1B and 3B models for on-device use, and vision-enabled Llama Guard 3. The article describes their capabilities, architecture, supported languages, inference examples, and integrations with Hugging Face Transformers, TGI, Inference Endpoints, Google Cloud, Amazon SageMaker, and DELL Enterprise Hub.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Google Cloud TPUs made available to Hugging Face users

DevFeed: [Google Cloud TPUs made available to Hugging Face users](<https://devfeed.tech/articles/google-cloud-tpus-made-available-to-hugging-face-users-7524.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/tpu-inference-endpoints-spaces>)

Author: Simon Pagezy; Michelle Habonneau; Philipp Schmid; Alvaro Moran

Published: 2024-07-09T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [Sovereign AI](<https://devfeed.tech/topics/sovereign-ai.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [spaces](<https://devfeed.tech/topics/spaces.md>), [optimum](<https://devfeed.tech/topics/optimum.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [gemma](<https://devfeed.tech/topics/gemma.md>), [llama](<https://devfeed.tech/topics/llama.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>)

Tags: [gcp](<https://devfeed.tech/tags/gcp.md>), [gemma](<https://devfeed.tech/tags/gemma.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llama](<https://devfeed.tech/tags/llama.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [optimum](<https://devfeed.tech/tags/optimum.md>), [partnerships](<https://devfeed.tech/tags/partnerships.md>), [performance](<https://devfeed.tech/tags/performance.md>), [spaces](<https://devfeed.tech/tags/spaces.md>), [tgi](<https://devfeed.tech/tags/tgi.md>), [tpu](<https://devfeed.tech/tags/tpu.md>)

### AI overview

Hugging Face announces that Google Cloud TPUs are available for Inference Endpoints and Spaces. Google TPU v5e configurations can deploy supported models through managed infrastructure, while Optimum TPU and Text Generation Inference help train and serve models on TPUs.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Deploy models on AWS Inferentia2 from Hugging Face

DevFeed: [Deploy models on AWS Inferentia2 from Hugging Face](<https://devfeed.tech/articles/deploy-models-on-aws-inferentia2-from-hugging-face-7285.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/inferentia-inference-endpoints>)

Author: Jeff Boudier; Philipp Schmid

Published: 2024-05-22T00:00:00Z

Content type: release

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [inference-endpoints](<https://devfeed.tech/topics/inference-endpoints.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [tgi](<https://devfeed.tech/topics/tgi.md>), [autoscaling](<https://devfeed.tech/topics/autoscaling.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [MLOps](<https://devfeed.tech/topics/mlops.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [mlops](<https://devfeed.tech/tags/mlops.md>), [model](<https://devfeed.tech/tags/model.md>), [optimum](<https://devfeed.tech/tags/optimum.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [tgi](<https://devfeed.tech/tags/tgi.md>)

### AI overview

Hugging Face announces support for deploying models on AWS Inferentia2 through Amazon SageMaker and Hugging Face Inference Endpoints. The update enables scalable inference for supported models, including Meta Llama 3, with multiple instance sizes, managed features, and autoscaling.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

[Next page](<https://devfeed.tech/topics/inference-endpoints.md?cursor=WyIyMDI0LTA1LTIyVDAwOjAwOjAwKzAwOjAwIiwgImYyNWNmNzg5LTJhODAtNGY5NC1iOWIxLWE5Yzg2NmZlYjgzOCJd>)