# self-hosted-llms

Published articles for self-hosted-llms.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Announcing distributed KV caching for self-hosted LLMs with Amazon ElastiCache

DevFeed: [Announcing distributed KV caching for self-hosted LLMs with Amazon ElastiCache](<https://devfeed.tech/articles/announcing-distributed-kv-caching-for-self-hosted-llms-with-amazon-elasticache-142000.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/database/announcing-distributed-kv-caching-for-self-hosted-llms-with-amazon-elasticache/>)

Author: Meet Bhagdev

Published: 2026-11-18T23:45:25Z

Content type: release

Language: en

Sources: [AWS Database Blog](<https://devfeed.tech/sources/aws-database-blog.md>)

Topics: [Amazon ElastiCache](<https://devfeed.tech/topics/amazon-elasticache.md>), [valkey](<https://devfeed.tech/topics/valkey.md>), [Self Hosted AI Tools](<https://devfeed.tech/topics/self-hosted-ai-tools.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>), [TensorRT-LLM](<https://devfeed.tech/topics/tensorrt-llm.md>)

Tags: [amazon-elasticache](<https://devfeed.tech/tags/amazon-elasticache.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [benefits](<https://devfeed.tech/tags/benefits.md>), [caching](<https://devfeed.tech/tags/caching.md>), [costs](<https://devfeed.tech/tags/costs.md>), [distributed-kv](<https://devfeed.tech/tags/distributed-kv.md>), [expert-400](<https://devfeed.tech/tags/expert-400.md>), [general-availability](<https://devfeed.tech/tags/general-availability.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-costs](<https://devfeed.tech/tags/inference-costs.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [llm-inference-costs](<https://devfeed.tech/tags/llm-inference-costs.md>), [llms](<https://devfeed.tech/tags/llms.md>), [self-hosted](<https://devfeed.tech/tags/self-hosted.md>), [self-hosted-llms](<https://devfeed.tech/tags/self-hosted-llms.md>), [valkey](<https://devfeed.tech/tags/valkey.md>)

### AI overview

Distributed KV caching with Amazon ElastiCache for Valkey adds a shared L2 tier across a GPU fleet, letting vLLM, SGLang, and TensorRT-LLM reuse prompt KV state when local GPU or host caches miss. An open-source LMCache connector requires a configuration change; cache.i8ge.48xlarge nodes use DRAM and NVMe tiering and RDMA over EFA. In tested workloads, gains depended on cache hits and prompt length: shared-document Q&A doubled requests per second and cut cost per million output tokens by 40% while meeting the stated interactivity targets; coding-agent and assistant throughput also increased. Reported TTFT reductions ranged from 55% to 93% across tested models. The post says the least benefit is expected for small deployments with KV state fitting in GPU memory, workloads without prompt reuse, and long outputs from short prompts. It recommends using the free KVTrace tool to estimate hit rate, savings, and cache size before deployment; estimates use hashed prompt data.

### Source excerpt

Today, we are announcing the general availability of distributed KV caching with Amazon ElastiCache for Valkey, a capability that cuts self-hosted LLM inference costs by up to 40% and time to first token by up to 80%. Learn how it works, review benchmark results across workloads and models, deploy it on your own GPU fleet, and size the savings with the open-source KVTrace tool.

## Self-host a local AI stack and access it from anywhere

DevFeed: [Self-host a local AI stack and access it from anywhere](<https://devfeed.tech/articles/self-host-a-local-ai-stack-and-access-it-from-anywhere-86939.md>)

Original publisher: [Read original article](<https://hackernoon.com/self-host-a-local-ai-stack-and-access-it-from-anywhere?source=rss>)

Author: ameliepangolin

Published: 2026-10-14T08:00:06Z

Content type: tutorial

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Self Hosted AI Tools](<https://devfeed.tech/topics/self-hosted-ai-tools.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [open webui](<https://devfeed.tech/topics/open-webui.md>), [NixOS](<https://devfeed.tech/topics/nixos.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [tailscale](<https://devfeed.tech/topics/tailscale.md>), [internal developer platform automation](<https://devfeed.tech/topics/internal-developer-platform-automation.md>), [Security](<https://devfeed.tech/topics/security.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [automation](<https://devfeed.tech/tags/automation.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [docker](<https://devfeed.tech/tags/docker.md>), [good-company](<https://devfeed.tech/tags/good-company.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [local-ai-lab-setup](<https://devfeed.tech/tags/local-ai-lab-setup.md>), [nixos](<https://devfeed.tech/tags/nixos.md>), [nixos-ai-configuration](<https://devfeed.tech/tags/nixos-ai-configuration.md>), [offline-ai-server](<https://devfeed.tech/tags/offline-ai-server.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [ollama-and-open-webui](<https://devfeed.tech/tags/ollama-and-open-webui.md>), [open-webui](<https://devfeed.tech/tags/open-webui.md>), [private-ai](<https://devfeed.tech/tags/private-ai.md>), [proxmox](<https://devfeed.tech/tags/proxmox.md>), [proxmox-gpu-passthrough](<https://devfeed.tech/tags/proxmox-gpu-passthrough.md>), [self-host](<https://devfeed.tech/tags/self-host.md>), [self-hosted-llms](<https://devfeed.tech/tags/self-hosted-llms.md>), [tailscale](<https://devfeed.tech/tags/tailscale.md>), [tailscale-remote-ai-access](<https://devfeed.tech/tags/tailscale-remote-ai-access.md>)

### AI overview

This video tutorial walks through building a private, self-hosted local AI lab using Proxmox, NixOS, Docker, an NVIDIA GPU, Ollama, and Open WebUI. It covers secure remote access through Tailscale and deployment automation with Ansible.

### Source excerpt

Build a private AI lab with Proxmox, NixOS, an NVIDIA GPU, Ollama, and Open WebUI, with automated deployment and secure remote access through Tailscale.