# pull through cache

A pull-through cache retrieves images from an upstream container registry, stores them locally, and serves subsequent requests from the cache.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Why Kubernetes Cannot Contain a Full Lazy-Pulling Cache: Two Impossibility Results and a Bounded, Honest Design for Model Serving

DevFeed: [Why Kubernetes Cannot Contain a Full Lazy-Pulling Cache: Two Impossibility Results and a Bounded, Honest Design for Model Serving](<https://devfeed.tech/articles/why-kubernetes-cannot-contain-a-full-lazy-pulling-cache-two-impossibility-results-and-a-bounded-honest-design-for-model-serving-126731.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11739>)

Author: Georgii Kliukovkin

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>), [declarative model serving](<https://devfeed.tech/topics/declarative-model-serving.md>), [KServe](<https://devfeed.tech/topics/kserve.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [accounting](<https://devfeed.tech/tags/accounting.md>), [actuator](<https://devfeed.tech/tags/actuator.md>), [all](<https://devfeed.tech/tags/all.md>), [article](<https://devfeed.tech/tags/article.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [audit](<https://devfeed.tech/tags/audit.md>), [aws](<https://devfeed.tech/tags/aws.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [cache](<https://devfeed.tech/tags/cache.md>), [contain](<https://devfeed.tech/tags/contain.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-os](<https://devfeed.tech/tags/cs-os.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [design](<https://devfeed.tech/tags/design.md>), [full](<https://devfeed.tech/tags/full.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [lazy](<https://devfeed.tech/tags/lazy.md>), [model](<https://devfeed.tech/tags/model.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [results](<https://devfeed.tech/tags/results.md>), [serving](<https://devfeed.tech/tags/serving.md>)

### AI overview

Lazy image pulling can leave model bytes in a node cache that Kubernetes neither accounts for nor bounds. In the reported experiments, a readiness sentinel stayed green while 272 of 280 files were unreadable; kubelet freed no cache space and evicted 26 bystander pods. The proposed fix operates in the cache owner without changing kubelet or the container runtime interface: enforce a byte budget using filesystem pressure as the signal, and let reads succeed even when cache writes fail. The budget held despite about 301,000 dropped accounting updates. Across six AWS campaigns at 90% cache pre-fill, upstream stargz-snapshotter failed on 273 of 280 model files, while the proposed system read 150.3 GB without errors at an 18.5% higher completion cost and maintained its budget during live reads, co-tenancy, and a three-node fleet. A source audit found the missing bound also in SOCI and Nydus.

### Source excerpt

arXiv:2611.11739v1 Announce Type: new Abstract: Lazy container-image pulling makes cold starts of autoscaled model-serving fleets on Kubernetes nearly independent of model size, but it moves the model's bytes into a node-level cache that the platform neither bounds nor accounts for. This article establishes, by experiment, two impossibility results for that cache. A monitoring-side remedy cannot catch its failure: a sentinel readiness probe stays green while 272 of 280 files are unreadable. A platform-side remedy cannot contain it: kubelet, given full disk-pressure visibility, frees 0 bytes and evicts 26 bystander pods, because its reclamation actuator is keyed to an accounting that does not know the cache exists. The article then eliminates the failure class inside the component that owns the bytes, with no change to kubelet or the container runtime interface, by two design rules: a byte budget whose pressure signal is keyed to filesystem truth, so that a never-blocking, lossy accounting still enforces a hard bound (held against about 301,000 dropped updates), and pass-through semantics under which a failed cache write never becomes a failed read, with the coupling traced to a single io.MultiWriter. Across six pre-specified campaigns on AWS, upstream stargz-snapshotter fails 273 of 280 model files at 90% cache pre-fill; the proposed system reads all 150.3 GB with zero errors at +18.5% completion cost and holds the budget under a live read, co-tenancy, and a three-node fleet. A source audit shows the missing bound generalizes to SOCI and Nydus. The implementation is open source and has been proposed upstream.

## Cached list queries now use up to 69% less memory and serve up to 29% more requests

DevFeed: [Cached list queries now use up to 69% less memory and serve up to 29% more requests](<https://devfeed.tech/articles/cached-list-queries-now-use-up-to-69-less-memory-and-serve-up-to-29-more-requests-66881.md>)

Original publisher: [Read original article](<https://appwrite.io/blog/post/faster-list-caching-binary-format>)

Author: Luke B. Silver

Published: 2026-10-07T00:00:00Z

Content type: release

Language: en

Sources: [Appwrite Blog](<https://devfeed.tech/sources/appwrite-blog.md>)

Topics: [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>), [Schema-less data format](<https://devfeed.tech/topics/schema-less-data-format.md>)

Tags: [announcements](<https://devfeed.tech/tags/announcements.md>), [cache](<https://devfeed.tech/tags/cache.md>), [json](<https://devfeed.tech/tags/json.md>), [key-value-cache](<https://devfeed.tech/tags/key-value-cache.md>), [latency](<https://devfeed.tech/tags/latency.md>), [saving](<https://devfeed.tech/tags/saving.md>)

### AI overview

Appwrite has replaced JSON with a binary format for key-value cache entries, reducing cached list-query memory use by up to 69% and increasing throughput by up to 29% in benchmarks. The format also speeds cache reads, with measured latency improvements, while response conversion to JSON still limits throughput gains for large pages. It ships in self-hosted Appwrite 2.3; upgrades require no migration or cache flush.

### Source excerpt

Appwrite now serializes key-value cache entries in a binary format instead of JSON. Cached list responses consume up to 69% less memory, and cached list throughput increases by up to 29%.

## JFrog Artifactory Supports LuaRocks Hosting for NGINX, OpenResty and Kong

DevFeed: [JFrog Artifactory Supports LuaRocks Hosting for NGINX, OpenResty and Kong](<https://devfeed.tech/articles/jfrog-artifactory-supports-luarocks-hosting-for-nginx-openresty-and-kong-80876.md>)

Original publisher: [Read original article](<https://jfrog.com/blog/jfrog-artifactory-luarocks-support-nginx-openresty-kong/>)

Author: Dean Lalezar

Published: 2026-09-16T13:15:32Z

Content type: article

Language: en

Sources: [From the Frog's mouth - JFrog Blog](<https://devfeed.tech/sources/from-the-frog-s-mouth-jfrog-blog.md>)

Topics: [jfrog artifactory](<https://devfeed.tech/topics/jfrog-artifactory.md>), [nginx](<https://devfeed.tech/topics/nginx.md>), [ingress-nginx](<https://devfeed.tech/topics/ingress-nginx.md>), [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>)

Tags: [artifactory](<https://devfeed.tech/tags/artifactory.md>), [devops](<https://devfeed.tech/tags/devops.md>), [kong-gateway](<https://devfeed.tech/tags/kong-gateway.md>), [lua-development](<https://devfeed.tech/tags/lua-development.md>), [luarocks](<https://devfeed.tech/tags/luarocks.md>), [nginx](<https://devfeed.tech/tags/nginx.md>), [openresty](<https://devfeed.tech/tags/openresty.md>), [private-registry](<https://devfeed.tech/tags/private-registry.md>), [software-supply-chain](<https://devfeed.tech/tags/software-supply-chain.md>)

### AI overview

JFrog Artifactory supports LuaRocks repositories, allowing teams to host private Lua packages and cache public rocks through local, remote, and virtual repositories. The article explains how this can improve availability and consistency for Lua modules used by NGINX/OpenResty and custom Kong plugins, including publishing versioned plugins through CI.

### Source excerpt

If your NGINX/OpenResty servers or Kong gateways pull Lua modules straight from public luarocks.org, one upstream outage can stall every build and deploy that depends on a "rock". With LuaRocks support in JFrog Artifactory, you host and proxy those modules from a private LuaRocks registry you control, using the native experience you expect, not a ...

## Tiered KV Cache Offloading in vLLM

DevFeed: [Tiered KV Cache Offloading in vLLM](<https://devfeed.tech/articles/tiered-kv-cache-offloading-in-vllm-79484.md>)

Original publisher: [Read original article](<https://vllm.ai/blog/2026-09-10-tiered-kv-offloading>)

Author: Or Ozeri, Danny Harnik, Ronen Schaffer, Itay Etelis, Varun Sundar Rabindranath

Published: 2026-09-10T00:00:00Z

Content type: article

Language: en

Sources: [vLLM Blog](<https://devfeed.tech/sources/vllm-blog.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>)

Tags: [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [host-memory](<https://devfeed.tech/tags/host-memory.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [offloading](<https://devfeed.tech/tags/offloading.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [vllm-blog](<https://devfeed.tech/tags/vllm-blog.md>)

### AI overview

vLLM's tiered KV cache offloading framework preserves evicted data across host memory, filesystems, object storage, and remote peers. By reloading cached data instead of recomputing it, the approach can reduce latency and increase serving capacity, while sharing KV data across instances and nodes. The article explains the host-centric design, offload and reload paths, secondary storage tiers, observability, and benchmark results.

### Source excerpt

A host-centric framework for scaling KV cache across host memory, filesystems, object stores, and remote peers -- reducing recomputation and increasing serving capacity.

## The Best Container Registries in 2026

DevFeed: [The Best Container Registries in 2026](<https://devfeed.tech/articles/the-best-container-registries-in-2026-95501.md>)

Original publisher: [Read original article](<https://blog.railway.com/p/best-container-registries-2026>)

Author: Angelo Saraceno

Published: 2026-05-26T00:00:00Z

Content type: article

Language: en

Sources: [Railway](<https://devfeed.tech/sources/railway.md>)

Topics: [container-security](<https://devfeed.tech/topics/container-security.md>), [container images](<https://devfeed.tech/topics/container-images.md>), [Docker Hub](<https://devfeed.tech/topics/docker-hub.md>), [image retention policy](<https://devfeed.tech/topics/image-retention-policy.md>), [GitHub Container Registry](<https://devfeed.tech/topics/github-container-registry.md>), [container image compliance](<https://devfeed.tech/topics/container-image-compliance.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>), [trivy](<https://devfeed.tech/topics/trivy.md>), [Container Image Building](<https://devfeed.tech/topics/container-image-building.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [best](<https://devfeed.tech/tags/best.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [container](<https://devfeed.tech/tags/container.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [platforms](<https://devfeed.tech/tags/platforms.md>), [railway](<https://devfeed.tech/tags/railway.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [registries](<https://devfeed.tech/tags/registries.md>), [registry](<https://devfeed.tech/tags/registry.md>)

### AI overview

The article argues that registry choice affects deploy latency, image security and compliance, access control, retention costs, and the operational burden of keeping deploy pipelines working. It compares ten platforms by fit and trade-offs: Railway keeps images within its deploy platform but cannot serve as a general-purpose registry; Docker Hub suits public images but imposes pull limits; GitHub and GitLab registries align with their CI and auth systems, with scanning or storage limitations; ECR, Artifact Registry, and ACR fit workloads in their respective clouds, with costs and scanning features that vary; Quay emphasizes scanning, while Cloudflare targets edge pulls but has a younger tooling ecosystem; Harbor supports self-hosting and air-gapped use but requires ongoing operations. The decision criteria are compute location, scanning needs, auth, image visibility, retention, and outage responsibility.

### Source excerpt

Container registries are the under-thought-about piece of cloud infrastructure. The six jobs a real registry has to do, and an honest ranking of the platforms in 2026.

## pyx: a Python-native package registry, now in Beta

DevFeed: [pyx: a Python-native package registry, now in Beta](<https://devfeed.tech/articles/pyx-a-python-native-package-registry-now-in-beta-82883.md>)

Original publisher: [Read original article](<https://astral.sh/blog/introducing-pyx>)

Author: charlie@astral.sh (Charlie Marsh)

Published: 2025-08-13T00:00:00Z

Content type: release

Language: en

Sources: [Astral](<https://devfeed.tech/sources/astral.md>)

Topics: [Python](<https://devfeed.tech/topics/python.md>), [pip](<https://devfeed.tech/topics/pip.md>), [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>)

Tags: [developer-tools](<https://devfeed.tech/tags/developer-tools.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [python](<https://devfeed.tech/tags/python.md>), [registry](<https://devfeed.tech/tags/registry.md>), [uv](<https://devfeed.tech/tags/uv.md>)

### AI overview

Astral introduces pyx, a Python-native package registry in beta and the first part of its planned platform. It is designed to work with uv as an optimized backend, supporting private packages and serving as a configurable frontend to public sources such as PyPI and the PyTorch index. Astral says pyx aims to improve installation speed, security, GPU-related workflows, and package reuse, and is working with early partners as it prepares for general availability.

### Source excerpt

pyx is a Python-native package registry from the creators of uv.

## How to use multiple Docker registry mirrors

DevFeed: [How to use multiple Docker registry mirrors](<https://devfeed.tech/articles/how-to-use-multiple-docker-registry-mirrors-26642.md>)

Original publisher: [Read original article](<https://blog.alexellis.io/how-to-configure-multiple-docker-registry-mirrors/>)

Author: Alex Ellis

Published: 2023-06-08T13:27:06Z

Content type: tutorial

Language: en

Sources: [Alex Ellis' Blog](<https://devfeed.tech/sources/alex-ellis-blog.md>)

Topics: [Docker](<https://devfeed.tech/topics/docker.md>), [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>), [Self-hosted](<https://devfeed.tech/topics/self-hosted.md>), [GitHub Actions](<https://devfeed.tech/topics/github-actions.md>), [Firecracker](<https://devfeed.tech/topics/firecracker.md>)

Tags: [buildx](<https://devfeed.tech/tags/buildx.md>), [cache](<https://devfeed.tech/tags/cache.md>), [container](<https://devfeed.tech/tags/container.md>), [container-registry](<https://devfeed.tech/tags/container-registry.md>), [docker](<https://devfeed.tech/tags/docker.md>), [docker-registry](<https://devfeed.tech/tags/docker-registry.md>), [firecracker](<https://devfeed.tech/tags/firecracker.md>), [github-actions](<https://devfeed.tech/tags/github-actions.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [linux](<https://devfeed.tech/tags/linux.md>), [mirror](<https://devfeed.tech/tags/mirror.md>), [registries](<https://devfeed.tech/tags/registries.md>), [registry](<https://devfeed.tech/tags/registry.md>), [self-hosted](<https://devfeed.tech/tags/self-hosted.md>)

### AI overview

This tutorial explains why self-hosted GitHub Actions runners can hit Docker Hub's anonymous image pull limits and how a Docker registry configured as a pull-through cache can help. It covers configuring Docker and buildx to use multiple registry mirrors, including considerations for local HTTP and Internet-facing HTTPS setups.

### Source excerpt

Why would you need to use a mirror for a container registry? And is there a way to use two or more? Find out from the actuated team.

## Introducing the GHES repository cache

DevFeed: [Introducing the GHES repository cache](<https://devfeed.tech/articles/introducing-the-ghes-repository-cache-68378.md>)

Original publisher: [Read original article](<https://github.blog/news-insights/product-news/introducing-the-ghes-repository-cache/>)

Author: Matt Cooper

Published: 2022-03-22T17:48:06Z

Content type: release

Language: en

Sources: [GitHub Blog](<https://devfeed.tech/sources/github-engineering.md>)

Topics: [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>), [Service fabric](<https://devfeed.tech/topics/service-fabric.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Internet Traffic](<https://devfeed.tech/topics/internet-traffic.md>), [monorepo](<https://devfeed.tech/topics/monorepo.md>)

Tags: [ci](<https://devfeed.tech/tags/ci.md>), [continuous-integration](<https://devfeed.tech/tags/continuous-integration.md>), [data](<https://devfeed.tech/tags/data.md>), [data-residency](<https://devfeed.tech/tags/data-residency.md>), [delivery](<https://devfeed.tech/tags/delivery.md>), [developer-workflows](<https://devfeed.tech/tags/developer-workflows.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [ghes](<https://devfeed.tech/tags/ghes.md>), [github](<https://devfeed.tech/tags/github.md>), [github-enterprise-server](<https://devfeed.tech/tags/github-enterprise-server.md>), [latency](<https://devfeed.tech/tags/latency.md>), [monorepo](<https://devfeed.tech/tags/monorepo.md>), [news-insights](<https://devfeed.tech/tags/news-insights.md>), [product](<https://devfeed.tech/tags/product.md>), [repository](<https://devfeed.tech/tags/repository.md>), [traffic](<https://devfeed.tech/tags/traffic.md>)

### AI overview

GitHub Enterprise Server repository cache, introduced in public beta, is an eventually consistent read-only replica that selectively copies Git data to improve locality for CI farms. Writes remain on the primary server, and new data may take several minutes to appear. The article also notes limitations involving Git LFS and replication policies.

### Source excerpt

If you're a GHES customer with heavy read traffic on your monorepo, check out the repository cache, especially if you have CI workloads distributed around the world.

## Docker Hub pull rate limits hit Rahti

DevFeed: [Docker Hub pull rate limits hit Rahti](<https://devfeed.tech/articles/docker-hub-pull-rate-limits-hit-rahti-19775.md>)

Original publisher: [Read original article](<https://cloud.blog.csc.fi/2021/03/docker-hub-pull-rate-limits-hit-rahti.html>)

Author: Unknown (noreply@blogger.com)

Published: 2021-03-04T08:59:00Z

Content type: news

Language: en

Sources: [CSC - IT Center For Science - Cloud Team](<https://devfeed.tech/sources/csc-it-center-for-science-cloud-team.md>)

Topics: [Docker Hub](<https://devfeed.tech/topics/docker-hub.md>), [rate-limiting](<https://devfeed.tech/topics/rate-limiting.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [pull through cache](<https://devfeed.tech/topics/pull-through-cache.md>)

Tags: [2020](<https://devfeed.tech/tags/2020.md>), [backend](<https://devfeed.tech/tags/backend.md>), [builds](<https://devfeed.tech/tags/builds.md>), [cache](<https://devfeed.tech/tags/cache.md>), [centos](<https://devfeed.tech/tags/centos.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [container](<https://devfeed.tech/tags/container.md>), [customer](<https://devfeed.tech/tags/customer.md>), [dependencies](<https://devfeed.tech/tags/dependencies.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [docker](<https://devfeed.tech/tags/docker.md>), [docker-container](<https://devfeed.tech/tags/docker-container.md>), [docker-hub](<https://devfeed.tech/tags/docker-hub.md>), [downtime](<https://devfeed.tech/tags/downtime.md>), [effect](<https://devfeed.tech/tags/effect.md>), [error](<https://devfeed.tech/tags/error.md>), [first](<https://devfeed.tech/tags/first.md>), [good](<https://devfeed.tech/tags/good.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [limits](<https://devfeed.tech/tags/limits.md>)

### AI overview

Docker Hub's November 2020 pull limits caused some Rahti builds to fail with a "toomanyrequests" error. Rahti admins first advised users to move images to repositories without rate limits; some could, but it was not feasible for everyone. They also shared instructions for manually caching public Docker Hub images in Rahti's registry. That workaround was complicated by a concurrent storage incident affecting Rahti's own registry, leaving users at times affected by both services' availability problems. The excerpt says resolving the issue took longer than anticipated and ends before describing the full outcome.

### Source excerpt

November 2020 was an eventful month for Rahti. Our backend storage had incident that affected our production clusters image repository for many weeks. That event is well described in Allas downtime November 2020 - technical deep-dive. Just before our storage incident, 2nd of November, Docker applied "rate limiting for Docker container pulls for some users". When we tried to prepare for this in advance, it was very hard to figure out what the exact effect would be for Rahti. It took longer to resolve than we anticipated. Limits beganWe were relieved that the first day after pull rates were enforced was peaceful. Rahti was still up and running as it should. Users didn't complain about failing deployment in large amounts. Admins survived another upstream change. Eventually we received few tickets on failing builds. Our first approach was to instruct people to change their images away from Docker Hub to alternative repositories which do not have rate limiting enabled. Some users were able to migrate their images but this was not feasible solution for everyone. Still, it was a good opportunity to remind users to check what images they use. The error looks like this: Pulling image "docker.io/centos/python-38-centos7@sha256:da83741689a8d7fe1548fefe7e001c45bcc56a08bc03fd3b29a5636163ca0353" ... pulling image error : toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Cumulative issues cause exponential griefWe gave our users instructions how to move their dependencies from public Docker Hub to Rahti image registry, How to manually cache images in Rahti's registry. This was harder in practice, as we did have issues with our own registry due to our storage incident. There were days when users suffered from both Docker Hub and Rahti internal registry availability. Those days were hard for our Rahti admins, who struggled to meet customer expectations of highly available ser