# Reduce inference cold starts on Amazon SageMaker HyperPod with model caching

DevFeed: [Reduce inference cold starts on Amazon SageMaker HyperPod with model caching](<https://devfeed.tech/articles/reduce-inference-cold-starts-on-amazon-sagemaker-hyperpod-with-model-caching-4739.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/reduce-inference-cold-starts-on-amazon-sagemaker-hyperpod-with-model-caching/>)

Author: Kareem Syed-Mohammed

Published: 2026-09-10T21:37:49Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Caching](<https://devfeed.tech/topics/caching.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [autoscaling](<https://devfeed.tech/topics/autoscaling.md>), [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>)

Tags: [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-hyperpod](<https://devfeed.tech/tags/amazon-sagemaker-hyperpod.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [caching](<https://devfeed.tech/tags/caching.md>), [expert-400](<https://devfeed.tech/tags/expert-400.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [llm](<https://devfeed.tech/tags/llm.md>)

## AI overview

Amazon SageMaker HyperPod model caching preloads model weights and container images onto cluster nodes, reducing inference-pod cold starts by serving assets from local NVMe storage.

## Source excerpt

Amazon SageMaker HyperPod now supports model caching for inference, which pre-loads model weights and container images onto cluster nodes so pods read from local NVMe storage instead of downloading over the network. Learn how model caching cuts cold starts from tens of minutes to seconds, how it works, and how to enable it.