# Lightbits Inferra KV Cache Engine Claims 16x Session Density and 10M-Token Contexts

DevFeed: [Lightbits Inferra KV Cache Engine Claims 16x Session Density and 10M-Token Contexts](<https://devfeed.tech/articles/lightbits-inferra-kv-cache-engine-claims-16x-session-density-and-10m-token-contexts-17436.md>)

Original publisher: [Read original article](<https://www.storagereview.com/news/lightbits-inferra-kv-cache-engine-claims-16x-session-density-and-10m-token-contexts>)

Author: Harold Fritts

Published: 2026-09-14T16:23:21Z

Content type: news

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [Inference Performance](<https://devfeed.tech/topics/inference-performance.md>), [Orchestration](<https://devfeed.tech/topics/orchestration.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Multi-tenancy](<https://devfeed.tech/topics/multi-tenancy.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [Security](<https://devfeed.tech/topics/security.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [cache](<https://devfeed.tech/tags/cache.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [nvme](<https://devfeed.tech/tags/nvme.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [security](<https://devfeed.tech/tags/security.md>)

## AI overview

Lightbits Labs is introducing Inferra, a KV cache orchestration engine for AI inference. It virtualizes GPU memory across DRAM and NVMe storage, preserving attention states for long-context and multi-session workloads. Lightbits claims up to 16 times more concurrent sessions, more than 100 times lower latency than recomputation, and context windows of up to 10 million tokens. Inferra supports vLLM, TensorRT, and SGLang and includes tiering, predictive prefetching, tenant isolation, and encrypted data transfer.

## Source excerpt

Lightbits Labs, the company that invented NVMe over TCP, is moving into inference software with Inferra, a KV cache orchestration engine that makes its public debut tomorrow, September 15, at the AI Infra Summit in Santa Clara. The software virtualizes GPU memory across DRAM and NVMe storage tiers and turns the KV cache into a The post Lightbits Inferra KV Cache Engine Claims 16x Session Density and 10M-Token Contexts appeared first on StorageReview.com.