# Gisting: Compressing LLM Agent context to ↑ throughput and ↓ cost

DevFeed: [Gisting: Compressing LLM Agent context to ↑ throughput and ↓ cost](<https://devfeed.tech/articles/gisting-compressing-llm-agent-context-to-throughput-and-cost-1403.md>)

Original publisher: [Read original article](<https://shopify.engineering/gisting>)

Author: Cody Mazza-Anthony

Published: 2026-08-19T14:32:58Z

Content type: article

Language: en

Sources: [Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering.md>), [Shopify Engineering - Shopify Engineering](<https://devfeed.tech/sources/shopify-engineering-shopify-engineering.md>)

Topics: [Compression](<https://devfeed.tech/topics/compression.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Inference Performance](<https://devfeed.tech/topics/inference-performance.md>), [Low-Latency Inference](<https://devfeed.tech/topics/low-latency-inference.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [GraphQL](<https://devfeed.tech/topics/graphql.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>)

Tags: [compression](<https://devfeed.tech/tags/compression.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm](<https://devfeed.tech/tags/llm.md>), [model](<https://devfeed.tech/tags/model.md>), [tokens](<https://devfeed.tech/tags/tokens.md>)

## AI overview

Gisting compresses an LLM agent's system prompt into learned gist tokens, preserving prediction quality while reducing inference latency, increasing throughput, and lowering GPU requirements.

## Source excerpt

Gisting compresses context into a set of learned tokens, preserving its quality while making the model faster and cheaper.