# llmops Published articles for llmops. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## KV Cache Engineering for LLM Serving DevFeed: [KV Cache Engineering for LLM Serving]() Original publisher: [Read original article]() Author: Avi Chawla Published: 2026-09-06T20:43:32Z Content type: tutorial Language: en Sources: [Daily Dose of Data Science]() Topics: [Caching](), [Large Language Model](), [LLM Techniques](), [GPU](), [Inference](), [llama](), [quantization]() Tags: [cache](), [engineering](), [gpu](), [inference](), [llama](), [llm](), [llmops](), [quantization]() ### AI overview This article explains how KV cache growth affects GPU memory and attention costs during LLM serving. It organizes twelve optimization techniques by whether they reduce heads, layers, retained tokens, representation width, precision, state size, allocation waste, or duplicate memory, while noting the trade-offs and limits of each approach. ### Source excerpt 12 techniques to manage KV cache in production. ## Instrumenting AI Agents: Why the Apology Metric Is a First Class Reliability Signal DevFeed: [Instrumenting AI Agents: Why the Apology Metric Is a First Class Reliability Signal]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-03-21T00:00:00Z Content type: article Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Artificial Intelligence](), [AI Agent](), [reliability](), [agent observability](), [context]() Tags: [ai](), [ai-agents](), [api](), [backend](), [context](), [dependency](), [distributed-systems](), [inference](), [json](), [kv-cache](), [latency](), [llm](), [llmops](), [metric](), [observability](), [orchestration](), [rag](), [reliability](), [reliability-engineering](), [signal](), [timeout]() ### AI overview This article argues that AI agents can return HTTP 200 responses while still failing business tasks. It proposes tracking apology responses as a reliability signal to expose context starvation caused by stalled dependencies, timeouts, stale synchronization, dropped data, or payload truncation in retrieval and orchestration pipelines. ### Source excerpt An AI agent that returns "I apologize, but I cannot find that order" ships an HTTP 200. Infrastructure dashboards call that healthy; the business calls it a failure. Tracking apology phrases as a first-class SLO surfaces context starvation, a stalled dependency, a timeout guard tripping, or silent payload truncation, none of which HTTP status codes and CPU graphs can see. ## From siloed DataOps, MLOps, and LLMOps to a unified data-intelligence platform DevFeed: [From siloed DataOps, MLOps, and LLMOps to a unified data-intelligence platform]() Original publisher: [Read original article]() Author: Rajit Saha Published: 2025-08-04T18:03:19Z Content type: opinion Language: en Sources: [Udemy Engineering]() Topics: [DataOps](), [MLOps](), [Generative AI](), [Apache Spark](), [Amazon Bedrock](), [Amazon S3](), [Amazon Redshift](), [Amazon SageMaker](), [apache-flink]() Tags: [ai](), [aiops](), [amazon-s3](), [analytics](), [apache-flink](), [apache-spark](), [bedrock](), [dataops](), [large-language-models](), [llmops](), [mlops]() ### AI overview The article describes how DataOps, MLOps, and AI/LLM Ops commonly rely on separate systems and teams for data processing, model deployment, inference, evaluation, orchestration, governance, and monitoring. It then introduces Databricks' Data Intelligence Platform as a unified environment intended to bring these domains together. ### Source excerpt Introduction In modern data-driven businesses, the pace of innovation in analytics and artificial intelligence has outstripped the capacity of many teams. Three distinct disciplines emerged to handle this expansion: Data platform (DataOps) teams built data lakes on cloud storage such as Amazon S3, processed them with Apache Spark and Hive on EMR, ingested streaming data with Spark Structured Streaming or Apache Flink, and loaded tabular copies into MPP warehouses like Redshift for interactive SQL and BI. Cataloguing and governance were offloaded to external tools such as DataHub, and fine-grained access controls required third-party services like Privacera. This architecture worked, but it required separate workflows for batch and streaming, extra systems for lineage and governance, and a mosaic of operational teams. MLOps teams provided an additional layer. Data scientists used notebook environments (for example, Amazon SageMaker) to preprocess data, train, and evaluate models. Deploying models meant writing integration code to move features into a serving layer, to register models in disparate registries and to build custom APIs for inference. Feature stores and model registries were bought from additional vendors. Updates and monitoring were often manual processes. AI/LLM Ops teams are a new addition because generative AI requires specialized components: LLM gateways (e.g., Amazon Bedrock) to proxy access to foundation models; evaluation tooling to compare large language models; orchestration frameworks for agents; vector databases for retrieval augmented generation; and of course another layer of security, access management and cost control. These tools seldom integrate seamlessly with existing data and ML pipelines. This fragmented state makes it difficult to react quickly when product requirements change. Each new capability requires another system, another integration, and another team. Meanwhile, budgets tighten and go-to-market timelines shrink. The questio