# Latency A computing performance measure of delay between a request and its response, including delay on network connections. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Inline CSS at Khan Academy: Aphrodite DevFeed: [Inline CSS at Khan Academy: Aphrodite]() Original publisher: [Read original article]() Author: Khan Academy Published: 2016-03-29T22:00:00Z Content type: article Language: en Sources: [Khan Academy]() Topics: [React](), [Server-side rendering](), [HTML](), [JavaScript](), [Latency](), [networking](), [Chrome]() Tags: [chrome](), [css](), [engineering](), [infrastructure](), [javascript](), [latency](), [network](), [news](), [react](), [server-side-rendering]() ### AI overview Jamie Wong discusses Aphrodite, Khan Academy's inline styling solution, the constraints that shaped it, and its use in production. The article explains how reducing bytes before content renders motivated server-side rendering and compares the initial-load byte requirements of server-side and client-side rendering. ### Source excerpt By Jamie Wong I won't ramble in this post about the maintenance woe that is CSS, as others ... Read more ## Profiling App Engine Memcached DevFeed: [Profiling App Engine Memcached]() Original publisher: [Read original article]() Author: Khan Academy Published: 2017-05-01T22:00:00Z Content type: article Language: en Sources: [Khan Academy]() Topics: [Latency](), [Logging](), [Data structures]() Tags: [engineering](), [infrastructure](), [latency](), [logging](), [memcached](), [news](), [patterns](), [profiling](), [tests]() ### AI overview Khan Academy engineers profile App Engine Memcached to estimate access latency and evaluate whether on-demand content fetching could support separate content trees for each language. Most requests showed 1-4 ms latency, usually below 10 ms, while a single multi-get performed similarly to a single get. ### Source excerpt By Ben Kraft Last year, Willow wrote about how we optimized our in-memory content data to take up ... Read more ## ML based ranking using Nrtsearch DevFeed: [ML based ranking using Nrtsearch]() Original publisher: [Read original article]() Author: Mohammad Mohtasham (Software Engineer); Tao Yu (Software Engineer) Published: 2026-09-16T00:00:00Z Content type: article Language: en Sources: [Yelp]() Topics: [Inference](), [Scalability](), [Latency](), [recommendation systems](), [Open Source]() Tags: [bridge](), [inference](), [machine-learning](), [overhead](), [performance](), [pipeline](), [scalability](), [service]() ### AI overview Yelp's Nrtsearch Inference Plugin embeds machine-learning ranking directly in the search layer. The article explains the scoring workflow, including model configuration, feature extraction, candidate ranking, and application-specific business logic. It describes how co-locating feature storage and inference reduces network transfer, serialization overhead, and latency compared with a standalone inference service. ### Source excerpt We've extended Nrtsearch with the Inference Plugin, which embeds ML-based ranking directly in the search layer -- eliminating the need for a standalone scoring service. We use Nrtsearch (read more information on the blog post), a Lucene-based open-source search engine built by Yelp, to power a variety of applications such as business search, reviews search, ad delivery and photo search. In this blog post, we give a high-level overview of the Machine Learning (ML) based scoring workflow in Nrtsearch. We'll show how ML models are configured and loaded, and how different applications use custom business logic to develop, test, and... ## Google and OpenAI take different approaches to reducing voice-agent latency DevFeed: [Google and OpenAI take different approaches to reducing voice-agent latency]() Original publisher: [Read original article]() Author: Amanda Caswell Published: 2026-09-15T21:50:15Z Content type: comparison Language: en Sources: [The New Stack]() Topics: [Latency](), [Google AI](), [OpenAI](), [API](), [Orchestration](), [real-time](), [Artificial Intelligence]() Tags: [ai](), [ai-agents](), [ai-engineering](), [api](), [comparison](), [google-ai](), [large-language-models](), [latency](), [openai](), [orchestration](), [real-time]() ### AI overview The article compares Google's Gemini 3.8 Live Extended Thinking with OpenAI's GPT-Live-1 for reducing latency in voice agents. Google keeps speech, reasoning, and asynchronous tool execution in one stateful session, while OpenAI uses a real-time conversation model alongside a backend reasoning model, shifting more orchestration to the application. ### Source excerpt Voice agents have a latency problem that shows up as soon as they have to do real work. Within five The post OpenAI's voice model doesn't think. That's the point. appeared first on The New Stack. ## Kubernetes v1.37: Pod-Level Resource Managers graduated to Beta DevFeed: [Kubernetes v1.37: Pod-Level Resource Managers graduated to Beta]() Original publisher: [Read original article]() Author: Kevin Torres Martinez Published: 2026-09-15T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [cpu](), [Hardware](), [Latency](), [API](), [gRPC](), [Monitoring]() Tags: [api](), [cpu](), [hardware](), [kubernetes](), [latency](), [monitoring](), [numa](), [release](), [v1]() ### AI overview Kubernetes v1.37 graduates Pod-Level Resource Managers to Beta, disabled by default. The feature lets Kubelet resource managers use pod-level declarations for hardware placement, enabling NUMA-aligned exclusive resources for primary containers while placing sidecars in a shared pod-isolated pool. The release also adds pod-level reporting to the PodResources gRPC API. ### Source excerpt With the release of Kubernetes v1.37, the Pod-Level Resource Managers feature has graduated to Beta status (disabled by default)! First introduced as an Alpha feature in Kubernetes v1.36, this enhancement builds on Pod-Level Resources by equipping Kubelet's Topology Manager, CPU Manager, and Memory Manager to use Pod-level resource declarations (.spec.resources) directly when making hardware placement decisions. Bringing pod-level resources to node managers Before this feature, obtaining exclusive NUMA-aligned CPU cores or memory for latency-critical applications forced cluster operators into an all-or-nothing choice: assign integer resource requests to every container in the Pod, or forfeit exclusive NUMA alignment entirely. For modern workloads running lightweight sidecars (such as logging agents or telemetry exporters), allocating dedicated physical cores to auxiliary containers was wasteful. Pod-Level Resource Managers solves this challenge by enabling hybrid allocation models. The Kubelet can reserve exclusive NUMA-aligned resources for primary application containers while placing non-Guaranteed sidecars into a pod-isolated shared pool. This ensures primary workloads get unthrottled, NUMA-local performance while sidecars benefit from running in a pod-isolated shared pool, enjoying local NUMA alignment and protection from external node interference without consuming dedicated physical cores. What's new in Beta Graduating to Beta brings key operational and API enhancements: Graduation to Beta: Controlled by the PodLevelResourceManagers feature gate, available to opt in (disabled by default) in Kubernetes v1.37. PodResources API Reporting: The v1 PodResources gRPC service (PodResourcesLister) introduces top-level cpu_ids and memory fields on PodResources responses. Monitoring tools and device plugins can query pod-level exclusive assignments directly without double-counting container allocations. Getting started and providing feedback For a deep dive into the tech ## Optimizing cost and latency with Amazon Bedrock prompt caching DevFeed: [Optimizing cost and latency with Amazon Bedrock prompt caching]() Original publisher: [Read original article]() Author: Daniel Abib Published: 2026-09-15T16:18:19Z Content type: tutorial Language: en Sources: [Artificial Intelligence]() Topics: [Amazon Bedrock](), [Caching](), [Latency](), [API](), [foundation-models](), [LangChain](), [Multi-tenancy]() Tags: [advanced-300](), [amazon-bedrock](), [api](), [caching](), [foundation-models](), [integration](), [langchain](), [latency](), [pricing](), [technical-how-to](), [tokens]() ### AI overview This AWS post explains how Amazon Bedrock prompt caching can reduce repeated input-token costs by up to 90 percent and lower time to first token when requests reuse the same context. It presents six scenarios using the Converse API, including document, system prompt, tool definition, mixed TTL, tenant-isolated, and LangChain caching. ### Source excerpt Prompt caching in Amazon Bedrock can cut input token costs by up to 90% when you repeatedly send the same context to foundation models. This post walks through six practical prompt caching scenarios using the Converse API: message content, system prompt, tool definition, mixed TTL, tenant isolation, and LangChain integration. ## Why your Redis is slow: it is single threaded and you sent it KEYS DevFeed: [Why your Redis is slow: it is single threaded and you sent it KEYS]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-09-15T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Redis](), [Latency](), [Concurrency]() Tags: [blocking](), [caching](), [command](), [latency](), [observability](), [performance](), [redis](), [single-threaded]() ### AI overview The article explains that Redis executes commands one at a time on a single thread, so expensive commands can block every client and cause latency spikes across all operations. It identifies KEYS as a key offender and also discusses large DEL operations, unbounded LRANGE, SMEMBERS on large sets, and FLUSHALL. Redis 6 threaded I/O helps with socket handling but does not change single-threaded command execution. ### Source excerpt Redis executes commands one at a time on a single thread, so any command that takes 200ms makes every other client wait 200ms regardless of how trivial their request was. KEYS scans the entire keyspace and is the obvious offender, but DEL on a large collection, unbounded LRANGE, SMEMBERS on a big set and FLUSHALL are all linear in the size of the data and block identically. The tell is a latency spike across all operations at once while CPU and network look normal. ## How to operate shared platforms safely at agent scale DevFeed: [How to operate shared platforms safely at agent scale]() Original publisher: [Read original article]() Author: Candace Shamieh; T Zhang; Gabriele Baldoni Published: 2026-09-15T00:00:00Z Content type: article Language: en Sources: [Datadog | The Monitor blog]() Topics: [Platform Engineering](), [Loop Engineering](), [AI Agent](), [sdlc](), [Latency](), [Low Latency](), [Concurrency](), [Software]() Tags: [agent](), [agent-observability](), [agentic-workflows](), [agents](), [ai](), [ai-agent](), [ci](), [concurrency](), [latency](), [low-latency](), [operational](), [performance](), [platform-engineering](), [platforms](), [queue](), [recovery](), [timeout]() ### AI overview This Datadog article explains how platform teams can operate shared platforms safely as AI agent workloads scale across teams. It discusses modeling demand across agent trajectories, planning capacity across dependencies such as CI queues and sandbox pools, handling contention and recovery behavior, and preserving control across system boundaries. ### Source excerpt Learn how Datadog models agent demand, allocates capacity under contention, and preserves control as AI agent workloads scale across shared platforms. ## Agoda Replaces 72-Shard SQL Server Price Cache with DragonflyDB DevFeed: [Agoda Replaces 72-Shard SQL Server Price Cache with DragonflyDB]() Original publisher: [Read original article]() Author: Leela Kumili Published: 2026-09-14T13:48:00Z Content type: news Language: en Sources: [InfoQ]() Topics: [Cache](), [Caching](), [migration](), [sql-server](), [Latency](), [Prometheus]() Tags: [a-b-testing](), [agoda-price-cache-dragonflydb](), [architecture-design](), [cache](), [caching](), [development](), [devops](), [distributed-cache](), [distributed-data](), [distributed-systems](), [latency](), [microservices](), [migration](), [news](), [prometheus](), [redis](), [sql-server]() ### AI overview Agoda migrated its 1.5 TB hotel Price Cache from 72 Microsoft SQL Server shards to DragonflyDB. The staged migration used workload testing, dual reads, parity metrics, and gradual traffic shifting; Agoda reported substantially lower P99 read latency. ### Source excerpt Agoda migrated its 1.5 TB hotel Price Cache from 72 SQL Server shards to DragonflyDB to handle growing read and write volumes. The migration used staged dual reads, parity validation, gradual traffic shifting, and decentralized failover detection. Agoda reports an approximately eightfold reduction in P99 read latency, with two DragonflyDB clusters providing high availability. By Leela Kumili ## Bringing QUIC to Seastar DevFeed: [Bringing QUIC to Seastar]() Original publisher: [Read original article]() Author: Cynthia Dunlop Published: 2026-09-14T13:02:07Z Content type: article Language: en Sources: [ScyllaDB]() Topics: [Seastar](), [networking](), [Remote Procedure Call (RPC)](), [C++](), [Latency](), [Protocol (disambiguation)]() Tags: [benchmarks](), [c-plus-plus](), [collaboration](), [engineering](), [latency](), [network](), [networking](), [rpc](), [scylladb](), [seastar](), [udp]() ### AI overview This article describes a University of Warsaw student project conducted with ScyllaDB to implement a QUIC transport for Seastar using the sans-I/O ngtcp2 library. It also adapts Seastar's RPC layer to operate over QUIC and reports benchmark results showing predictable overhead on a lossless loopback and benefits when packets are dropped. ### Source excerpt We built a QUIC transport for Seastar on top of ngtcp2's sans-I/O state machine, then adapted RPC to it twice: 1) as a one-to-one socket replacement, and 2) a QUIC-aware approach that opens a fresh stream per call. ## Chip Huyen explains how to cut inference costs without new hardware DevFeed: [Chip Huyen explains how to cut inference costs without new hardware]() Original publisher: [Read original article]() Author: Tim Koopmans Published: 2026-09-13T15:00:00Z Content type: article Language: en Sources: [The New Stack]() Topics: [Inference Performance](), [Low-Latency Inference](), [model-deployment](), [Optimization](), [Frontier Model](), [AI Engineering](), [Latency](), [math]() Tags: [ai](), [ai-agents](), [ai-engineering](), [ai-infrastructure](), [frontier-model](), [inference](), [low-latency](), [metrics](), [optimization](), [performance](), [post-contributed](), [reasoning](), [scylladb](), [sponsor-scylladb](), [sponsored](), [sponsored-post-contributed](), [tokens]() ### AI overview Chip Huyen explains why inference costs can outweigh one-time frontier-model training costs and outlines ways to optimize inference without new hardware. The article emphasizes latency metrics such as time to first token, time per output token, end-to-end latency, and goodput, especially for reasoning models. ### Source excerpt Last October, the P99 conference -- the online gathering for developers focused on high-performance, low-latency applications -- featured a cracking The post Chip Huyen explains how to cut inference costs without new hardware appeared first on The New Stack. ## 4 Speculative Decoding Variants DevFeed: [4 Speculative Decoding Variants]() Original publisher: [Read original article]() Author: Avi Chawla Published: 2026-09-12T20:03:46Z Content type: article Language: en Sources: [Daily Dose of Data Science]() Topics: [dynatrace](), [MCP](), [MCP Server](), [Traces](), [Latency]() Tags: [coding-agents](), [dynatrace](), [latency](), [mcp](), [mcp-server](), [traces]() ### AI overview This article explains how Dynatrace's open-source MCP server and related skills bring production traces and logs into coding agents for performance investigations. It also introduces speculative decoding, in which a cheaper drafter proposes tokens and a target model verifies them in parallel, then outlines variants based on how drafts are produced. ### Source excerpt ...covered with tradeoffs in production. ## Second-Gen Single-Rack AWS Outposts Puts 2,688 vCPUs and 100TB of EBS in One 42U Rack DevFeed: [Second-Gen Single-Rack AWS Outposts Puts 2,688 vCPUs and 100TB of EBS in One 42U Rack]() Original publisher: [Read original article]() Author: Harold Fritts Published: 2026-09-12T18:24:22Z Content type: news Language: en Sources: [StorageReview.com]() Topics: [AWS Outposts](), [Amazon Web Services](), [cloud-infrastructure](), [Network](), [Latency](), [data-processing](), [Security]() Tags: [5g](), [automation](), [aws](), [aws-outposts](), [cloud](), [compute](), [connectx](), [data-processing](), [enterprise](), [governance](), [latency](), [low-latency](), [networking](), [security]() ### AI overview AWS has generally released a second-generation single-rack AWS Outposts configuration: a self-contained 42U rack combining compute, storage, and networking with up to 2,688 vCPUs and 100 TB of Amazon EBS. The article describes its on-premises cloud compatibility, compact footprint, supported instance families, and accelerated networking options for trading floors and 5G cores. ### Source excerpt AWS has made second-generation single-rack AWS Outposts generally available, a self-contained 42U rack that puts compute, storage, and networking together with up to 2,688 vCPUs and 100 TB of Amazon EBS. It runs the same APIs, console, automation, governance policies, and security controls as the multi-rack second-generation Outposts and the parent AWS Region, so an The post Second-Gen Single-Rack AWS Outposts Puts 2,688 vCPUs and 100TB of EBS in One 42U Rack appeared first on StorageReview.com. ## KDE Plasma 6.8 Remote Desktop To Enjoy Lower Latency Performance DevFeed: [KDE Plasma 6.8 Remote Desktop To Enjoy Lower Latency Performance]() Original publisher: [Read original article]() Author: Michael Larabel Published: 2026-09-12T10:07:48Z Content type: news Language: en Sources: [Phoronix]() Topics: [Latency](), [client](), [Wayland](), [Protocol (disambiguation)](), [configuration](), [GUI]() Tags: [configuration](), [desktop](), [desktop-linux](), [image](), [latency](), [linux-benchmarking](), [linux-hardware-benchmarks](), [linux-hardware-reviews](), [linux-how-to](), [linux-performance](), [linux-server-benchmarks](), [open-source-graphics](), [performance](), [phoronix](), [phoronix-test-suite](), [release](), [remote](), [switching](), [time](), [ubuntu-benchmarks](), [ubuntu-hardware]() ### AI overview KDE Plasma 6.8 improves KRDP remote desktop latency by dropping incoming frames when the encoder outpaces the client. The release also includes input-method, clipboard, Wayland protocol, wallpaper dialog, and XWayland session configuration fixes. ### Source excerpt Along with KDE Plasma 6.8 beta releasing this week, there were some other interesting Plasma changes this week worthy of a shout-out... ## How to track Webflow Cloud app metrics with Datadog Dashboards DevFeed: [How to track Webflow Cloud app metrics with Datadog Dashboards]() Original publisher: [Read original article]() Author: Ismail Ajagbe Published: 2026-09-12T00:00:00Z Content type: tutorial Language: en Sources: [Webflow Blog]() Topics: [dashboards](), [Latency](), [API](), [API keys](), [Next.js](), [Node.js](), [Application Performance Management (APM)]() Tags: [api](), [api-keys](), [apm](), [dashboards](), [guides](), [latency](), [metrics](), [node-js]() ### AI overview Tutorial showing how to monitor Webflow Cloud applications by sending request counts, error counts, and latency metrics from Next.js Route Handlers to Datadog's v2 metrics API, then visualizing them in a Datadog dashboard. ### Source excerpt Learn how to track Webflow Cloud app metrics with Datadog by sending request, latency, and error data from your Route Handlers to the metrics API and dashboard. ## New Cache Aware Scheduling Fixes Out For Testing DevFeed: [New Cache Aware Scheduling Fixes Out For Testing]() Original publisher: [Read original article]() Author: Michael Larabel Published: 2026-09-11T18:00:14Z Content type: news Language: en Sources: [Phoronix]() Topics: [Cache](), [Linux](), [cpu](), [intel](), [Latency]() Tags: [cache](), [cpu](), [desktop-linux](), [feature](), [intel](), [kernel](), [latency](), [linux](), [linux-benchmarking](), [linux-hardware-benchmarks](), [linux-hardware-reviews](), [linux-how-to](), [linux-performance](), [linux-server-benchmarks](), [open-source-graphics](), [phoronix](), [phoronix-test-suite](), [processors](), [series](), [testing](), [ubuntu-benchmarks](), [ubuntu-hardware]() ### AI overview New fixes to Linux kernel Cache Aware Scheduling are available for testing on Intel hybrid CPUs. The patches address tasks being stranded or moved away from their preferred last-level cache, along with a use-after-free issue, while known interference with Intel Turbo Boost Max Technology remains unresolved. ### Source excerpt Since being merged back during the Linux 7.2 cycle, the Cache Aware Scheduling code continues seeing more fixes as users test out this scheduling enhancement on different processors and system configurations... ## Evolving Pinterest's Embedding Retrieval Platform DevFeed: [Evolving Pinterest's Embedding Retrieval Platform]() Original publisher: [Read original article]() Author: Pinterest Engineering Published: 2026-09-11T15:01:03Z Content type: article Language: en Sources: [Pinterest Engineering Blog - Medium]() Topics: [Architecture & Design](), [Algorithms](), [Scalability](), [Latency](), [IO]() Tags: [ann](), [cost](), [embedding](), [embeddings](), [engineering](), [infrastructure](), [latency](), [memory](), [models](), [pinterest](), [platform](), [quantization](), [real-time](), [retrieval](), [scalability](), [search]() ### AI overview Pinterest describes evolving its Manas embedding-retrieval platform to address the cost, scale, and flexibility challenges of serving billions of embeddings. The excerpt covers ANN search, vector quantization, and SSD-based serving. ### Source excerpt Authors: Bowen Zhou | Staff Software Engineer; Shan Gao | Senior Software Engineer; Jingwen Hu | Software Engineer II; Wenjiang Chu | Staff Software Engineer The Billion-Embedding Challenge At Pinterest, the "signal" is our lifeblood. Whether it's a home decor enthusiast finding the perfect rug or a fashion seeker discovering a new aesthetic, our discovery engine relies on understanding deep semantic relationships to help our users find inspirations. Over the last few years, the explosive growth of embedding-based retrieval has fundamentally transformed how we surface these signals -- and at the heart of that transformation is Manas, Pinterest's in-house distributed search platform. Embedding Retrieval is one of the core capabilities of Manas, supporting multiple approximate nearest neighbor search algorithms, hybrid queries with both token and embedding clauses, as well as real-time updates to ensure fresh contents become searchable within seconds. Deployed on over 80 clusters and serving billions of embeddings, Manas embedding retrieval powers all major product surfaces at Pinterest including Home Feed, Search, Related Pins, Ads, and Notifications. However, as our corpus scales toward tens of billions of embeddings and our models capture increasingly complex interactions, we face mounting challenges around cost efficiency, scalability, and flexibility. On the infrastructure side, traditional ANN algorithms like HNSW are notoriously memory-hungry -- they require the entire index to reside in RAM to maintain low query latency, making cost grow linearly with corpus size. On the modeling side, the classic two-tower retrieval paradigm is too restrictive: it reduces each candidate to a single embedding and scores relevance through a simple dot product, leaving little room to express richer, context-dependent notions of similarity. To tackle these challenges, our team has been evolving Manas's embedding retrieval stack across three fronts: Quantization. We reduce the memor ## How LinkedIn Trains AI Job Search 8x Faster with Multi-Teacher Distillation DevFeed: [How LinkedIn Trains AI Job Search 8x Faster with Multi-Teacher Distillation]() Original publisher: [Read original article]() Author: Claudio Masolo Published: 2026-09-11T10:00:00Z Content type: news Language: en Sources: [InfoQ]() Topics: [recommendation systems](), [Large Language Model](), [Language models](), [sglang](), [Latency]() Tags: [agentic-ai-architecture](), [ai](), [ai-ml-data-engineering](), [artificial-intelligence](), [large-language-models](), [latency](), [liger](), [linkedin](), [linkedin-ai-multi-teacher](), [llms](), [ml-data-engineering](), [news](), [search](), [sglang](), [training]() ### AI overview LinkedIn describes a multi-teacher distillation pipeline for AI-powered job search that trains a 0.6B-parameter ranking model. The article focuses on SGLang-based teacher serving, online and offline distillation, and training optimizations reported to produce roughly an eightfold speedup. ### Source excerpt LinkedIn has published details of the training infrastructure behind its AI-powered job search, describing a multi-teacher distillation pipeline that compresses knowledge from large teacher models into a compact 0.6B-parameter ranking model. By Claudio Masolo ## Latency: What It Is and What Limits It (Part II) DevFeed: [Latency: What It Is and What Limits It (Part II)]() Original publisher: [Read original article]() Author: Franco Fernando Published: 2026-09-11T09:07:39Z Content type: article Language: en Sources: [The Polymathic Engineer]() Topics: [Latency]() Tags: [latency]() ### AI overview An article explaining where latency occurs in the stack, how delays accumulate, and how to measure them. ### Source excerpt Where delays hide in the stack, how they add up, and how to measure them. ## DeepSeek's new model sets a template for powerful LLMs that run lean DevFeed: [DeepSeek's new model sets a template for powerful LLMs that run lean]() Original publisher: [Read original article]() Author: Tobias Mann Published: 2026-09-11T07:15:00Z Content type: news Language: en Sources: [www.theregister.com - Articles]() Topics: [deepseek](), [Large Language Model](), [Caching](), [Latency](), [Google]() Tags: [ai-and-ml](), [cache](), [compute](), [cost](), [datacenter](), [deepseek](), [embedding](), [flash](), [google](), [gpu](), [latency](), [llm](), [llms](), [model](), [release]() ### AI overview DeepSeek V4.1 Flash is a larger LLM whose architectural changes aim to reduce serving memory and compute needs. The article highlights lower KV-cache consumption, improved prompt processing, and an N-gram-based conditional memory module. ### Source excerpt DeepSeek V4.1 Flash proves that just because you build a bigger model doesn't mean you need more GPUs to serve it ## Choosing among write-through, write-behind, and write-around caching strategies DevFeed: [Choosing among write-through, write-behind, and write-around caching strategies]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-09-11T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Caching](), [Databases](), [Redis](), [consistency](), [Latency]() Tags: [cache](), [caching](), [consistency](), [database](), [latency](), [orders](), [postgres](), [redis](), [strategies](), [system-design](), [write](), [write-behind](), [write-through]() ### AI overview This article compares write-through, write-behind, and write-around caching strategies. It explains their trade-offs among database latency, cache hit rates, consistency, and the risk of losing acknowledged writes when a process fails. ### Source excerpt The three write strategies differ in what happens between the cache write and the database write, and that gap is where correctness lives. Write-through updates both before acknowledging, so it is safe and pays the full database latency. Write-behind acknowledges after the cache write and flushes later, which is fast and can lose acknowledged writes if the process dies. Write-around skips the cache entirely on write, which avoids caching data nobody reads and costs a miss on the next read. ## 118 million queries per second on Neki DevFeed: [118 million queries per second on Neki]() Original publisher: [Read original article]() Author: Hirad Pourtahmasbi Published: 2026-09-11T00:00:00Z Content type: article Language: en Sources: [Blog -- PlanetScale]() Topics: [Database](), [benchmarking](), [Latency]() Tags: [benchmark](), [database](), [engineering](), [errors](), [latency](), [neki](), [postgres]() ### AI overview The article reports a Neki benchmark that sustained 118 million queries per second across 512 primary-only Postgres shards holding 1.22 PiB of data. The read-only, single-shard point-select workload ran for 16 minutes, with reported router and client p99 latency plus fleet IOPS, network throughput, and error rate. ### Source excerpt We ran a massive, sharded Postgres database at 118.5 million queries per second, with 200k queries per second on each shard across 512 shards. ## Building Pinterest's VLM Serving Stack on NVIDIA Dynamo DevFeed: [Building Pinterest's VLM Serving Stack on NVIDIA Dynamo]() Original publisher: [Read original article]() Author: Pinterest Engineering Published: 2026-09-10T23:08:16Z Content type: article Language: en Sources: [Pinterest Engineering Blog - Medium]() Topics: [vlm](), [Inference](), [model-deployment](), [Orchestration](), [Latency](), [Caching](), [Open Source Models & Datasets]() Tags: [blackwell](), [cache](), [dynamo](), [embeddings](), [engineering](), [inference](), [latency](), [llm](), [multimodal](), [multimodal-ai](), [nvidia](), [nvidia-blackwell](), [open-source](), [orchestration](), [performance](), [pinterest](), [vllm](), [vlm](), [vlm-serving]() ### AI overview Pinterest describes its VLM serving stack built on NVIDIA Blackwell GPUs and NVIDIA Dynamo. The stack addresses multimodal inference demands such as image processing, variable prefill costs, KV-cache pressure, routing, and cache offloading. ### Source excerpt Lei Pan | Senior Software Engineer; Salina Wu | Senior Software Engineer; Cristian Lopez | Software Engineer I; Guangtong Bai | Staff Software Engineer; Soam Acharya | Principal Engineer; Saurabh Vishwas Joshi | Principal Engineer; Chia-Wei Chen | Staff Software Engineer; Ambud Sharma | Principal Engineer Why VLM Serving Matters at Pinterest Pinterest is a visual search and discovery platform, so its AI systems must reason over both language and visual content. Vision-language models (VLMs), which can interpret images, compare visual candidates, and respond naturally to user intent, are becoming the foundation for the next generation of Pinterest experiences: Pinterest Assistant, hybrid search, multimodal reranking, content understanding, signal generation, content safety, and more. This direction also reflects Pinterest's broader strategy to customize open-source models to meet its product & scale needs. Pinterest Assistant is a standout example. This multi-turn conversational experience covers both user language and visual content. Serving it requires low-latency VLM inference over rich multimodal context as well as reworking Qwen3-VL with proprietary multimodal embeddings to cut runtime cost while improving performance. Serving VLMs, however, introduces more challenges compared to text-only LLM workloads. Requests may carry multiple images, require extra vision encoder computation, incur larger and more variable prefill cost, and create higher KV cache pressure. To support this new class of models & product experiences, we built Pinterest's VLM serving stack on top of NVIDIA Blackwell GPUs and NVIDIA Dynamo. Blackwell GPUs incorporate many architectural innovations that are uniquely positioned for today's most demanding AI workloads -- including higher BF16/FP8 compute throughput, increased memory bandwidth, and larger HBM memory capacity -- that enable dramatically higher performance for inference. Dynamo provides a distributed inference orchestration layer that g ## GPT Images 2.5 promises edits that leave the rest of your image alone DevFeed: [GPT Images 2.5 promises edits that leave the rest of your image alone]() Original publisher: [Read original article]() Author: Meredith Shubel Published: 2026-09-10T19:46:32Z Content type: news Language: en Sources: [The New Stack]() Topics: [Artificial Intelligence](), [Latency]() Tags: [ai](), [applications](), [cost](), [generation](), [gpt](), [images](), [latency](), [model](), [models](), [openai](), [production](), [prototyping](), [search](), [workflows]() ### AI overview OpenAI's GPT Images 2.5 introduces two image-editing models: Flare, optimized for speed and lower latency, and Sunburst, designed for greater precision and control. Both have the same listed token rates, but OpenAI does not explain their actual token consumption or comparative per-image costs. ### Source excerpt When OpenAI launched GPT Images 2.5 this week, the company promised better results for a common editing task: changing one The post GPT Images 2.5 promises edits that leave the rest of your image alone appeared first on The New Stack. [Next page]()