# Latency Published articles for Latency. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Scaling Enterprise Testing on Kubernetes with Argo DevFeed: [Scaling Enterprise Testing on Kubernetes with Argo]() Original publisher: [Read original article]() Author: Capital One Tech Published: 2026-09-18T14:01:33Z Content type: article Language: en Sources: [Capital One Tech]() Topics: [Testing](), [Kubernetes](), [scaling](), [Orchestration](), [Multitenancy](), [Open Source](), [Cloud Native Ecosystem](), [parallel](), [Latency]() Tags: [argo](), [ci-cd](), [cloud-native](), [cluster](), [cncf](), [enterprise](), [infrastructure](), [kubernetes](), [latency](), [open-source](), [parallel](), [scaling](), [software-engineering](), [testing]() ### AI overview A case study of Capital One's rebuilt enterprise testing platform, which uses Kubernetes and open-source Argo ecosystem projects to run large numbers of parallel tests across teams. It describes the limitations of the previous pipeline and the platform's focus on scalability, multi-tenancy, and stability. ### Source excerpt How we built a resilient, multi-tenant test platform on open-source workflows. Every engineer knows the quiet anxiety of watching a CI/CD pipeline run. You commit, you wait for the build and then you wait some more. Now imagine that at enterprise scale: thousands of test agents running at once, across dozens of teams, all validating business-critical services on the same shared infrastructure. That's the problem our platform exists to solve, and for years, the way we solved it was quietly holding us back. At Capital One, we're a technology company that happens to operate in one of the most tightly regulated industries there is. That combination raises the bar: our internal platforms strive to be fast and safe, to scale for many teams at once and to prevent any single workload from destabilizing the rest. So when our first-generation testing pipeline started hitting hard limits, we didn't just move it; we rebuilt it on Kubernetes using open-source, Cloud Native Computing Foundation (CNCF) projects from the Argo ecosystem. In this post, we'll walk through three things: Why we moved off our first-generation pipeline How Argo orchestrates a single test run, end to end The guardrails that keep the platform stable when thousands of tests arrive at once The headline isn't just "we moved to Kubernetes." It's that we learned to run massively parallel test workloads without letting any single burst threaten a shared, multi-tenant cluster. Why we moved off our first-generation pipeline Our original platform got the job done, but it was built from a highly complex set of managed services linked together by a proprietary orchestrator. Execution logic lived inside cloud-specific state-machine definitions, far away from the containers actually running the tests. Three pain points stood out: State-machine sprawl. Changing execution logic meant editing large, cloud-specific workflow definitions. The orchestration lived far away from the code it was orchestrating. Cold-start latency. ## Introducing Amazon SageMaker HyperPod Inference Gateway DevFeed: [Introducing Amazon SageMaker HyperPod Inference Gateway]() Original publisher: [Read original article]() Author: Vinay Arora Published: 2026-09-18T13:08:34Z Content type: release Language: en Sources: [Artificial Intelligence]() Topics: [Amazon SageMaker HyperPod](), [Model Routing](), [model-serving](), [Inference Performance](), [Amazon Elastic Kubernetes Service](), [Kubernetes](), [AI Inference](), [GPU](), [AI Platform](), [Prometheus]() Tags: [amazon-eks](), [amazon-sagemaker-hyperpod](), [announcements](), [expert-400](), [gpu](), [inference](), [kubernetes](), [latency](), [metrics](), [model-serving](), [performance](), [routing]() ### AI overview Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals and model-serving metrics to route inference requests to suitable pods, aiming to reduce GPU waste and first-token latency without application changes. ### Source excerpt Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals to send each inference request to the best-suited pod, cutting first-token latency by up to 82% with no changes to your model servers or client applications. ## Your agent is only as good as your infrastructure DevFeed: [Your agent is only as good as your infrastructure]() Original publisher: [Read original article]() Author: Selene Cecchinel Published: 2026-09-18T13:00:00Z Content type: opinion Language: en Sources: [The New Stack]() Topics: [AI Agent](), [Inference](), [Latency](), [reliability]() Tags: [agent](), [ai](), [ai-agent](), [ai-agents](), [ai-infrastructure](), [ai-operations](), [coreweave](), [infrastructure](), [latency](), [post-contributed](), [production](), [reliability](), [sponsor-coreweave](), [sponsored-post-contributed]() ### AI overview This article explains why AI agents require different infrastructure from chatbots. Agent workflows chain inference calls, tool execution, database access, API calls, and code execution, making infrastructure a major determinant of latency, reliability, and cost in production. ### Source excerpt You built a great agent, but something happened when it moved into production. In testing, your agent reviewed pull requests The post Your agent is only as good as your infrastructure appeared first on The New Stack. ## Article: Architecting Secure and Scalable Facial Verification Systems DevFeed: [Article: Architecting Secure and Scalable Facial Verification Systems]() Original publisher: [Read original article]() Author: Praveen Kumar Gopalakrishnan Published: 2026-09-18T09:00:00Z Content type: article Language: en Sources: [InfoQ]() Topics: [distributed-systems](), [systems](), [Computer vision](), [scaling](), [Concurrency](), [Zero Trust](), [API](), [Data Quality](), [client](), [Access Control](), [Latency](), [Encryption]() Tags: [access-control](), [ai](), [ai-ml-data-engineering](), [architecture](), [architecture-design](), [article](), [cloud](), [computer-vision](), [concurrency](), [data-quality](), [development](), [devops](), [distributed-systems](), [encryption](), [latency](), [ml-data-engineering](), [risk](), [scaling](), [secure-scalable-facial-verification](), [security](), [systems](), [zero-trust]() ### AI overview This article presents a four-layer architecture for high-volume facial verification, covering client-side data validation, asynchronous processing, decoupled detection and verification, risk-based thresholds, and zero-trust handling of personal data. ### Source excerpt When three thousand employees verify at once, synchronous API calls collapse. This article presents a four-layer architecture for high-volume face verification: client-side filtering that cut cloud costs 30%, decoupled detection and verification enabling 10x scaling, risk-based dynamic thresholds, and zero-trust privacy with consent gates and automated data purging for GDPR and HIPAA. By Praveen Kumar Gopalakrishnan ## Presto: A match-action TCP stack for the terabit era DevFeed: [Presto: A match-action TCP stack for the terabit era]() Original publisher: [Read original article]() Author: Rajath Shashidhara Published: 2026-09-18T03:34:15Z Content type: article Language: en Sources: [APNIC Blog]() Topics: [presto](), [Computing](), [Hardware](), [cpu](), [Latency](), [Network](), [rdma](), [Programming]() Tags: [amd](), [architecture](), [connectx](), [cpu](), [data-centres](), [efficiency](), [guest-post](), [hardware](), [high-performance-computing](), [latency](), [network](), [networks](), [nvidia](), [performance](), [presto](), [processing](), [programming](), [protocol](), [rdma](), [scale](), [software](), [tcp](), [tech-matters]() ### AI overview The post presents Presto, a complete TCP stack implemented as match-action operations on reconfigurable match-action table architectures. It argues that software TCP consumes substantial CPU resources at high data-center network speeds, while hardware transports such as RDMA can be rigid and difficult to operate at scale. Presto aims to combine programmable software behavior with line-rate packet processing, low latency, and ASIC-like power efficiency. ### Source excerpt Guest Post: Presto is built on the RMT architecture that underlies programmable switches, and addresses RMT's main constraints. ## DigitalOcean Announces General Availability of Advanced Managed Databases for MySQL and PostgreSQL DevFeed: [DigitalOcean Announces General Availability of Advanced Managed Databases for MySQL and PostgreSQL]() Original publisher: [Read original article]() Author: Waverly Swinton Published: 2026-09-17T22:30:18Z Content type: release Language: en Sources: [DigitalOcean]() Topics: [Databases](), [MySQL](), [PostgreSQL](), [Digital Ocean](), [Latency](), [reliability](), [Availability](), [Monitoring](), [retry](), [VPC]() Tags: [availability](), [databases](), [digitalocean](), [failover](), [latency](), [monitoring](), [mysql](), [postgresql](), [product-updates](), [production](), [reliability](), [vpc]() ### AI overview DigitalOcean announces general availability of its Managed Databases Advanced Edition for MySQL and PostgreSQL. The release highlights performance and reliability improvements, failover behavior, lower storage pricing, monitoring, and VPC connectivity for production workloads. ### Source excerpt As your business scales, your database shifts from a simple storage layer to the critical heart of your application architecture. For years, DigitalOcean has helped thousands of startups and growing businesses effortlessly launch and scale fully managed PostgreSQL, MySQL, Valkey, and MongoDB databases without the burden of complex routine maintenance. But when traffic surges, data footprints expand, and uptime becomes non-negotiable, high-growth workloads demand a stronger foundation. That is why we are announcing general availability of DigitalOcean Managed Databases Advanced Edition for both MySQL and PostgreSQL. General Availability: Enterprise-Grade Performance and Reliability for Production Workloads Since our public preview in April, more than 150 customers have run workloads on Advanced Edition. We've been focused on improving performance and reliability across both engines: Performance Gains at Scale: As database activity accelerates, both engines demonstrate marked efficiency improvements, with Managed PostgreSQL internal benchmarks delivering up to 38% higher throughput* alongside a 50% reduction in p99 latency.** Rapid Failover Capabilities: Integrated proxy architecture is designed to avoid application reconnects in most failover events. Across twenty primary-loss simulations in internal benchmarking, MySQL Advanced Edition clusters promoted a replacement primary in under 3 seconds on average, remaining well within standard client connection retry thresholds. Lower Total Cost of Ownership (TCO): Building on Standard Edition's ease of use, built-in monitoring, and zero egress fees, we've also reduced storage prices for Advanced Edition by 46% (down to $0.115 per GiB/month), (see our pricing page for current rates) materially lowering TCO for teams running large-scale workloads. In addition to these performance gains and efficiencies, we've also extended the platform so you can run your database your way. Connect securely over VPC, offload reads with conne ## Multi-modal autoscaling with Amazon EC2 Auto Scaling: adding signals for faster, more reliable scaling DevFeed: [Multi-modal autoscaling with Amazon EC2 Auto Scaling: adding signals for faster, more reliable scaling]() Original publisher: [Read original article]() Author: Shubhendu Dubey Published: 2026-09-17T18:18:38Z Content type: tutorial Language: en Sources: [AWS Compute Blog]() Topics: [autoscaling](), [Amazon EC2](), [scaling](), [Amazon CloudWatch](), [Amazon Web Services](), [AWS CloudFormation](), [Deployment](), [dashboards]() Tags: [advanced-300](), [amazon-cloudwatch](), [amazon-ec2](), [api](), [auto-scaling](), [autoscaling](), [aws](), [aws-cloudformation](), [deployment](), [forecasting](), [latency](), [load](), [load-balancer](), [scaling](), [technical-how-to]() ### AI overview This tutorial explains multi-modal autoscaling with Amazon EC2 Auto Scaling by combining CPU, request-count, application-level, and predictive signals. It describes a controlled test in which application metrics avoided rejected checkout sessions and reduced p99 latency compared with CPU-only scaling. ### Source excerpt Multi-modal autoscaling with Amazon EC2 Auto Scaling combines infrastructure metrics like CPU with application-level signals, so a group scales on the demand its users create. In this post, we show you how to implement it, with code samples and results from a controlled test. ## Planning for disaster recovery using AWS Local Zones and AWS Outposts racks DevFeed: [Planning for disaster recovery using AWS Local Zones and AWS Outposts racks]() Original publisher: [Read original article]() Author: Brianna Rosentrater Published: 2026-09-17T17:24:21Z Content type: article Language: en Sources: [AWS Compute Blog]() Topics: [Disaster Recovery](), [AWS Local Zones](), [AWS Outposts](), [Replication](), [Load Balancing](), [Availability]() Tags: [advanced-300](), [availability](), [aws](), [aws-local-zones](), [aws-outposts](), [aws-outposts-rack](), [database](), [disaster-recovery](), [failover](), [hybrid](), [hybrid-cloud](), [infrastructure](), [latency](), [load-balancing](), [replication](), [technical-how-to]() ### AI overview This post presents disaster recovery architectures using AWS Outposts racks and AWS Local Zones, including DNS-based failover, active/active load balancing, and hybrid database replication. It explains their availability, RTO/RPO, complexity, cost, and data-residency trade-offs. ### Source excerpt Build highly available architectures that span two AWS Outposts racks, or an Outpost rack and an AWS Local Zone, without single points of failure. This post covers three disaster recovery approaches: DNS-based failover, active/active load balancing, and hybrid database replication, with the RTO/RPO trade-offs of each. ## The guest journey, updated in real time: extending Airbnb's sequence recommender with Chronon DevFeed: [The guest journey, updated in real time: extending Airbnb's sequence recommender with Chronon]() Original publisher: [Read original article]() Author: Pengyu Hou Published: 2026-09-17T17:01:02Z Content type: article Language: en Sources: [The Airbnb Tech Blog - Medium]() Topics: [real-time](), [recommendation systems](), [data](), [Transformer](), [Inference](), [Latency](), [Open Source]() Tags: [ai](), [data](), [embeddings](), [engineering](), [inference](), [infrastructure](), [latency](), [machine-learning](), [open-source](), [ranking](), [real-time](), [repo](), [results](), [sequence](), [technology]() ### AI overview Airbnb describes extending its sequence-based recommender with Chronon's Push Mode and Near-real-time Model Transform capabilities. The changes update guest activity and model features more quickly, reducing the staleness of search-ranking inputs compared with the previous daily batch pipeline. ### Source excerpt How two new Chronon capabilities, Push Mode and NRT Model Transform, allows us to provide more relevant search results instantly as a guest explores, rather than waiting for the next batch run. By: Pengyu Hou, Yuli Han, Daochen Zha, Haozhen Ding, Xin Liu, Sophie Wang, Pallavi Adusumilli, Sherry Li, Henry Saputra, Chun How Tan, Huiji Gao, Yan Zhang, Stephanie Moyerman, Yi Li, and Sanjeev Katariya A guest's interaction with Airbnb doesn't pause to wait for a nightly batch job. Someone might browse a dozen listings on a Tuesday afternoon, run a new search that evening, and expect the next search to reflect the recent activity; it's also to Airbnb's benefit for that to be the case. In our previous post, Personalizing Airbnb search by learning from the guest journey, we described how we built a Transformer-based sequence encoder that creates better, more personalized search rankings for a guest using the booking, review, and browsing data that is most relevant to them -- their own. That system ran as a daily batch job: each night it processed the previous day's activity and refreshed embeddings for guests who had something new to show for it. That design worked well, but it left a gap. Activity from earlier the same day wouldn't show up in the embedding until the following day's run, on top of the pipeline's own processing lag -- in practice, up to nearly two days of staleness. For a guest actively planning a trip, that meant the ranking model was often working from a slightly outdated picture of what they wanted, and the recent activities are often highly relevant to current search needs. This is a limitation that our original JourneyFormer research had already flagged as needing new serving infrastructure to solve. In this post, we describe how we closed that gap by adding two new capabilities to Chronon, Airbnb's feature platform: Near-real-time Model Transform and Push Mode. Chronon is an open source project, and these capabilities have been contributed back to our pub ## Implement a correctness-safe Bloom filter lookup with Amazon ElastiCache for Valkey and Amazon Aurora PostgreSQL DevFeed: [Implement a correctness-safe Bloom filter lookup with Amazon ElastiCache for Valkey and Amazon Aurora PostgreSQL]() Original publisher: [Read original article]() Author: Chintan Agrawal Published: 2026-09-17T16:01:43Z Content type: tutorial Language: en Sources: [AWS Database Blog]() Topics: [bloom-filter](), [valkey](), [Amazon Aurora](), [Cache](), [PostgreSQL](), [Caching](), [consistency](), [Latency]() Tags: [advanced-300](), [amazon-aurora](), [amazon-elasticache](), [architecture](), [bloom-filter](), [cache](), [consistency](), [latency](), [performance](), [postgresql](), [postgresql-compatible](), [technical-how-to](), [valkey]() ### AI overview This post explains a three-tier membership lookup that combines a Bloom filter in Amazon ElastiCache for Valkey, an exact-match cache, and Amazon Aurora PostgreSQL as the relational source of truth. The design provides fast negative checks and preserves correctness when false positives would affect business outcomes. ### Source excerpt This post shows how to compose a Bloom filter with an exact-match cache and a relational source of truth into a three-tier, correctness-safe membership lookup using Amazon ElastiCache for Valkey and Amazon Aurora PostgreSQL, serving sub-millisecond decisions at peak throughput without false-positive risk. ## Beyond Two Towers: Launching the 3-Tower Engagement Co-Train Model (Part 2) DevFeed: [Beyond Two Towers: Launching the 3-Tower Engagement Co-Train Model (Part 2)]() Original publisher: [Read original article]() Author: Pinterest Engineering Published: 2026-09-17T15:01:05Z Content type: article Language: en Sources: [Pinterest Engineering Blog - Medium]() Topics: [recommendation systems](), [Embeddings](), [Latency](), [Inference](), [AI, ML & Data Engineering]() Tags: [ads](), [architecture](), [cache](), [embeddings](), [engineering](), [latency](), [machine-learning](), [model](), [monetization](), [pinterest](), [ranking](), [recommendation-system](), [scale](), [two-tower-model]() ### AI overview Pinterest engineers describe a three-tower engagement co-train model for lightweight ads ranking. The model jointly predicts click, good-click, and outbound-click probabilities while balancing richer interactions against latency and cost constraints at large candidate volumes. ### Source excerpt Authors: Longyu Zhao (Staff Machine Learning Engineer), Gwendolyn Zhao (Staff Machine Learning Engineer), Peng Yan (Senior Machine Learning Engineer), Yuanlu Bai (Senior Machine Learning Engineer), Yuan Wang (Senior Machine Learning Engineer), Yao Cheng (Staff Machine Learning Engineer), Ang Xu (Principal Machine Learning Engineer), Zhaohong Han (Manager II, Ads Lightweight Ranking) Introduction Previously¹, we launched the next-generation serving stack for standard ads, which we call Nexus. Nexus decoupled candidate generation from scoring and moved us beyond the classic two-tower-only world, enabling richer model architectures while still meeting stringent latency and cost constraints. Building on this system, we set out to design the first ads lightweight ranking model that goes beyond two towers. It jointly predicts three probabilities for each candidate ad: pCTR, the probability of a click; pGCTR30, the probability of a good click that lasts at least 30 seconds; and pOCTR, the probability of an outbound click to the advertiser's destination. To support these objectives efficiently, we partition the query and Pin embeddings into task-specific CTR, gCTR30, and oCTR segments. For the CTR task, the fast two-tower prediction uses the first 64 dimensions of the CTR segment, while the three-tower prediction uses the full CTR segment together with richer cross features. For gCTR30 and oCTR tasks, full embeddings are shared between two-tower and three-tower predictions. This lets each task learn dedicated representations while sharing the overall model. In principle, Nexus places very few hard constraints on the architecture we can serve: cross-attention, sequence modeling, and more expressive interaction modules are all on the table. However, in practice we quickly ran into the fundamental reality of ads lightweight ranking at Pinterest scale: for a typical request, we need to score on the order of hundreds of thousands of candidates (P99 post-targeting candidate count ## Optimize your team's price-performance with hosted open weight models DevFeed: [Optimize your team's price-performance with hosted open weight models]() Original publisher: [Read original article]() Author: Brittany Lutz Published: 2026-09-17T00:00:00Z Content type: release Language: en Sources: [GitLab]() Topics: [GitLab](), [Artificial Intelligence](), [Agentic development](), [AI Development](), [AI-assisted coding](), [Latency](), [long-context](), [Security]() Tags: [agentic-ai](), [ai](), [coding-agents](), [cost](), [features](), [gitlab](), [latency](), [models](), [open](), [optimize](), [product]() ### AI overview GitLab announces three hosted open weight models for GitLab Duo Agent Platform: Kimi K3, GLM 5.3, and MiniMax M3. The release gives software teams more model choices for balancing quality, latency, and cost across different development workloads. ### Source excerpt There's no single best model for every software development task. Implementing a new feature, diagnosing a failed pipeline, and resolving security vulnerabilities all place different demands on the model handling them. GitLab Duo Agent Platform is expanding GitLab-managed model choice with three hosted open weight models: Kimi K3, GLM 5.3, and MiniMax M3. Together with the frontier models already available in GitLab, your team now has more control over how you optimize for quality, latency, and cost, tuned to the needs of each workload. Point a hard task at Kimi K3 or GLM 5.3, which outperformed comparable frontier models in internal testing and costs less per call, or hand routine, high-volume work to MiniMax M3. Either way, your team gets up to 4x more calls per GitLab Credit than some comparable frontier models, with more AI model options to match cost to task complexity. GitLab Transcend returns in October Coding agents are increasing your speed of development, but your reviews, security policies, and release cycles still have to keep pace. Our Transcend event on October 6 will demonstrate how GitLab is helping teams close that gap and explore what it takes to carry the speed of agentic AI across the software lifecycle. Register for the livestream today! Why one model has been the easy way out Different tasks in agentic development need different things from an AI model. A long-running refactor needs a large context window and deeper reasoning, while a routine, high-volume task is often better served by a faster, more cost-efficient model. Optimizing for one task type means giving up ground on the other. As teams work through the technical tradeoff, whether they can actually access a new model emerges as an additional governance challenge. In regulated environments, every model and the infrastructure it runs on has to clear security, compliance, and internal review before the team can use it. Reviews and approvals move slowly enough that many teams standardize o ## Inline CSS at Khan Academy: Aphrodite DevFeed: [Inline CSS at Khan Academy: Aphrodite]() Original publisher: [Read original article]() Author: Khan Academy Published: 2016-03-29T22:00:00Z Content type: article Language: en Sources: [Khan Academy]() Topics: [React](), [Server-side rendering](), [HTML](), [JavaScript](), [Latency](), [networking](), [Chrome]() Tags: [chrome](), [css](), [engineering](), [infrastructure](), [javascript](), [latency](), [network](), [news](), [react](), [server-side-rendering]() ### AI overview Jamie Wong discusses Aphrodite, Khan Academy's inline styling solution, the constraints that shaped it, and its use in production. The article explains how reducing bytes before content renders motivated server-side rendering and compares the initial-load byte requirements of server-side and client-side rendering. ### Source excerpt By Jamie Wong I won't ramble in this post about the maintenance woe that is CSS, as others ... Read more ## Profiling App Engine Memcached DevFeed: [Profiling App Engine Memcached]() Original publisher: [Read original article]() Author: Khan Academy Published: 2017-05-01T22:00:00Z Content type: article Language: en Sources: [Khan Academy]() Topics: [Latency](), [Logging](), [Data structures]() Tags: [engineering](), [infrastructure](), [latency](), [logging](), [memcached](), [news](), [patterns](), [profiling](), [tests]() ### AI overview Khan Academy engineers profile App Engine Memcached to estimate access latency and evaluate whether on-demand content fetching could support separate content trees for each language. Most requests showed 1-4 ms latency, usually below 10 ms, while a single multi-get performed similarly to a single get. ### Source excerpt By Ben Kraft Last year, Willow wrote about how we optimized our in-memory content data to take up ... Read more ## Google and OpenAI take different approaches to reducing voice-agent latency DevFeed: [Google and OpenAI take different approaches to reducing voice-agent latency]() Original publisher: [Read original article]() Author: Amanda Caswell Published: 2026-09-15T21:50:15Z Content type: comparison Language: en Sources: [The New Stack]() Topics: [Latency](), [Google AI](), [OpenAI](), [API](), [Orchestration](), [real-time](), [Artificial Intelligence]() Tags: [ai](), [ai-agents](), [ai-engineering](), [api](), [comparison](), [google-ai](), [large-language-models](), [latency](), [openai](), [orchestration](), [real-time]() ### AI overview The article compares Google's Gemini 3.8 Live Extended Thinking with OpenAI's GPT-Live-1 for reducing latency in voice agents. Google keeps speech, reasoning, and asynchronous tool execution in one stateful session, while OpenAI uses a real-time conversation model alongside a backend reasoning model, shifting more orchestration to the application. ### Source excerpt Voice agents have a latency problem that shows up as soon as they have to do real work. Within five The post OpenAI's voice model doesn't think. That's the point. appeared first on The New Stack. ## Kubernetes v1.37: Pod-Level Resource Managers graduated to Beta DevFeed: [Kubernetes v1.37: Pod-Level Resource Managers graduated to Beta]() Original publisher: [Read original article]() Author: Kevin Torres Martinez Published: 2026-09-15T18:30:00Z Content type: release Language: en Sources: [Kubernetes Blog]() Topics: [Kubernetes](), [cpu](), [Hardware](), [Latency](), [API](), [gRPC](), [Monitoring]() Tags: [api](), [cpu](), [hardware](), [kubernetes](), [latency](), [monitoring](), [numa](), [release](), [v1]() ### AI overview Kubernetes v1.37 graduates Pod-Level Resource Managers to Beta, disabled by default. The feature lets Kubelet resource managers use pod-level declarations for hardware placement, enabling NUMA-aligned exclusive resources for primary containers while placing sidecars in a shared pod-isolated pool. The release also adds pod-level reporting to the PodResources gRPC API. ### Source excerpt With the release of Kubernetes v1.37, the Pod-Level Resource Managers feature has graduated to Beta status (disabled by default)! First introduced as an Alpha feature in Kubernetes v1.36, this enhancement builds on Pod-Level Resources by equipping Kubelet's Topology Manager, CPU Manager, and Memory Manager to use Pod-level resource declarations (.spec.resources) directly when making hardware placement decisions. Bringing pod-level resources to node managers Before this feature, obtaining exclusive NUMA-aligned CPU cores or memory for latency-critical applications forced cluster operators into an all-or-nothing choice: assign integer resource requests to every container in the Pod, or forfeit exclusive NUMA alignment entirely. For modern workloads running lightweight sidecars (such as logging agents or telemetry exporters), allocating dedicated physical cores to auxiliary containers was wasteful. Pod-Level Resource Managers solves this challenge by enabling hybrid allocation models. The Kubelet can reserve exclusive NUMA-aligned resources for primary application containers while placing non-Guaranteed sidecars into a pod-isolated shared pool. This ensures primary workloads get unthrottled, NUMA-local performance while sidecars benefit from running in a pod-isolated shared pool, enjoying local NUMA alignment and protection from external node interference without consuming dedicated physical cores. What's new in Beta Graduating to Beta brings key operational and API enhancements: Graduation to Beta: Controlled by the PodLevelResourceManagers feature gate, available to opt in (disabled by default) in Kubernetes v1.37. PodResources API Reporting: The v1 PodResources gRPC service (PodResourcesLister) introduces top-level cpu_ids and memory fields on PodResources responses. Monitoring tools and device plugins can query pod-level exclusive assignments directly without double-counting container allocations. Getting started and providing feedback For a deep dive into the tech ## AI Infra Summit: NVIDIA Vera Rubin and DSX Platform Advancements Showcase Energy Efficiencies of Optimizing Tokens Per Watt for AI Factories DevFeed: [AI Infra Summit: NVIDIA Vera Rubin and DSX Platform Advancements Showcase Energy Efficiencies of Optimizing Tokens Per Watt for AI Factories]() Original publisher: [Read original article]() Author: NVIDIA Writers Published: 2026-09-15T16:55:40Z Content type: news Language: en Sources: [NVIDIA Blog]() Topics: [AI Infrastructure](), [Nvidia](), [DSX](), [Vera Rubin](), [Low-Latency Inference](), [NVLink](), [Conversational AI]() Tags: [ai](), [ai-infrastructure](), [dsx](), [inference](), [infra](), [latency](), [low-latency-inference](), [networking](), [nvidia](), [nvidia-blackwell](), [nvidia-dsx](), [nvidia-vera](), [nvidia-vera-rubin](), [nvlink]() ### AI overview NVIDIA's AI Infra Summit coverage describes collaborations and platform updates focused on improving AI factory efficiency. The article highlights Vera Rubin systems, DSX MaxLPS, Dynamo inference software, NVLink and networking technologies, including claims of up to 1.4x more tokens per megawatt through factory-wide power optimization. ### Source excerpt Ian Buck, vice president of hyperscale and high-performance computing at NVIDIA, Tuesday spoke on AI factory efficiency at the AI Infra Summit, the Santa Clara Convention Center event that has morphed into a Coachella of infrastructure tech. Before a packed audience -- with more than 8,000 attendees this year, up from 3,500 last year -- [...] ## How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin DevFeed: [How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin]() Original publisher: [Read original article]() Author: Tanya Lenz Published: 2026-09-15T16:55:00Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [Groq 3 LPX](), [LPX](), [NVIDIA Vera Rubin](), [Vera Rubin NVL72](), [groq](), [Inference](), [Low Latency](), [long-context]() Tags: [agentic-ai-generative-ai](), [ai-factory](), [ai-inference](), [ai-platform](), [drive](), [dsx](), [groq](), [groq-3-lpx](), [inference](), [inference-performance](), [latency](), [low-latency](), [lpx](), [nvidia-vera-rubin](), [nvl72](), [performance](), [power-management](), [vera-rubin](), [vera-rubin-nvl72]() ### AI overview This NVIDIA developer article explains how Groq 3 LPX uses deterministic execution across 256 LPU chips to support low-latency inference on NVIDIA Vera Rubin. It describes compiler-scheduled execution and power-management techniques including Preemptive Power and Clock Period Synthesis. ### Source excerpt Power is a defining constraint for AI factories. As AI workloads demand a full compute platform to serve them, each component of that platform must maximize... ## Optimizing cost and latency with Amazon Bedrock prompt caching DevFeed: [Optimizing cost and latency with Amazon Bedrock prompt caching]() Original publisher: [Read original article]() Author: Daniel Abib Published: 2026-09-15T16:18:19Z Content type: tutorial Language: en Sources: [Artificial Intelligence]() Topics: [Amazon Bedrock](), [Caching](), [Latency](), [API](), [foundation-models](), [LangChain](), [Multi-tenancy]() Tags: [advanced-300](), [amazon-bedrock](), [api](), [caching](), [foundation-models](), [integration](), [langchain](), [latency](), [pricing](), [technical-how-to](), [tokens]() ### AI overview This AWS post explains how Amazon Bedrock prompt caching can reduce repeated input-token costs by up to 90 percent and lower time to first token when requests reuse the same context. It presents six scenarios using the Converse API, including document, system prompt, tool definition, mixed TTL, tenant-isolated, and LangChain caching. ### Source excerpt Prompt caching in Amazon Bedrock can cut input token costs by up to 90% when you repeatedly send the same context to foundation models. This post walks through six practical prompt caching scenarios using the Converse API: message content, system prompt, tool definition, mixed TTL, tenant isolation, and LangChain integration. ## How LLMs Handle Memory Through Context and Surrounding Applications DevFeed: [How LLMs Handle Memory Through Context and Surrounding Applications]() Original publisher: [Read original article]() Author: ByteByteGo Published: 2026-09-15T15:31:12Z Content type: article Language: en Sources: [ByteByteGo]() Topics: [Large Language Model](), [context window](), [App](), [long-context]() Tags: [article](), [context](), [context-window](), [cost](), [latency](), [llms](), [memory]() ### AI overview LLMs do not usually retain personal or persistent memory between interactions. Surrounding applications create the appearance of memory by storing messages, maintaining summaries, retrieving relevant information, and supplying it to the model. As conversations grow, this processing increases cost and latency, while context-window limits require older information to be removed, summarized, or stored elsewhere. ### Source excerpt In this article, we will learn how LLMs handle memory so that they are useful to end users in performing complex tasks that require conversation and holding context. ## Rail Network Modernization: FRMCS, Sovereignty and the AI Edge DevFeed: [Rail Network Modernization: FRMCS, Sovereignty and the AI Edge]() Original publisher: [Read original article]() Author: Steve Payne Published: 2026-09-15T13:31:46Z Content type: article Language: en Sources: [Cisco Blogs]() Topics: [Network](), [Critical Infrastructure](), [digital sovereignty](), [Resilience](), [Availability](), [Cybersecurity](), [Artificial Intelligence]() Tags: [2026](), [availability](), [cisco](), [cisco-industrial-iot-iiot](), [critical-infrastructure](), [digital-sovereignty](), [end-of-life](), [eu](), [europe](), [industries](), [infrastructure](), [internet-of-things-iot](), [latency](), [modernization](), [network](), [networks](), [nis2](), [resilience](), [systems](), [transportation](), [unified-edge]() ### AI overview Cisco's pre-InnoTrans 2026 perspective examines the modernization of European rail communications from GSM-R to FRMCS. It argues that the transition requires changes to the underlying IP network, including high availability, deterministic latency, cybersecurity, and large-scale automation, while operators address resilience, digital sovereignty, investment, and regulatory requirements. ### Source excerpt Join Cisco at InnoTrans 2026 to explore the next generation of rail networks. Discover how FRMCS, AI edge computing, and digital sovereignty are transforming critical infrastructure into secure, automated systems. ## Why your Redis is slow: it is single threaded and you sent it KEYS DevFeed: [Why your Redis is slow: it is single threaded and you sent it KEYS]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-09-15T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Redis](), [Latency](), [Concurrency]() Tags: [blocking](), [caching](), [command](), [latency](), [observability](), [performance](), [redis](), [single-threaded]() ### AI overview The article explains that Redis executes commands one at a time on a single thread, so expensive commands can block every client and cause latency spikes across all operations. It identifies KEYS as a key offender and also discusses large DEL operations, unbounded LRANGE, SMEMBERS on large sets, and FLUSHALL. Redis 6 threaded I/O helps with socket handling but does not change single-threaded command execution. ### Source excerpt Redis executes commands one at a time on a single thread, so any command that takes 200ms makes every other client wait 200ms regardless of how trivial their request was. KEYS scans the entire keyspace and is the obvious offender, but DEL on a large collection, unbounded LRANGE, SMEMBERS on a big set and FLUSHALL are all linear in the size of the data and block identically. The tell is a latency spike across all operations at once while CPU and network look normal. ## How to operate shared platforms safely at agent scale DevFeed: [How to operate shared platforms safely at agent scale]() Original publisher: [Read original article]() Author: Candace Shamieh; T Zhang; Gabriele Baldoni Published: 2026-09-15T00:00:00Z Content type: article Language: en Sources: [Datadog | The Monitor blog]() Topics: [Platform Engineering](), [Loop Engineering](), [AI Agent](), [sdlc](), [Latency](), [Low Latency](), [Concurrency](), [Software]() Tags: [agent](), [agent-observability](), [agentic-workflows](), [agents](), [ai](), [ai-agent](), [ci](), [concurrency](), [latency](), [low-latency](), [operational](), [performance](), [platform-engineering](), [platforms](), [queue](), [recovery](), [timeout]() ### AI overview This Datadog article explains how platform teams can operate shared platforms safely as AI agent workloads scale across teams. It discusses modeling demand across agent trajectories, planning capacity across dependencies such as CI queues and sandbox pools, handling contention and recovery behavior, and preserving control across system boundaries. ### Source excerpt Learn how Datadog models agent demand, allocates capacity under contention, and preserves control as AI agent workloads scale across shared platforms. ## And the winners are: Announcing the results of the OpenSearch Agent Skills Hackathon DevFeed: [And the winners are: Announcing the results of the OpenSearch Agent Skills Hackathon]() Original publisher: [Read original article]() Author: James McIntyre Published: 2026-09-14T23:00:09Z Content type: article Language: en Sources: [OpenSearch]() Topics: [Agent Skills](), [Hackathon](), [Code]() Tags: [agent-skills](), [audit](), [blog](), [github](), [hackathon](), [latency](), [opensearch](), [root-cause-analysis](), [technical]() ### AI overview This OpenSearch article announces the winners of the OpenSearch Agent Skills Hackathon. It describes the competition requirements and judging criteria, then highlights the first-place Unclosed skill, which performs auditable log root-cause analysis through premise audits, hypothesis trees, and closure checks. ### Source excerpt Meet the three winners of the OpenSearch Agent Skills Hackathon. Their skills tackle log root-cause analysis, GDPR compliance, and slow-query diagnostics, all built read-only, fully auditable, and shipped with real evaluation. The post And the winners are: Announcing the results of the OpenSearch Agent Skills Hackathon appeared first on OpenSearch. [Next page]()