# performance-engineering

Performance engineering is the practice of optimizing software and IT systems throughout the development lifecycle to meet performance requirements such as throughput, latency, reliability, and efficiency.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Uber Cuts Translation Cache Memory by up to 90% While Keeping P99 Latency Below 1 MS

DevFeed: [Uber Cuts Translation Cache Memory by up to 90% While Keeping P99 Latency Below 1 MS](<https://devfeed.tech/articles/uber-cuts-translation-cache-memory-by-up-to-90-while-keeping-p99-latency-below-1-ms-146238.md>)

Original publisher: [Read original article](<https://www.infoq.com/news/2026/11/uber-go-translation-cache/>)

Author: Leela Kumili

Published: 2026-11-20T14:38:00Z

Content type: news

Language: en

Sources: [InfoQ](<https://devfeed.tech/sources/infoq.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [memory handling](<https://devfeed.tech/topics/memory-handling.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [cost-optimization](<https://devfeed.tech/topics/cost-optimization.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [architecture-design](<https://devfeed.tech/tags/architecture-design.md>), [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [development](<https://devfeed.tech/tags/development.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [garbage-collection](<https://devfeed.tech/tags/garbage-collection.md>), [garbagecollection](<https://devfeed.tech/tags/garbagecollection.md>), [go](<https://devfeed.tech/tags/go.md>), [go-language](<https://devfeed.tech/tags/go-language.md>), [hit](<https://devfeed.tech/tags/hit.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [hybrid-architecture](<https://devfeed.tech/tags/hybrid-architecture.md>), [hybrid-work](<https://devfeed.tech/tags/hybrid-work.md>), [keys](<https://devfeed.tech/tags/keys.md>), [latency](<https://devfeed.tech/tags/latency.md>), [low-latency](<https://devfeed.tech/tags/low-latency.md>), [memory](<https://devfeed.tech/tags/memory.md>), [news](<https://devfeed.tech/tags/news.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [p99](<https://devfeed.tech/tags/p99.md>), [performance-tuning](<https://devfeed.tech/tags/performance-tuning.md>), [reduce](<https://devfeed.tech/tags/reduce.md>), [storage](<https://devfeed.tech/tags/storage.md>), [translation](<https://devfeed.tech/tags/translation.md>), [uber](<https://devfeed.tech/tags/uber.md>), [uber-go-translation-cache](<https://devfeed.tech/tags/uber-go-translation-cache.md>), [values](<https://devfeed.tech/tags/values.md>)

### AI overview

Uber reduced memory use in its Go translation cache by keeping keys and frequently accessed values in memory while storing other values in disk-backed files. Replacing sync.Map with a map protected by an RWMutex cut object allocations and improved CPU utilization by more than 30% in an affected service, but did not address dataset size or refreshes that temporarily held two full caches. The hybrid design shards storage across 256 locks and files, uses compact hash-based keys with collision fallback, and preloads recently used entries during refresh. Reported results include memory falling from about 2.7 GB to 750 MB, cache hit rates above 99.9%, and p99 disk-backed miss latency below 100 microseconds. In one workload, around 30,000 of more than four million entries were in memory.

### Source excerpt

Uber redesigned its Go-based translation cache to reduce its initial memory footprint by more than 90% while maintaining a cache hit rate above 99.9% and p99 disk-backed cache miss latency below 100 microseconds. The hybrid architecture combines in-memory keys and frequently accessed values with disk-backed storage, reducing garbage-collection overhead and memory consumption. By Leela Kumili

## Intent-Defined Radio: A Goal-Native Architecture for Next-Generation Wireless Networks

DevFeed: [Intent-Defined Radio: A Goal-Native Architecture for Next-Generation Wireless Networks](<https://devfeed.tech/articles/intent-defined-radio-a-goal-native-architecture-for-next-generation-wireless-networks-146146.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25657>)

Author: Wencheng Yan (Dept. of Electronic Engineering & BNRist, Tsinghua University, Beijing, China), Binrui Xu (Dept. of Electronic Engineering & BNRist, Tsinghua University, Beijing, China), Yihao Shen (Sch

Published: 2026-11-20T05:00:00Z

Content type: opinion

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [networking news](<https://devfeed.tech/topics/networking-news.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>)

Tags: [abstraction](<https://devfeed.tech/tags/abstraction.md>), [arbitration](<https://devfeed.tech/tags/arbitration.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [control](<https://devfeed.tech/tags/control.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-it](<https://devfeed.tech/tags/cs-it.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [design](<https://devfeed.tech/tags/design.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [goals](<https://devfeed.tech/tags/goals.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [intent-based-networking](<https://devfeed.tech/tags/intent-based-networking.md>), [interoperability](<https://devfeed.tech/tags/interoperability.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [math-it](<https://devfeed.tech/tags/math-it.md>), [networks](<https://devfeed.tech/tags/networks.md>), [radio](<https://devfeed.tech/tags/radio.md>), [wireless](<https://devfeed.tech/tags/wireless.md>)

### AI overview

Intent-Defined Radio (IDR) uses large language models to connect explicit service goals such as latency, reliability, throughput, and continuity to executable radio behavior within a constrained design space. The abstract describes Intent2MAC simulations in which MAC behavior is synthesized and refined offline using evaluation feedback. One study improved service satisfaction at a throughput cost; another improved remote-control performance by replacing random contention with distributed urgency-based arbitration. The authors identify open challenges in intent specification, evidence, hardware execution, interoperability, and lifecycle cost, and propose a staged path toward deployment.

### Source excerpt

arXiv:2611.25657v1 Announce Type: new Abstract: Wireless systems are becoming harder to operate because service goals are increasingly expressed in outcome-oriented terms, while radio platforms still expose mainly low-level configurations and protocol-specific controls. This mismatch makes it difficult to translate latency, reliability, throughput, and continuity requirements into coherent radio behavior. This article argues for Intent-Defined Radio (IDR) as a new system abstraction for next-generation wireless networks. Rather than treating the radio stack as a fixed mechanism set to be manually engineered and then tuned, IDR connects explicit service goals to executable radio behavior within a constrained design space by leveraging the capacity of large language models (LLMs). We position IDR relative to programmable radio, intent-based networking, and learning-based protocol design. We also present two simulation studies using Intent2MAC to illustrate the design loop, demonstrating executable MAC synthesis and offline refinement guided by evaluation feedback. The results show improved service satisfaction at a throughput cost and improved remote-control performance when random contention is replaced by distributed urgency-based arbitration. We finally discuss the open challenges in intent specification, evidence, hardware execution, interoperability, and lifecycle cost, and outline a staged path toward deployment.

## Aggron: Aggressive Compression of Agentic KV Caches during Head-of-Line Stalls in the Request Queue

DevFeed: [Aggron: Aggressive Compression of Agentic KV Caches during Head-of-Line Stalls in the Request Queue](<https://devfeed.tech/articles/aggron-aggressive-compression-of-agentic-kv-caches-during-head-of-line-stalls-in-the-request-queue-146134.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26435>)

Author: Rishabh Ravi, Tony Hong, Chen Wang, Yue Zhu, Mengmei Ye, Hubertus Franke, Fan Lai, Josep Torrellas

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Compression](<https://devfeed.tech/topics/compression.md>), [queueing theory](<https://devfeed.tech/topics/queueing-theory.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [vllm](<https://devfeed.tech/topics/vllm.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [cache](<https://devfeed.tech/tags/cache.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [compression](<https://devfeed.tech/tags/compression.md>), [context-compression](<https://devfeed.tech/tags/context-compression.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [engine](<https://devfeed.tech/tags/engine.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm](<https://devfeed.tech/tags/llm.md>), [load](<https://devfeed.tech/tags/load.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [queue](<https://devfeed.tech/tags/queue.md>), [request](<https://devfeed.tech/tags/request.md>), [serving](<https://devfeed.tech/tags/serving.md>), [stalled](<https://devfeed.tech/tags/stalled.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

Aggron addresses KV-cache limits in long-running LLM agents by summarizing selected cache content at every turn, keeping each summarization request lightweight. It runs these requests in a separate queue while the corresponding agent request is stalled behind a head-of-line block, using capacity left idle in the main queue; it launches summarization only when it predicts completion before that agent request executes. The abstract says this approach retains accuracy and, versus state-of-the-art asynchronous summarization at high load, reduces P99 program latency by 27-31% and increases throughput by 1.34-1.40x. The supplied text is an abstract, so it does not provide evaluation details or broader comparisons.

### Source excerpt

arXiv:2611.26435v1 Announce Type: new Abstract: LLM agents run long, multi-turn programs whose contexts grow at every step, and serving them is limited by KV cache capacity. To alleviate this limitation, systems use context compression techniques, which typically wait until the KV cache crosses a large threshold and then summarize large spans of history at once, either in the background while the agent progresses or on the critical path. These summarizations can be done using the same model running on the same engine, or using the same or a different model running on extra hardware. When running on the same engine, summarization competes with the requests being served for both compute and memory capacity. To maximize the cost-effectiveness of summarization, this paper presents a new serving system called {\em Aggron}. Aggron summarizes agentic requests at every turn and selectively summarizes only certain parts of the KV cache. As a result, Aggron retains accuracy and keeps summarization requests lightweight. Moreover, Aggron launches summarization requests from a separate {\em Summarization Queue} while the corresponding agentic request whose KV cache we want to compress is stalled in the main request queue due to head-of-line stalls. The Summarization queue utilizes compute and memory capacity left over by the head-of-line stalls in the main queue. Further, to maximize efficiency, the server issues summarizations only when it predicts that they can complete before the corresponding agentic request will finally execute. In our evaluation, we compare Aggron to state-of-the-art asynchronous summarization. At high load, Aggron reduces P99 program latency by $27$--$31\%$, and raises throughput by $1.34$--$1.40\times$.

## Amazon SageMaker AI now detects latency drift, idle GPUs, and missed optimizations on your inference endpoints

DevFeed: [Amazon SageMaker AI now detects latency drift, idle GPUs, and missed optimizations on your inference endpoints](<https://devfeed.tech/articles/amazon-sagemaker-ai-now-detects-latency-drift-idle-gpus-and-missed-optimizations-on-your-inference-endpoints-144471.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-ai-now-detects-latency-drift-idle-gpus-and-missed-optimizations-on-your-inference-endpoints/>)

Author: Hazim Qudah

Published: 2026-11-19T21:28:18Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [gpu monitoring](<https://devfeed.tech/topics/gpu-monitoring.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [vllm](<https://devfeed.tech/topics/vllm.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [amazon-sagemaker-studio](<https://devfeed.tech/tags/amazon-sagemaker-studio.md>), [announcements](<https://devfeed.tech/tags/announcements.md>), [aws-user-notifications](<https://devfeed.tech/tags/aws-user-notifications.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [drift](<https://devfeed.tech/tags/drift.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [gpu-utilization](<https://devfeed.tech/tags/gpu-utilization.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [latency](<https://devfeed.tech/tags/latency.md>), [optimizations](<https://devfeed.tech/tags/optimizations.md>), [performance](<https://devfeed.tech/tags/performance.md>), [serving](<https://devfeed.tech/tags/serving.md>), [studio](<https://devfeed.tech/tags/studio.md>)

### AI overview

Amazon SageMaker AI monitors eligible generative AI inference endpoints by default when they use vLLM serving containers and GPU instances. Using aggregated latency, GPU utilization, and serving configuration signals, it can flag week-over-week latency drift, mostly idle GPUs, or supported optimizations and updates that are missing. Monitoring does not read request or response content and has no GPU cost. Findings arrive through AWS Health and email; users can open them in SageMaker Studio and benchmark alternatives on separate compute, comparing response time, throughput, and cost per million tokens. No endpoint change occurs until the user chooses to apply a result, and keeping the current configuration is also an option. Benchmarks incur standard SageMaker charges or use an ML reservation. Monitoring can be disabled by setting OPT_OUT_TRACKING on a model container; doing so prevents telemetry collection and notifications for that endpoint.

### Source excerpt

Amazon SageMaker AI now monitors your generative AI inference endpoints by default and tells you when a change could improve performance or lower cost. It tracks latency, GPU utilization, and serving configuration, then sends a finding through AWS Health with a one-click path to benchmark alternatives in SageMaker Studio before you apply any change.

## Performance impact of excessive logging in Amazon Aurora PostgreSQL

DevFeed: [Performance impact of excessive logging in Amazon Aurora PostgreSQL](<https://devfeed.tech/articles/performance-impact-of-excessive-logging-in-amazon-aurora-postgresql-144312.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/database/performance-impact-of-excessive-logging-in-amazon-aurora-postgresql/>)

Author: Baji Shaik

Published: 2026-11-19T19:10:08Z

Content type: tutorial

Language: en

Sources: [AWS Database Blog](<https://devfeed.tech/sources/aws-database-blog.md>)

Topics: [Amazon Aurora](<https://devfeed.tech/topics/amazon-aurora.md>), [Logging](<https://devfeed.tech/topics/logging.md>), [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>), [RDS for PostgreSQL](<https://devfeed.tech/topics/rds-for-postgresql.md>), [database-optimization](<https://devfeed.tech/topics/database-optimization.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Amazon RDS](<https://devfeed.tech/topics/amazon-rds.md>), [Amazon CloudWatch](<https://devfeed.tech/topics/amazon-cloudwatch.md>)

Tags: [amazon-aurora](<https://devfeed.tech/tags/amazon-aurora.md>), [amazon-rds](<https://devfeed.tech/tags/amazon-rds.md>), [audit-logging](<https://devfeed.tech/tags/audit-logging.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [logging](<https://devfeed.tech/tags/logging.md>), [performance](<https://devfeed.tech/tags/performance.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [rds-for-postgresql](<https://devfeed.tech/tags/rds-for-postgresql.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

In a pgbench test on Aurora PostgreSQL 17, verbose logging had little effect at 8 clients, but at 800 clients, logging every statement reduced throughput by about 10%; adding `pgaudit.log = ALL` cut whole-run throughput from 16,202 to 5,039 TPS. Sampling showed sessions mostly active on CPU rather than waiting on instrumented resources, and a 537 MB log file written in under a minute supported a bottleneck in log generation and writing. The article recommends checking `pg_settings`, correlating slow periods with `pg_stat_activity` wait events and RDS log-file sizes, and testing at production-like concurrency. For audit needs, it suggests narrowing logged statement classes or using `pgaudit.log_statement = off` to retain structured audit details while dropping embedded SQL text; it advises confirming the reduction on the workload.

### Source excerpt

Excessive database and audit logging can quietly become a performance bottleneck in Amazon Aurora and Amazon RDS for PostgreSQL. Using a reproducible pgbench example, this post shows when logging overhead is negligible and when it cuts throughput by about three times, how to spot it with wait events and log file evidence, and how to reduce it.

## How OpenAI's Performance Factory keeps ChatGPT fast

DevFeed: [How OpenAI's Performance Factory keeps ChatGPT fast](<https://devfeed.tech/articles/how-openai-s-performance-factory-keeps-chatgpt-fast-145835.md>)

Original publisher: [Read original article](<https://openai.com/index/how-openai-performance-factory-keeps-chatgpt-fast>)

Published: 2026-11-19T08:00:00Z

Content type: article

Language: en

Sources: [OpenAI News](<https://devfeed.tech/sources/openai-news.md>)

Topics: [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [codex](<https://devfeed.tech/topics/codex.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Production Engineering](<https://devfeed.tech/topics/production-engineering.md>), [Code review](<https://devfeed.tech/topics/code-review.md>), [ChatGPT](<https://devfeed.tech/topics/chatgpt.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [codex](<https://devfeed.tech/tags/codex.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [factory](<https://devfeed.tech/tags/factory.md>), [fast](<https://devfeed.tech/tags/fast.md>), [openai](<https://devfeed.tech/tags/openai.md>), [performance](<https://devfeed.tech/tags/performance.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

### AI overview

OpenAI's Performance Factory automates performance regression work from alert to post-deployment verification, while keeping engineers involved in reviewing and merging fixes. A cloud dispatch system assigns Codex agents to tasks that preserve findings across steps and can resume after events such as code review or deployment. Agents use the `perf-forensics` skill and production telemetry to locate when slowdowns began, identify affected requests, examine changes, and propose fixes. Separate Codex reviews challenge diagnoses and fixes; engineers make decisions that require service context or tradeoff judgment. In a manual review of 100 root-cause suggestions, 70% were accurate, while 30% needed refinement or did not fully establish causality. In just over three months, the workflow tracked more than 1,700 investigations and root-caused over 600 cases. OpenAI also backtests investigations and task summaries, monitors agent tool failures, and is exploring proactive detection and agent-implemented low-risk fixes.

### Source excerpt

Learn how OpenAI built the Performance Factory, a system of Codex-powered workflows that finds and fixes performance issues to help keep ChatGPT fast.

## Optimize the Cob, Then the Kernel: Optimizing Beam Search for Generative Recommendation Inference

DevFeed: [Optimize the Cob, Then the Kernel: Optimizing Beam Search for Generative Recommendation Inference](<https://devfeed.tech/articles/optimize-the-cob-then-the-kernel-optimizing-beam-search-for-generative-recommendation-inference-142637.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23339>)

Author: Shing Wai Pun, Kai Wu, Pengfei Xia, Hui Zang, Wei Guo, Yong Liu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [ascend](<https://devfeed.tech/tags/ascend.md>), [attention](<https://devfeed.tech/tags/attention.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [beam-search](<https://devfeed.tech/tags/beam-search.md>), [cache](<https://devfeed.tech/tags/cache.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [custom](<https://devfeed.tech/tags/custom.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [dispatch](<https://devfeed.tech/tags/dispatch.md>), [engine](<https://devfeed.tech/tags/engine.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generative](<https://devfeed.tech/tags/generative.md>), [graph](<https://devfeed.tech/tags/graph.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [optimize](<https://devfeed.tech/tags/optimize.md>), [optimizing](<https://devfeed.tech/tags/optimizing.md>), [recommendation](<https://devfeed.tech/tags/recommendation.md>)

### AI overview

Generative recommendation inference combines long prompts, wide beam searches, and few output tokens, which can make conventional LLM serving inefficient. The abstract says vLLM treats beams as independent sequences, repeating KV cache accesses over the shared prompt, and invokes the engine at each generation step, adding host-side processing and PagedAttention block-management overhead. The proposed Ascend NPU pipeline concatenates beams with Tree Attention and adds pre-generated partial-attention masks, static KV allocation, loop-unrolled dispatch, TorchAir graph compilation, operator fusion, and vocabulary-constrained decoding. Without custom kernels, it reports up to 25x lower end-to-end latency than vLLM and generation latency comparable to a state-of-the-art custom-kernel solution. These results are stated for Ascend NPUs.

### Source excerpt

arXiv:2611.23339v1 Announce Type: cross Abstract: Generative recommendation (GR) inference differs substantially from conventional LLM serving, featuring long input sequences, wide beam search, and only a few fixed output tokens. Existing LLM serving engines such as vLLM process beams as batched independent sequences, causing redundant KV cache accesses over the shared prompt prefix. Furthermore, vLLM executes each beam search generation step as a separate engine invocation, incurring host-side beam processing overhead between decoding steps and substantial runtime KV block management in PagedAttention. Using Tree Attention, we concatenate beams along the sequence dimension to eliminate redundant KV accesses, and further optimize execution through pre-generated partial-attention masks, static KV cache allocation, loop-unrolled generation dispatch, TorchAir graph compilation, automatic operator fusion, and vocabulary-constrained decoding. Without the use of custom kernels, our approach reduces end-to-end latency by up to 25x over vLLM on Ascend NPUs, and achieves generation latency comparable to a SOTA solution using custom kernels, showing that the generation pipeline should be optimized holistically before resorting to custom kernel development.

## Continuum: Boundary-State-Aware Deterministic Scheduling in Dynamic LEO Networks

DevFeed: [Continuum: Boundary-State-Aware Deterministic Scheduling in Dynamic LEO Networks](<https://devfeed.tech/articles/continuum-boundary-state-aware-deterministic-scheduling-in-dynamic-leo-networks-142998.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24021>)

Author: Siqi Yang, Zonghui Li, Chaoqun You, Yue Gao

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [scheduling](<https://devfeed.tech/topics/scheduling.md>), [real-time](<https://devfeed.tech/topics/real-time.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>)

Tags: [boundary](<https://devfeed.tech/tags/boundary.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [framework](<https://devfeed.tech/tags/framework.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [inheritance](<https://devfeed.tech/tags/inheritance.md>), [jitter](<https://devfeed.tech/tags/jitter.md>), [latency](<https://devfeed.tech/tags/latency.md>), [leo](<https://devfeed.tech/tags/leo.md>), [low-earth-orbit](<https://devfeed.tech/tags/low-earth-orbit.md>), [mission](<https://devfeed.tech/tags/mission.md>), [networks](<https://devfeed.tech/tags/networks.md>), [node](<https://devfeed.tech/tags/node.md>), [optimizing](<https://devfeed.tech/tags/optimizing.md>), [performance](<https://devfeed.tech/tags/performance.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [timing](<https://devfeed.tech/tags/timing.md>), [topology](<https://devfeed.tech/tags/topology.md>), [traffic](<https://devfeed.tech/tags/traffic.md>), [transmission](<https://devfeed.tech/tags/transmission.md>)

### AI overview

Continuum uses boundary-state-aware rolling-horizon scheduling for time-triggered traffic in dynamic LEO networks. At topology transitions, it carries forward buffered-node and arrival-time information, then jointly optimizes remaining routes and transmission slots. The approach refines latency to improve timing consistency while preserving original timing constraints. In ablations, boundary-state inheritance improved schedulability by up to 16.6 percentage points, while joint alignment reduced mean jitter by 10.9-37.7 times. The abstract also reports scalability experiments on workloads of up to 10,000 flows, but gives no synthesis performance results.

### Source excerpt

arXiv:2611.24021v1 Announce Type: new Abstract: Low Earth Orbit (LEO) constellations support mission-critical services requiring bounded latency and predictable delivery. However, conventional snapshot-based schedulers discard the forwarding progress of unfinished time-triggered traffic across topology transitions, degrading schedulability and timing consistency. We present Continuum, aboundary-state-aware rolling-horizon scheduling framework. Continuum inherits buffered-node and arrival-time information, jointly optimizing remaining routes and transmission slots, and refines latency to improve timing consistency while preserving original timing constraints. Ablations show up to 16.6 percentage points higher schedulability from boundary-state inheritance and a 10.9-37.7 times reduction in mean jitter from joint alignment. Scalability experiments evaluate synthesis performance for workloads of up to 10K flows.

## Quantifying the Intrusiveness of Retrofitting Retrieval-Augmented Generation onto Legacy Software Systems

DevFeed: [Quantifying the Intrusiveness of Retrofitting Retrieval-Augmented Generation onto Legacy Software Systems](<https://devfeed.tech/articles/quantifying-the-intrusiveness-of-retrofitting-retrieval-augmented-generation-onto-legacy-software-systems-142202.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23931>)

Author: K. M. Tahlil Mahfuz Faruk, Rakib Abdullah

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Legacy Modernization](<https://devfeed.tech/topics/legacy-modernization.md>), [legacy systems](<https://devfeed.tech/topics/legacy-systems.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [software-architecture](<https://devfeed.tech/topics/software-architecture.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [backends](<https://devfeed.tech/tags/backends.md>), [call](<https://devfeed.tech/tags/call.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [churn](<https://devfeed.tech/tags/churn.md>), [code](<https://devfeed.tech/tags/code.md>), [commits](<https://devfeed.tech/tags/commits.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [delay](<https://devfeed.tech/tags/delay.md>), [dependency](<https://devfeed.tech/tags/dependency.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [generation](<https://devfeed.tech/tags/generation.md>), [legacy](<https://devfeed.tech/tags/legacy.md>), [legacy-software](<https://devfeed.tech/tags/legacy-software.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [retrieval-augmented-generation](<https://devfeed.tech/tags/retrieval-augmented-generation.md>), [software](<https://devfeed.tech/tags/software.md>), [software-systems](<https://devfeed.tech/tags/software-systems.md>), [systems](<https://devfeed.tech/tags/systems.md>)

### AI overview

The paper proposes three ways to add retrieval-augmented generation (RAG) to legacy software--sidecar, SDK-embed and naive-invasive--and a Legacy Intrusion Score (LIS) for comparing code changes, extended by LIS-X to cover operational effects. Across five systems, the code-level ranking is sidecar < SDK-embed < naive-invasive, though the authors say this follows from the metrics' component-wise dominance and validates the score rather than testing a hypothesis. In a live study on one host and one system, sidecar showed no detectable endpoint latency increase; SDK-embed and naive-invasive added 569 ms and 475 ms, with no significant difference. Both passed a 2-second dependency delay through to legacy requests, and RAG failure could leave successful responses without the AI field. RAG load slowed the legacy service on a shared host, while sidecar data staleness averaged 911 ms at a 2-second polling interval. These operational findings are limited to that single setup.

### Source excerpt

arXiv:2611.23931v1 Announce Type: new Abstract: Enterprises increasingly add retrieval-augmented generation (RAG) to existing REST and monolithic systems, usually by writing an ad hoc HTTP call into a handler, and have no way to quantify how intrusive that choice is. We present (1) a taxonomy of three retrofit patterns, sidecar, SDK-embed and naive-invasive; (2) the Legacy Intrusion Score (LIS), three code sub-metrics (churn, breadth, coupling), and an extension, LIS-X, with three operational sub-metrics measured on a live system (run-time overhead, fault propagation and silent feature degradation); (3) a static evaluation on five legacy systems, two of them real open-source codebases fetched from their repositories at pinned commits; and (4) a live case study that deploys all three patterns as separate OS processes. The code-level ranking sidecar < SDK-embed < naive-invasive holds in all five systems and under 2,000 random weightings, but it follows from component-wise dominance of the sub-metrics, so it validates the metric rather than testing a hypothesis. In the live study, outcomes are not fixed by definition. The sidecar adds no detectable latency to the legacy endpoint (0.07 ms, 95% CI [-0.10, 0.24]), whereas SDK-embed and naive-invasive add 569 and 475 ms and are not distinguishable from each other (p=0.62). Both pass a 2 s dependency delay on to legacy requests, and when the RAG layer fails fast they keep returning successful responses that no longer contain the AI field. Load on the RAG layer slows the legacy service by 1.51-1.57x on a shared host, and sidecar staleness averages 911 ms with a 2 s polling interval. Retrieval hit@1 is 0.95; three hosted LLMs answer all 12 test questions correctly and a local model 9/12, and the four backends are swapped by changing one request field. All dynamic results come from one host and one system.

## DispersedBFT: Bottleneck-Free Primary-Backup Consensus using Dispersed Replication

DevFeed: [DispersedBFT: Bottleneck-Free Primary-Backup Consensus using Dispersed Replication](<https://devfeed.tech/articles/dispersedbft-bottleneck-free-primary-backup-consensus-using-dispersed-replication-142290.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24833>)

Author: Nancy Kansal, Celine Sana, Jelle Hellings

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [consensus algorithm](<https://devfeed.tech/topics/consensus-algorithm.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Blockchain](<https://devfeed.tech/topics/blockchain.md>)

Tags: [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [consensus](<https://devfeed.tech/tags/consensus.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [replication](<https://devfeed.tech/tags/replication.md>)

### AI overview

DispersedBFT aims to remove the primary bottleneck in PBFT-like consensus by using information dispersal to distribute request replication from the primary to backup replicas. Its variants trade CPU cost against communication cost: one uses Message Authentication Codes and Merkle proofs for lower CPU cost, while another uses digital signatures for lower communication cost. The abstract reports throughput of 3.4 million transactions per second with 43 replicas, up to 6.2 times faster than PBFT and 2.6 times faster than HotStuff. These are reported experimental results; the supplied text does not provide evaluation details.

### Source excerpt

arXiv:2611.24833v1 Announce Type: new Abstract: The advent of blockchains is fueling the development of novel resilient data management systems (RDSs). Typically, these RDSs are managed using a primary-backup design using a consensus protocol such as PBFT. In these primary-backup designs, a single participant, the primary, is responsible for coordinating the replication and execution of requests. Unfortunately, the central role of the primary makes it a bottleneck that drastically limits performance. In this paper, we propose DispersedBFT, a high performance PBFT-like consensus protocol that utilizes dispersed replication to eliminate the primary bottleneck of PBFT-like consensus. In specific, DispersedBFT uses information dispersal to distribute the task of replicating request from the primary to the backups. We propose several variants of DispersedBFT with either low CPU cost (by using Message Authentication Codes and Merkle proofs) or communication cost (by using digital signatures). We practically analyze the performance of DispersedBFT and show it greatly outperforms PBFT and HotStuff: DispersedBFT supports a throughput of 3.4Mtxn/sec with 43 replicas and is up-to 6.2x faster than PBFT and 2.6x faster than HotStuff.

## Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

DevFeed: [Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips](<https://devfeed.tech/articles/exa-boosts-embedding-throughput-by-50-to-index-billions-of-documents-on-aws-ai-chips-141408.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/exa-boosts-embedding-throughput-by-50-to-index-billions-of-documents-on-aws-ai-chips/>)

Author: Summer Devlin

Published: 2026-11-18T23:58:22Z

Content type: news

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [AWS AI chips](<https://devfeed.tech/topics/aws-ai-chips.md>), [neuron](<https://devfeed.tech/topics/neuron.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [search-engines](<https://devfeed.tech/topics/search-engines.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-ai-chips](<https://devfeed.tech/tags/aws-ai-chips.md>), [aws-inferentia](<https://devfeed.tech/tags/aws-inferentia.md>), [aws-neuron](<https://devfeed.tech/tags/aws-neuron.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [cost](<https://devfeed.tech/tags/cost.md>), [custom](<https://devfeed.tech/tags/custom.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [database](<https://devfeed.tech/tags/database.md>), [economics](<https://devfeed.tech/tags/economics.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [index](<https://devfeed.tech/tags/index.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [performance](<https://devfeed.tech/tags/performance.md>), [search](<https://devfeed.tech/tags/search.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [unit-economics](<https://devfeed.tech/tags/unit-economics.md>), [vector](<https://devfeed.tech/tags/vector.md>)

### AI overview

Exa reports a 47% improvement in performance and cost per million documents embedded on AWS Neuron hardware versus its initial baseline. Starting with FP16 data parallelism on Inferentia2, it optimized weight layout and precision, used a drop-in attention kernel, and built a fused SwiGLU MLP kernel to reduce memory spill. Custom attention kernels tailored to the model's shapes then cut padding work and instruction steps. Each change passed a correctness gate of 0.999 cosine similarity to the reference; performance was measured with a 30-minute saturation benchmark. The workload uses one NeuronCore per pod, and the same approach scales through data parallelism on Trainium2. Exa says tuning across hardware and Regions helps its scheduler match shifting demand and availability. The excerpt describes these results and methods but does not provide absolute throughput or cost figures.

### Source excerpt

Exa's neural search engine is backed by an in-house vector database, so the unit economics of embedding tens of billions of web documents drive the business. This post shares how Exa optimized embedding inference on AWS Neuron hardware (Inferentia2 and Trainium2) with custom NKI kernels, reaching a 47% gain in performance and cost per million documents.

## Thoughts on Verification Engineering in the Agentic SDLC

DevFeed: [Thoughts on Verification Engineering in the Agentic SDLC](<https://devfeed.tech/articles/thoughts-on-verification-engineering-in-the-agentic-sdlc-142073.md>)

Original publisher: [Read original article](<https://www.dataengineeringweekly.com/p/thoughts-on-verification-engineering>)

Author: Ananth Packkildurai

Published: 2026-11-18T21:59:55Z

Content type: article

Language: en

Sources: [Data Engineering Weekly](<https://devfeed.tech/sources/data-engineering-weekly.md>)

Topics: [Software Engineering](<https://devfeed.tech/topics/software-engineering.md>), [reliability-engineering](<https://devfeed.tech/topics/reliability-engineering.md>), [observability](<https://devfeed.tech/topics/observability.md>), [Spec Driven Development](<https://devfeed.tech/topics/spec-driven-development.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-sdlc](<https://devfeed.tech/tags/agentic-sdlc.md>), [agentic-software](<https://devfeed.tech/tags/agentic-software.md>), [agentic-software-development](<https://devfeed.tech/tags/agentic-software-development.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-coding-assistants](<https://devfeed.tech/tags/ai-coding-assistants.md>), [code](<https://devfeed.tech/tags/code.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [manufacturing](<https://devfeed.tech/tags/manufacturing.md>), [sdlc](<https://devfeed.tech/tags/sdlc.md>), [software](<https://devfeed.tech/tags/software.md>), [software-supply-chain](<https://devfeed.tech/tags/software-supply-chain.md>), [supply-chain](<https://devfeed.tech/tags/supply-chain.md>), [thoughts](<https://devfeed.tech/tags/thoughts.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

The article argues that AI accelerates code production faster than existing review and testing loops can verify it, shifting the engineering bottleneck toward proving behavior before release. Static analysis and human diff reviews may miss cross-service runtime failures, while relying on canaries risks exposing users to too many regressions. It proposes Verification Engineering built around disposable production-equivalent sandboxes, synthetic test data shaped by domain distributions and failure patterns, automatically generated test harnesses and correctness/performance/resource assertions, and simulated end-to-end user workflows. Requirements and designs would become executable specifications, while production logs, traces, and interactions would feed anomalies back into future sandbox tests. These are presented as a proposed discipline, not as a comparison of specific products or evidence of measured results.

### Source excerpt

When AI accelerates code manufacturing, the software supply chain bottleneck shifts from writing code to building the feedback loop that proves it works.

## CoreWeave Tops NVIDIA Blackwell Inference Throughput, Cuts TCO Up to 65%

DevFeed: [CoreWeave Tops NVIDIA Blackwell Inference Throughput, Cuts TCO Up to 65%](<https://devfeed.tech/articles/coreweave-tops-nvidia-blackwell-inference-throughput-cuts-tco-up-to-65-142075.md>)

Original publisher: [Read original article](<https://wf.coreweave.com/blog/coreweave-tops-nvidia-blackwell-inference-throughput-cuts-tco-up-to-65>)

Author: CoreWeave

Published: 2026-11-18T15:38:38Z

Content type: news

Language: en

Sources: [Coreweave Blog Posts](<https://devfeed.tech/sources/coreweave-blog-posts.md>)

Topics: [NVIDIA Blackwell](<https://devfeed.tech/topics/nvidia-blackwell.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [GB200](<https://devfeed.tech/topics/gb200.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [AI Factory](<https://devfeed.tech/topics/ai-factory.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [api](<https://devfeed.tech/tags/api.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [blog](<https://devfeed.tech/tags/blog.md>), [cks](<https://devfeed.tech/tags/cks.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-infrastructure](<https://devfeed.tech/tags/cloud-infrastructure.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [coreweave](<https://devfeed.tech/tags/coreweave.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cost-per-token](<https://devfeed.tech/tags/cost-per-token.md>), [costs](<https://devfeed.tech/tags/costs.md>), [inference](<https://devfeed.tech/tags/inference.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-blackwell](<https://devfeed.tech/tags/nvidia-blackwell.md>), [scale](<https://devfeed.tech/tags/scale.md>), [tco](<https://devfeed.tech/tags/tco.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [tokens](<https://devfeed.tech/tags/tokens.md>)

### AI overview

Signal65's modeled 5,000-GPU, three-year comparison reports CoreWeave had the lowest TCO in six of seven configurations and the highest per-GPU inference throughput among the four clouds evaluated. For GB200 NVL72, its modeled cost was 22% to 65% below the hyperscalers; the 22% comparison uses an estimated Hyperscaler B price. On DeepSeek-R1, CoreWeave delivered 4,867 tokens per second per GPU and $0.75 per million tokens, versus $1.23-$2.22 elsewhere, yielding up to 2.9 times as many tokens per dollar. The account says the model omits some hyperscaler storage fees and archive deletion penalties. For hosted inference, CoreWeave was cheaper on 11 of 17 comparable models and tied on four; on gpt-oss-120b, its input and output token prices were respectively 67%-80% and 53%-72% lower. The results are reported in a CoreWeave-authored summary of the study.

### Source excerpt

What does NVIDIA Blackwell cost at scale? Signal65 found CoreWeave delivered the lowest three-year TCO in six of seven configurations and up to 2.9x more tokens per dollar.

## ReBoT-MiCS: Reliability-Aware Bounding Time for Enhancing Timing Behavior of Embedded Mixed-Criticality Systems

DevFeed: [ReBoT-MiCS: Reliability-Aware Bounding Time for Enhancing Timing Behavior of Embedded Mixed-Criticality Systems](<https://devfeed.tech/articles/rebot-mics-reliability-aware-bounding-time-for-enhancing-timing-behavior-of-embedded-mixed-criticality-systems-139610.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22588>)

Author: Mohammad Abbasinia, Behnaz Ranjbar, Akash Kumar

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Embedded Systems](<https://devfeed.tech/topics/embedded-systems.md>), [scheduling](<https://devfeed.tech/topics/scheduling.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ce](<https://devfeed.tech/tags/cs-ce.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [design](<https://devfeed.tech/tags/design.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fault-tolerance](<https://devfeed.tech/tags/fault-tolerance.md>), [mc](<https://devfeed.tech/tags/mc.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [qos](<https://devfeed.tech/tags/qos.md>), [rebot](<https://devfeed.tech/tags/rebot.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [scheme](<https://devfeed.tech/tags/scheme.md>), [selection](<https://devfeed.tech/tags/selection.md>), [switches](<https://devfeed.tech/tags/switches.md>), [switching](<https://devfeed.tech/tags/switching.md>), [time](<https://devfeed.tech/tags/time.md>), [timing](<https://devfeed.tech/tags/timing.md>)

### AI overview

ReBoT-MiCS selects low-criticality worst-case execution-time bounds at design time, jointly accounting for processor utilization, quality of service, mode-switch probability, and probabilistic reliability constraints under a transient-fault exposure model. The approach treats execution-time bounds as part of reliability reasoning, where prior approaches described in the abstract typically handled reliability separately through fault-tolerance mechanisms. Experiments on synthetic and embedded benchmarks report utilization improvements of up to 27.6% and 11.67% on average, with mode-switch probability bounded by up to 5.48% while meeting specified reliability constraints. The abstract does not provide further details on the method or benchmark setup.

### Source excerpt

arXiv:2611.22588v1 Announce Type: cross Abstract: In Mixed-Criticality (MC) systems, considering multiple Worst-Case Execution Times (WCETs) for the same task, corresponding to different system operation modes, presents a fundamental challenge in MC system design. Identifying suitable WCETs for lower (LO) criticality system operational modes is inherently difficult. Tight low WCET values may trigger frequent mode switches that degrade Quality-of-Service (QoS) by dropping/degrading the lower-criticality tasks and reducing their functionality. However, more conservative values reduce resource efficiency. Moreover, low WCET selection affects exposure assumptions under the adopted transient-fault model, which is used in reliability-aware scheduling analysis. Although some studies have considered low WCET determination to make a reasonable trade-off between processor utilization, mode switching probability and QoS, reliability constraints are typically addressed separately through fault-tolerance mechanisms. This separation overlooks the opportunity to use execution-time bounds as part of design-time reasoning about exposure-related reliability constraints. In this regard, we propose a reliability-aware analytical scheme, named ReBoT-MiCS, to determine low WCETs at design time, jointly balancing utilization, QoS, mode switches, and probabilistic reliability constraints of each task criticality level under a transient-fault exposure model. Experiments conducted with various synthetic and embedded benchmarks demonstrate the efficacy of the proposed scheme, in which utilization can be improved by up to 27.6% and 11.67% on average, while bounding the mode switching probability by up to 5.48%, and satisfying the specified reliability constraints

## PAISF: A Partially Adaptive In-Order Selection Function for Routing in On-Chip, Inter-Chiplet, and Wafer-Scale Networks

DevFeed: [PAISF: A Partially Adaptive In-Order Selection Function for Routing in On-Chip, Inter-Chiplet, and Wafer-Scale Networks](<https://devfeed.tech/articles/paisf-a-partially-adaptive-in-order-selection-function-for-routing-in-on-chip-inter-chiplet-and-wafer-scale-networks-140406.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22772>)

Author: Patrick Iff, Luca Oertig, Andrin Weber, Luca Benini, Torsten Hoefler

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [IP routing](<https://devfeed.tech/topics/ip-routing.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [chiplet](<https://devfeed.tech/tags/chiplet.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [delivery](<https://devfeed.tech/tags/delivery.md>), [energy](<https://devfeed.tech/tags/energy.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [irregular](<https://devfeed.tech/tags/irregular.md>), [networks](<https://devfeed.tech/tags/networks.md>), [routing](<https://devfeed.tech/tags/routing.md>), [selection](<https://devfeed.tech/tags/selection.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [wafer-scale-integration](<https://devfeed.tech/tags/wafer-scale-integration.md>)

### AI overview

ISF and PAISF are selection functions for irregular on-chip, inter-chiplet, and wafer-scale networks that preserve in-order delivery by hashing each packet's entropy value to choose its path, so packets with the same entropy follow the same path. Across the reported workloads and topologies, the methods improve throughput by 15.2% and reduce energy per packet by 12.1% and latency under load by 11.3% on average, compared with a deterministic routing function, a deterministic selection function, and Multi. The tradeoff is a 3.3% average increase in zero-load latency. The supplied text gives abstract-level results, not implementation details.

### Source excerpt

arXiv:2611.22772v1 Announce Type: cross Abstract: The growing number of compute elements per chip, along with advances such as chiplet systems and wafer-scale integration, drives the need for scalable and efficient routing algorithms for irregular topologies. A routing algorithm consists of a routing function (RF) that ensures deadlock freedom and a selection function (SF) that avoids congestion and, in some cases, guarantees in-order packet delivery. We propose two novel SFs for irregular topologies, ISF and PAISF, which ensure in-order delivery by hashing a packet's entropy value to determine its path. Packets with the same entropy follow the same path and arrive in order. Compared to a deterministic RF, a deterministic SF, and Multi, three alternative in-order delivery strategies, (PA)ISF improves throughput by 15.2% and reduces energy per packet and latency under load by 12.1% and 11.3%, at the cost of a zero-load latency increase by 3.3% on average across workloads and topologies.

## SLED: Sensor SLicing from Edge to Datacenter

DevFeed: [SLED: Sensor SLicing from Edge to Datacenter](<https://devfeed.tech/articles/sled-sensor-slicing-from-edge-to-datacenter-139930.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21937>)

Author: Alan Subedi, Avhishek Biswas, Apala Pramanik, Amlan Balabantaray, Mikhail Titov, Santosh Pitla, Ozgur Kilic, Mehmet C. Vuran

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [systems](<https://devfeed.tech/topics/systems.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [1-4](<https://devfeed.tech/tags/1-4.md>), [2](<https://devfeed.tech/tags/2.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [access-control](<https://devfeed.tech/tags/access-control.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [ai](<https://devfeed.tech/tags/ai.md>), [air](<https://devfeed.tech/tags/air.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [coding](<https://devfeed.tech/tags/coding.md>), [compute](<https://devfeed.tech/tags/compute.md>), [computing](<https://devfeed.tech/tags/computing.md>), [control](<https://devfeed.tech/tags/control.md>), [critical](<https://devfeed.tech/tags/critical.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ce](<https://devfeed.tech/tags/cs-ce.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-et](<https://devfeed.tech/tags/cs-et.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [data](<https://devfeed.tech/tags/data.md>), [data-volume](<https://devfeed.tech/tags/data-volume.md>), [datacenter](<https://devfeed.tech/tags/datacenter.md>), [edge](<https://devfeed.tech/tags/edge.md>), [sensor](<https://devfeed.tech/tags/sensor.md>)

### AI overview

SLED isolates individual sensor streams within one device using deterministic MAC-level Sensor Slicing in O-RAN, then carries per-sensor radio and network isolation across edge, campus, and remote HPC tiers. This addresses contention in which bulk transfers can starve latency-critical telemetry. In over-the-air experiments, conventional network slicing increased critical delays by more than 2,300-fold under high traffic, while SLED maintained stable latency. As sensor slices scaled, throughput stayed within 0.9% and 1.4% of expected allocations on the two evaluated RAN stacks. The abstract also reports that agentic coding produced O-RAN implementations in OCUDU and OAI with substantially lower development time, prompts, and token use; it does not describe the coding process in detail.

### Source excerpt

arXiv:2611.21937v1 Announce Type: new Abstract: Modern sensing systems generate large volumes of heterogeneous data streams from multiple sensor modalities, increasing reliance on datacenters for computation, artificial intelligence/superintelligence (AI/SI) inference, and AI/SI training. Each sensor stream may have different network and compute requirements depending on its type and data volume. However, current wireless systems typically allocate resources at the device level, leading to contention among co-located sensor streams, where bulk data transfers may starve latency-critical telemetry. To this end, this paper introduces Sensor Slicing (SL). SL provides deterministic, medium access control (MAC)-level isolation for individual sensors within a single user equipment (UE) in open radio access networks (O-RAN). Using agentic coding techniques, SL is implemented in two O-RAN-compliant packages, reducing development time by 400 times (2 h 40 min [OCUDU] vs. 6.5 weeks [OAI]), 228-fold fewer human prompts (4 vs. 913), and less than half the tokens (0.64B vs. 1.43B). An associated end-to-end framework, SLED, is implemented to establish per-sensor radio and network isolation across edge, campus, and remote high-performance computing (HPC) tiers. Over-the-air wireless experiments demonstrate that while conventional network slicing causes critical delays to increase more than 2,300-fold under high traffic load, SLED maintains stable latency. Scaling evaluations on the two RAN stacks show that throughput remains within 0.9% and 1.4% of expected allocations even as the number of sensor slices increases. The results demonstrate that SLED preserves latency-critical telemetry while effectively utilizing resources across the sensor-to-HPC pipeline. The SLED software and evaluation datasets are publicly available at https://github.com/UNL-CPN-Lab/SLED-Sensor-SLicing-from-Edge-to-Datacenter.

## Revisiting Index Performance under High Cache-Miss Penalties

DevFeed: [Revisiting Index Performance under High Cache-Miss Penalties](<https://devfeed.tech/articles/revisiting-index-performance-under-high-cache-miss-penalties-140625.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21835>)

Author: Mathew M. Kim, Beomseok Nam

Published: 2026-11-18T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [application](<https://devfeed.tech/tags/application.md>), [applications](<https://devfeed.tech/tags/applications.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cache](<https://devfeed.tech/tags/cache.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [cxl](<https://devfeed.tech/tags/cxl.md>), [designs](<https://devfeed.tech/tags/designs.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [index](<https://devfeed.tech/tags/index.md>), [indexes](<https://devfeed.tech/tags/indexes.md>), [isolation](<https://devfeed.tech/tags/isolation.md>), [latency](<https://devfeed.tech/tags/latency.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-bandwidth](<https://devfeed.tech/tags/memory-bandwidth.md>), [modern-data](<https://devfeed.tech/tags/modern-data.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

AuxBench evaluates byte-addressable indexes as part of larger application workloads, showing that index-only benchmarks can mispredict performance. Across four indexes, controlled post-lookup memory activity changed their relative performance, and designs optimized for concurrency could lose their advantage when application work competed for cache, memory bandwidth, and CPU. Time-varying request arrivals also produced severe tail latency near saturation. The abstract describes the tested factors and findings but does not name the indexes or provide quantitative results.

### Source excerpt

arXiv:2611.21835v1 Announce Type: new Abstract: Modern data-intensive applications often query very large datasets, making index performance critical. Recent index structures exploit the fine-grained access capabilities of emerging storage devices such as DCPMM and CXL. However, existing benchmarks typically evaluate indexes in isolation using saturated, closed-loop workloads that emphasize maximum throughput. Such workloads favor highly concurrent designs while overlooking the application work that follows an index lookup and competes for cache, memory bandwidth, and CPU resources. We present AuxBench - a microbenchmark that evaluates indexes as auxiliary access paths within larger application workloads. Using AuxBench, we evaluate four byte-addressable indexes under controlled post-lookup memory activity, concurrency, and request arrival patterns. Our experiments show that post-lookup memory activity can change the relative performance of competing indexes, that concurrency-oriented designs may lose their advantages when substantial application work is introduced, and that time-varying request arrivals can produce severe tail latency near saturation. These findings demonstrate why index-only benchmarks may not adequately predict performance in larger application workloads

## Can Verifiable PIR Match the Online Efficiency of SimplePIR?

DevFeed: [Can Verifiable PIR Match the Online Efficiency of SimplePIR?](<https://devfeed.tech/articles/can-verifiable-pir-match-the-online-efficiency-of-simplepir-141502.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22911>)

Author: Xinxuan Zhang, Ruida Wang, Zirui Wang, Yuming Guo, Jikang Bai, Yuanju Wei, Yi Deng, Xianhui Lu

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Cryptography](<https://devfeed.tech/topics/cryptography.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>)

Tags: [communication](<https://devfeed.tech/tags/communication.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [online](<https://devfeed.tech/tags/online.md>), [pir](<https://devfeed.tech/tags/pir.md>), [protocol](<https://devfeed.tech/tags/protocol.md>)

### AI overview

The proposed verifiable PIR protocol matches SimplePIR's online communication and server computation costs, with nearly identical online client computation. It uses an error-correcting-code-based commitment as the database digest, enabling efficient proofs of linear relations and precise range proofs without the SIS assumption. Compared with VeriSimplePIR, the authors report about 20% lower online communication and client computation, and over 35% lower online server computation. These are claims from the abstract excerpt.

### Source excerpt

arXiv:2611.22911v1 Announce Type: new Abstract: Verifiable Private Information Retrieval (vPIR) ensures that a malicious server cannot return tampered database records, but it traditionally incurs significant overhead compared to classical PIR.Recent advancements, notably VeriSimplePIR, approach the online efficiency of SimplePIR by introducing a proof preprocessing phase and requiring only lightweight client-side online verification. However, this scheme relies on an additional Short Integer Solution (SIS) assumption and SIS-based commitments, which only support relaxed range proofs. Consequently, these schemes are forced to adopt significantly weaker parameter choices, resulting in additional online overhead. In this paper, we propose a novel vPIR protocol that employs a newly designed error-correcting code-based commitment scheme as the database digest. This approach (i)supports efficient proofs of linear relations, (ii) achieves precise range proofs and (iii) eliminates the reliance on the SIS assumption. Thus, our scheme successfully adopts parameter choices identical to those of SimplePIR. As a result, our vPIR protocol is the first to achieve exactly the same online communication and server computation costs as SimplePIR, while keeping online client computation nearly identical. Additionally, compared to VeriSimplePIR,our protocol reduces online communication and client-side computation by approximately 20%, and decreases online server-side computation by over 35%.

## XM-Sparse: High-Throughput Sparse-Attention LLM Serving over Scale-up Memory Pools

DevFeed: [XM-Sparse: High-Throughput Sparse-Attention LLM Serving over Scale-up Memory Pools](<https://devfeed.tech/articles/xm-sparse-high-throughput-sparse-attention-llm-serving-over-scale-up-memory-pools-140386.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22280>)

Author: Yijun Sun, Xudong Liao, Zhuobin Huang, Kai Chen, Nikita Lazarev, Zhizhen Zhong

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [memory pool](<https://devfeed.tech/topics/memory-pool.md>), [dma](<https://devfeed.tech/topics/dma.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [cache](<https://devfeed.tech/tags/cache.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decode](<https://devfeed.tech/tags/decode.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [dma](<https://devfeed.tech/tags/dma.md>), [echo](<https://devfeed.tech/tags/echo.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-serving](<https://devfeed.tech/tags/llm-serving.md>), [memory](<https://devfeed.tech/tags/memory.md>), [scale-up](<https://devfeed.tech/tags/scale-up.md>), [serving](<https://devfeed.tech/tags/serving.md>), [sparse-attention](<https://devfeed.tech/tags/sparse-attention.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

XM-Sparse targets an HBM-capacity bottleneck in sparse-attention serving: hot KV buffers and the fully resident indexer-key cache still consume substantial GPU memory, even with scale-up memory pools. Built on SGLang, it offloads both caches using different I/O strategies: dynamically selected KV entries are retrieved through a TMA-based tile pipeline fused with attention, while predictable K-index pages are prefetched with GPU-initiated DMA coordinated around latency-critical decoding. On DeepSeek-V3.2 and GLM-5.3 with an emulated scale-up memory pool, the authors report up to 4.98x and 4.52x higher decode throughput than PCIe-based ECHO and HiSparse, respectively, and up to 1.34x over a naive integration on the same platform across 16K-1M contexts.

### Source excerpt

arXiv:2611.22280v1 Announce Type: new Abstract: Sparse attention has been increasingly adopted by production LLMs for long-context inference, yet serving systems must still retain the full KV history, making GPU HBM capacity a key limit on serving concurrency. Existing systems offload the KV cache to host memory, but suffer from the limited bandwidth of PCIe and the limited capacity of host memory. Emerging scale-up memory pools (e.g., X-Mem) relieve both limits and open new opportunities for sparse-attention offloading. In this paper, we first conduct an in-depth analysis and reveal that, even with scale-up memory pools, decode throughput remains bounded by HBM capacity, as hot KV buffers and the fully resident indexer key (K-index) cache occupy substantial HBM. Moreover, the two caches exhibit distinct access patterns. Based on these insights, we present XM-Sparse, the first sparse-attention offloading system over scale-up memory expansion. The core idea of XM-Sparse is to offload both caches aggressively while matching each with an I/O mechanism tailored to its access pattern. Specifically, for dynamically selected KV entries, XM-Sparse fuses TMA-based retrieval with attention in a tile-level pipeline to hide on-demand retrieval latency. For predictable K-index accesses, it prefetches pages via SM-efficient GPU-initiated DMA and coordinates prefetching with latency-critical operations to avoid interfering with decoding. We have implemented XM-Sparse atop SGLang, and our experiments on DeepSeek-V3.2 and GLM-5.3 with an emulated scale-up memory pool show that XM-Sparse delivers up to 4.98$\times$ and 4.52$\times$ higher decode throughput than PCIe-based ECHO and HiSparse, respectively, and up to 1.34$\times$ over a naive integration on the same platform, across 16K-1M contexts.

## Angelfish: Leader, DAG, or Anywhere in Between

DevFeed: [Angelfish: Leader, DAG, or Anywhere in Between](<https://devfeed.tech/articles/angelfish-leader-dag-or-anywhere-in-between-140375.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2509.15847>)

Author: Qianyu Yu, Giuliano Losa, Nibesh Shrestha, Xuechao Wang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [blockchain consensus protocol](<https://devfeed.tech/topics/blockchain-consensus-protocol.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [protocols](<https://devfeed.tech/topics/protocols.md>)

Tags: [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [bft-consensus](<https://devfeed.tech/tags/bft-consensus.md>), [blockchain](<https://devfeed.tech/tags/blockchain.md>), [broadcast](<https://devfeed.tech/tags/broadcast.md>), [broadcasting](<https://devfeed.tech/tags/broadcasting.md>), [communication](<https://devfeed.tech/tags/communication.md>), [consensus](<https://devfeed.tech/tags/consensus.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [dag](<https://devfeed.tech/tags/dag.md>), [data](<https://devfeed.tech/tags/data.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [latency](<https://devfeed.tech/tags/latency.md>), [leader](<https://devfeed.tech/tags/leader.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [load](<https://devfeed.tech/tags/load.md>), [nodes](<https://devfeed.tech/tags/nodes.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [protocols](<https://devfeed.tech/tags/protocols.md>)

### AI overview

Angelfish combines leader-based and DAG-based BFT consensus, adapting between them by dynamically selecting parties to send lightweight votes via best-effort broadcast instead of reliably broadcasting costlier DAG vertices. The authors report that this reduces communication, helps lagging nodes catch up, and lowers latency compared with prior DAG-based protocols. Their evaluation reports state-of-the-art peak throughput and significantly lower latency under moderate load; the supplied excerpt gives no benchmark figures or further evaluation details.

### Source excerpt

arXiv:2509.15847v4 Announce Type: replace Abstract: To maximize performance, many modern blockchain systems rely on eventually synchronous, Byzantine fault-tolerant (BFT) consensus protocols. Two protocol designs have emerged in this space: protocols that minimize latency using a leader that drives both data dissemination and consensus, and protocols that maximize throughput using a separate, asynchronous data dissemination layer. Recent protocols such as Partially Synchronous Bullshark and Sailfish combine elements of both approaches by using a DAG to enable parallel data dissemination and a leader that paces DAG formation. This improves latency while achieving state-of-the-art throughput. Yet leader-based protocols still achieve better latency under moderate loads, which are common in practice. We present Angelfish, a hybrid protocol that adapts smoothly across this design space, from leader-based to Sailfish-like DAG-based consensus. Angelfish lets a dynamically adjusted subset of parties use best-effort broadcast to issue lightweight votes instead of reliably broadcasting costlier DAG vertices. This reduces communication, helps lagging nodes catch up, and lowers latency in practice compared to prior DAG-based protocols. Our empirical evaluation shows that Angelfish attains state-of-the-art peak throughput while significantly lowering latency under moderate load, delivering the best of both worlds.

## Fivetran vs Stitch: Performance, Pricing, and More

DevFeed: [Fivetran vs Stitch: Performance, Pricing, and More](<https://devfeed.tech/articles/fivetran-vs-stitch-performance-pricing-and-more-140231.md>)

Original publisher: [Read original article](<https://wf.fivetran.com/blog/fivetran-vs-stitch>)

Author: Fivetran

Published: 2026-11-18T04:57:01Z

Content type: comparison

Language: en

Sources: [Fivetran Blog](<https://devfeed.tech/sources/fivetran-blog.md>)

Topics: [build vs buy database solutions](<https://devfeed.tech/topics/build-vs-buy-database-solutions.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [Service-Level Agreements](<https://devfeed.tech/topics/service-level-agreements.md>)

Tags: [automation](<https://devfeed.tech/tags/automation.md>), [batch](<https://devfeed.tech/tags/batch.md>), [compare](<https://devfeed.tech/tags/compare.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [compatibility](<https://devfeed.tech/tags/compatibility.md>), [compliance](<https://devfeed.tech/tags/compliance.md>), [connectors](<https://devfeed.tech/tags/connectors.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [data-modeling](<https://devfeed.tech/tags/data-modeling.md>), [data-pipelines](<https://devfeed.tech/tags/data-pipelines.md>), [data-readiness](<https://devfeed.tech/tags/data-readiness.md>), [dbt](<https://devfeed.tech/tags/dbt.md>), [elt](<https://devfeed.tech/tags/elt.md>), [features](<https://devfeed.tech/tags/features.md>), [fivetran](<https://devfeed.tech/tags/fivetran.md>), [performance](<https://devfeed.tech/tags/performance.md>), [pricing](<https://devfeed.tech/tags/pricing.md>), [stitch](<https://devfeed.tech/tags/stitch.md>), [use-cases](<https://devfeed.tech/tags/use-cases.md>), [vs](<https://devfeed.tech/tags/vs.md>)

### AI overview

Fivetran emphasizes managed operations and broad, in-house-maintained connector coverage, which the text presents as reducing engineering work and supporting scalable, near-real-time syncs. Stitch's Singer-based open framework gives developer-oriented teams more ability to customize connectors, but requires more manual maintenance and scaling; its batch syncs may suit less frequent data needs. The text also says Fivetran standardizes schemas and streamlines post-load transformations, while Stitch leaves more modeling and transformation work to teams. Fivetran is described as having stronger support, documentation, and enterprise security features. Its active-row pricing may be efficient, though complex data flows can make monthly costs harder to predict. The comparison offers no specific prices, and its performance and compliance claims are presented without supporting measurements or citations.

### Source excerpt

Choosing between Fivetran and Stitch for automated ELT? Compare features, pricing, and use cases to find the right platform.

## Benchmarked: A data pipeline latency analysis

DevFeed: [Benchmarked: A data pipeline latency analysis](<https://devfeed.tech/articles/benchmarked-a-data-pipeline-latency-analysis-140198.md>)

Original publisher: [Read original article](<https://wf.fivetran.com/blog/benchmarked-a-data-pipeline-latency-analysis>)

Author: Eric O'Connor

Published: 2026-11-18T04:00:45Z

Content type: article

Language: en

Sources: [Fivetran Blog](<https://devfeed.tech/sources/fivetran-blog.md>)

Topics: [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Data analysis](<https://devfeed.tech/topics/data-analysis.md>), [data transformation](<https://devfeed.tech/topics/data-transformation.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [change-data-capture](<https://devfeed.tech/tags/change-data-capture.md>), [data](<https://devfeed.tech/tags/data.md>), [data-pipeline](<https://devfeed.tech/tags/data-pipeline.md>), [data-pipeline-performance](<https://devfeed.tech/tags/data-pipeline-performance.md>), [fivetran](<https://devfeed.tech/tags/fivetran.md>), [latency](<https://devfeed.tech/tags/latency.md>)

### AI overview

Fivetran's incremental-sync benchmark reports that, in most cases, it delivered data to Snowflake in under 17 minutes while handling a simulated workload of about 240 GB per hour and roughly 16,000 transactions per second on a single-node database. The test used HammerDB's TPROC-C workload, augmented with detail columns to bring average row width to about 1 KB, and measured freshness from gaps between periodic micro-batch syncs. Latency depends on workload size; the authors expect less intensive workloads to support lower latency. The excerpt describes Oracle-to-Snowflake results and says the framework was being extended to other configurations.

### Source excerpt

Learn how Fivetran measures the latency of our pipelines.

## Benchmarked: A data pipeline throughput analysis

DevFeed: [Benchmarked: A data pipeline throughput analysis](<https://devfeed.tech/articles/benchmarked-a-data-pipeline-throughput-analysis-140199.md>)

Original publisher: [Read original article](<https://wf.fivetran.com/blog/benchmarked-a-data-pipeline-throughput-analysis>)

Author: Eric O'Connor

Published: 2026-11-18T04:00:44Z

Content type: news

Language: en

Sources: [Fivetran Blog](<https://devfeed.tech/sources/fivetran-blog.md>)

Topics: [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [elt](<https://devfeed.tech/topics/elt.md>), [rdbms](<https://devfeed.tech/topics/rdbms.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [Analytics Pipeline](<https://devfeed.tech/topics/analytics-pipeline.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [change-data-capture](<https://devfeed.tech/tags/change-data-capture.md>), [connectors](<https://devfeed.tech/tags/connectors.md>), [conversion](<https://devfeed.tech/tags/conversion.md>), [data-pipeline](<https://devfeed.tech/tags/data-pipeline.md>), [data-pipelines](<https://devfeed.tech/tags/data-pipelines.md>), [data-warehouse](<https://devfeed.tech/tags/data-warehouse.md>), [database](<https://devfeed.tech/tags/database.md>), [databases](<https://devfeed.tech/tags/databases.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [details](<https://devfeed.tech/tags/details.md>), [fivetran](<https://devfeed.tech/tags/fivetran.md>), [load-testing](<https://devfeed.tech/tags/load-testing.md>), [testing](<https://devfeed.tech/tags/testing.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

Fivetran's excerpt describes a HammerDB TPROC-C benchmark for historical database loads, using an approximately 1 TB dataset with rows widened to about 1 KB to approximate customer data. The test measures throughput as dataset size divided by elapsed sync time; the stated full sync to Snowflake took 1 hour 30 minutes to 1 hour 40 minutes, but the excerpt omits the numeric throughput results and detailed configuration. Fivetran says its sync process uses checkpoints to resume after interruptions and splits large tables into chunks to limit transaction duration. It attributes performance improvements to greater parallelism and optimized warehouse loading patterns.

### Source excerpt

Looking under the hood and load testing Fivetran's data pipelines.

## Doubling Query Embedding Throughput with Dynamic Batching in Vespa Cloud

DevFeed: [Doubling Query Embedding Throughput with Dynamic Batching in Vespa Cloud](<https://devfeed.tech/articles/doubling-query-embedding-throughput-with-dynamic-batching-in-vespa-cloud-144297.md>)

Original publisher: [Read original article](<https://blog.vespa.ai/doubling-query-embedding-throughput-with-dynamic-batching/>)

Author: gleb ingrid

Published: 2026-11-18T00:00:00Z

Content type: article

Language: en

Sources: [Vespa Blog](<https://devfeed.tech/sources/vespa-blog.md>)

Topics: [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Dynamo-Triton](<https://devfeed.tech/topics/dynamo-triton.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>)

Tags: [batching](<https://devfeed.tech/tags/batching.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [efficiently](<https://devfeed.tech/tags/efficiently.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [embedding-model](<https://devfeed.tech/tags/embedding-model.md>), [embeddinggemma-2](<https://devfeed.tech/tags/embeddinggemma-2.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [latency](<https://devfeed.tech/tags/latency.md>), [local](<https://devfeed.tech/tags/local.md>), [models](<https://devfeed.tech/tags/models.md>), [onnx](<https://devfeed.tech/tags/onnx.md>), [performance](<https://devfeed.tech/tags/performance.md>), [queries](<https://devfeed.tech/tags/queries.md>), [query](<https://devfeed.tech/tags/query.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [vespa-cloud](<https://devfeed.tech/tags/vespa-cloud.md>)

### AI overview

Vespa Cloud's inference runtime (Vespa 8.784.16) uses NVIDIA Triton to queue concurrent embedding requests and combine waiting requests into GPU batches. A concurrency limit creates the queue that makes batching possible; maximum batch size and token padding buckets are configurable, and the runtime does not wait for batches to fill. In tests on an NVIDIA L4 with short queries, batch size 8, buckets 32 and 64, and concurrency 1 served twice the query load at the same median latency; peak throughput was about 2.5 times higher, with lower p99 latency at all but low loads. The benchmark included queueing, embedding, search, and result fetching, but excluded rendering and network round trips. The article recommends leaving batching defaults on CPU nodes, where cores already process requests in parallel and padding adds work; long inputs may also gain little from batching. The feature is for Vespa Cloud container nodes.

### Source excerpt

Vespa Cloud now supports dynamic batching of concurrent requests for local embedding models. In our benchmarks, this more than doubles the number of queries embedded per second on a GPU, while adding just a few milliseconds of latency at low loads.

[Next page](<https://devfeed.tech/topics/performance-engineering.md?cursor=WyIyMDI2LTExLTE4VDAwOjAwOjAwKzAwOjAwIiwgImZlZjQxMzE2LTI1YWMtNDg1Ni05MjFhLTBhZjAyZTMyNTc0NSJd>)