# Performance Testing

A software testing practice that evaluates an application or workload's speed, stability, scalability, and responsiveness under varying workloads.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Conversational AI Testing: Types, Workflow, Challenges & Best Practices

DevFeed: [Conversational AI Testing: Types, Workflow, Challenges & Best Practices](<https://devfeed.tech/articles/conversational-ai-testing-types-workflow-challenges-best-practices-145968.md>)

Original publisher: [Read original article](<https://testgrid.io/blog/conversational-ai-testing/>)

Author: Tejas Patel

Published: 2026-11-19T10:59:06Z

Content type: article

Language: en

Sources: [TestGrid](<https://devfeed.tech/sources/testgrid-blog.md>)

Topics: [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Chat Bot](<https://devfeed.tech/topics/chatbot.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [access-controls](<https://devfeed.tech/tags/access-controls.md>), [actions](<https://devfeed.tech/tags/actions.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [adversarial-testing](<https://devfeed.tech/tags/adversarial-testing.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-testing](<https://devfeed.tech/tags/ai-testing.md>), [api](<https://devfeed.tech/tags/api.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [call](<https://devfeed.tech/tags/call.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [conversational-ai](<https://devfeed.tech/tags/conversational-ai.md>), [conversational-ai-testing](<https://devfeed.tech/tags/conversational-ai-testing.md>), [customer](<https://devfeed.tech/tags/customer.md>), [order](<https://devfeed.tech/tags/order.md>), [request](<https://devfeed.tech/tags/request.md>), [software-testing](<https://devfeed.tech/tags/software-testing.md>), [test-automation](<https://devfeed.tech/tags/test-automation.md>), [testing](<https://devfeed.tech/tags/testing.md>), [types](<https://devfeed.tech/tags/types.md>), [workflow](<https://devfeed.tech/tags/workflow.md>)

### AI overview

Conversational AI testing must evaluate meaning and outcomes across the full interaction stack, since responses can vary and fluent answers may still be fabricated or trigger incorrect actions. The source recommends testing intent and entity recognition, context retention, grounded accuracy, task and tool execution, safety and fallback, performance, security, and voice behavior where applicable. A workflow is to define objectives and pass criteria, design standard and edge-case scenarios, prepare varied and privacy-conscious data, run tests through components and end-to-end journeys, assess task completion, relevance, groundedness, tool success and latency, then investigate failures and rerun regression tests. Exact-string checks suit deterministic behavior, while open-ended answers need semantic or rubric-based evaluation, potentially with calibrated LLM judges and human review for subjective or high-risk cases. TestGrid supplies device, browser, audio-routing, network and execution capabilities; the test framework or evaluator judges response quality.

### Source excerpt

Traditional software testing loves an answer key. Click this button. Expect this screen. Send this request. Expect 200 OK. Then conversational AI walked in and decided that five different answers could all be correct, the sixth could sound correct while being completely fabricated, and the seventh might call an API and cancel your customer's order. [...]

## Clover-35B-A3B-Preview Technical Report

DevFeed: [Clover-35B-A3B-Preview Technical Report](<https://devfeed.tech/articles/clover-35b-a3b-preview-technical-report-142816.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24636>)

Author: DiffuSpace Technical Team

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [text-generation](<https://devfeed.tech/topics/text-generation.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [attention](<https://devfeed.tech/tags/attention.md>), [batch](<https://devfeed.tech/tags/batch.md>), [bf16](<https://devfeed.tech/tags/bf16.md>), [block](<https://devfeed.tech/tags/block.md>), [clover](<https://devfeed.tech/tags/clover.md>), [context](<https://devfeed.tech/tags/context.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decision](<https://devfeed.tech/tags/decision.md>), [decision-model](<https://devfeed.tech/tags/decision-model.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [experts](<https://devfeed.tech/tags/experts.md>), [gb300](<https://devfeed.tech/tags/gb300.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [technical](<https://devfeed.tech/tags/technical.md>)

### AI overview

Clover-35B-A3B-Preview is a 35-billion-parameter mixture-of-experts diffusion language model that activates 3 billion parameters and supports thinking-mode generation. Its block-diffusion architecture combines recurrent memory of earlier context with bidirectional softmax attention within the current block; both compete in a shared softmax, with learnable block-size calibration and a separate zero-initialized right-context convolution branch. The report says Clover leads seven evaluated agent metrics, scoring 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reports 680 tokens/s on one NVIDIA GB300 at BF16 and batch size one. A decision-model variant predicts answers to related questions in one denoising pass; on 500 internal groups, it raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%. These are abstract-level reported results; the supplied text does not give detailed evaluation methods.

### Source excerpt

arXiv:2611.24636v1 Announce Type: cross Abstract: We introduce Clover-35B-A3B-Preview (Clover), a mixture-of-experts diffusion language model with 35 billion total parameters and 3 billion activated parameters, supporting thinking-mode generation. Clover adapts Gated Delta Networks to block diffusion by combining recurrent memory of previously generated context with bidirectional softmax attention inside the current block. Historical memory readout competes with current-block tokens in a shared softmax, with learnable block-size calibration and a separate, zero-initialized right-context convolution branch. Clover leads the seven evaluated agent metrics, including 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reaches a reported generation speed of 680 tokens/s using one NVIDIA GB300 GPU at BF16 precision and batch size one. We organize the evaluation around model quality, drafting and verification costs, and deployment under constrained memory. We also describe a decision-model variant that generates answers to related questions in one denoising pass. On an internal set of 500 groups, joint prediction raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%.

## Revisiting Index Performance under High Cache-Miss Penalties

DevFeed: [Revisiting Index Performance under High Cache-Miss Penalties](<https://devfeed.tech/articles/revisiting-index-performance-under-high-cache-miss-penalties-140625.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21835>)

Author: Mathew M. Kim, Beomseok Nam

Published: 2026-11-18T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [application](<https://devfeed.tech/tags/application.md>), [applications](<https://devfeed.tech/tags/applications.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cache](<https://devfeed.tech/tags/cache.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [cxl](<https://devfeed.tech/tags/cxl.md>), [designs](<https://devfeed.tech/tags/designs.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [index](<https://devfeed.tech/tags/index.md>), [indexes](<https://devfeed.tech/tags/indexes.md>), [isolation](<https://devfeed.tech/tags/isolation.md>), [latency](<https://devfeed.tech/tags/latency.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-bandwidth](<https://devfeed.tech/tags/memory-bandwidth.md>), [modern-data](<https://devfeed.tech/tags/modern-data.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

AuxBench evaluates byte-addressable indexes as part of larger application workloads, showing that index-only benchmarks can mispredict performance. Across four indexes, controlled post-lookup memory activity changed their relative performance, and designs optimized for concurrency could lose their advantage when application work competed for cache, memory bandwidth, and CPU. Time-varying request arrivals also produced severe tail latency near saturation. The abstract describes the tested factors and findings but does not name the indexes or provide quantitative results.

### Source excerpt

arXiv:2611.21835v1 Announce Type: new Abstract: Modern data-intensive applications often query very large datasets, making index performance critical. Recent index structures exploit the fine-grained access capabilities of emerging storage devices such as DCPMM and CXL. However, existing benchmarks typically evaluate indexes in isolation using saturated, closed-loop workloads that emphasize maximum throughput. Such workloads favor highly concurrent designs while overlooking the application work that follows an index lookup and competes for cache, memory bandwidth, and CPU resources. We present AuxBench - a microbenchmark that evaluates indexes as auxiliary access paths within larger application workloads. Using AuxBench, we evaluate four byte-addressable indexes under controlled post-lookup memory activity, concurrency, and request arrival patterns. Our experiments show that post-lookup memory activity can change the relative performance of competing indexes, that concurrency-oriented designs may lose their advantages when substantial application work is introduced, and that time-varying request arrivals can produce severe tail latency near saturation. These findings demonstrate why index-only benchmarks may not adequately predict performance in larger application workloads

## Accelerate Bench: Evaluating Agents for Application-Level Program Optimization

DevFeed: [Accelerate Bench: Evaluating Agents for Application-Level Program Optimization](<https://devfeed.tech/articles/accelerate-bench-evaluating-agents-for-application-level-program-optimization-137626.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20882>)

Author: Bowen Liu, Yuhan Li, Nuo Chen, Weile Luo, Miao Peng, Qintong Zhang, Qifan Zhang, Jia Li, Ziqi Gao, Lydell Huang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [App](<https://devfeed.tech/topics/app.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [application](<https://devfeed.tech/tags/application.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [build](<https://devfeed.tech/tags/build.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [cpu-optimization](<https://devfeed.tech/tags/cpu-optimization.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [faster](<https://devfeed.tech/tags/faster.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [program](<https://devfeed.tech/tags/program.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [source-code](<https://devfeed.tech/tags/source-code.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

Accelerate Bench evaluates coding agents on 32 CPU optimization tasks adapted from scientific applications and supercomputing competitions. Each task specifies a working program, workload, correctness requirements, timing boundaries, and permitted edits; agents choose their optimizations, use public tests iteratively, and submit for separate verification. Across 16 model-and-harness agents on version 1.1, Score@1 ranges from 25.5% to 67.4%. Most outputs are correct, but speedups differ: the two leaders reach at least 2x on 22 or more tasks, while the other fourteen do so on 8-17, with 125 of 412 correct outcomes below 1.25x. Controlled replays find that identical code can run at 0.29x-2.12x depending on build and runtime settings, and a code change can yield 1.01x or 6.4x depending on another change. The benchmark includes executable workloads, agent programs, and trajectories to assess both gains and optimization process.

### Source excerpt

arXiv:2611.20882v1 Announce Type: new Abstract: Application optimization requires coding agents to discover useful changes and coordinate them across a complete program while preserving the required computation. We introduce Accelerate Bench, a benchmark of 32 CPU optimization tasks adapted from scientific applications and supercomputing competitions. Each task provides a working program and specifies the workload, correctness requirements, timing boundaries, and permitted edits, leaving agents to choose where and how to optimize. Public testing supports iterative development, and a separate verifier scores the final submission. On benchmark version 1.1, we evaluate 16 coding agents, each pairing a model with a harness. Their Score@1 ranges from 25.5% to 67.4%. Most of their programs are correct, and the size of the speedup separates them: the two leading agents are at least 2x faster on 22 or more of the 32 tasks, while the other fourteen reach 2x on 8 to 17 and leave 125 of their 412 correct outcomes below 1.25x. Controlled replays show why coordination across an application matters: identical source code runs between 0.29x and 2.12x depending only on build and runtime settings, and one code change is worth 1.01x or 6.4x depending on another. Accelerate Bench pairs executable workloads with agents' programs and trajectories to evaluate both their gains and the process behind them.

## Testing at the speed of AI: what our guest, a Forrester analyst, says enterprise buyers need next

DevFeed: [Testing at the speed of AI: what our guest, a Forrester analyst, says enterprise buyers need next](<https://devfeed.tech/articles/testing-at-the-speed-of-ai-what-our-guest-a-forrester-analyst-says-enterprise-buyers-need-next-138235.md>)

Original publisher: [Read original article](<https://www.uipath.com/blog/ai/ai-speed-testing-what-forrester-analyst-says-enterprise-buyers-need-next>)

Author: Dany Benavides

Published: 2026-11-17T00:00:00Z

Content type: article

Language: en

Sources: [UiPath RSS Feed](<https://devfeed.tech/sources/uipath-rss-feed.md>)

Topics: [Test automation](<https://devfeed.tech/topics/test-automation.md>), [trust](<https://devfeed.tech/topics/trust.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [analyst](<https://devfeed.tech/tags/analyst.md>), [autonomous-testing](<https://devfeed.tech/tags/autonomous-testing.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [forrester](<https://devfeed.tech/tags/forrester.md>), [next](<https://devfeed.tech/tags/next.md>), [speed](<https://devfeed.tech/tags/speed.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

Forrester analyst Diego Lo Giudice says AI-assisted software development is making testing a bottleneck, so testing must become more automated and start earlier. Applications that include LLMs, agents, or natural-language interfaces require testing both conventional software and AI behavior through newer evaluation methods; because AI is probabilistic, the goal is trust rather than simply checking whether it works. He expects testers to shift toward setting quality standards, prioritizing work, choosing tests, and overseeing testing agents while AI handles execution. In buyer evaluations he has conducted over five years, customers have favored platforms that integrate functional, API, end-to-end, performance, data, and AI testing instead of assembling separate tools. For autonomous testing, he emphasizes visibility, a control plane, and human ability to intervene; organizational change and willingness to extend trust remain constraints. The article also promotes UiPath Test Cloud and cites one customer's choice, but provides no comparative product evaluation.

### Source excerpt

Discover Forrester analyst Diego Lo Giudice's insights on AI-powered testing, enterprise buyer priorities, and what it takes to trust autonomous testing.

## How Salesforce efficiently right-sizes Kubernetes workloads at scale on Amazon EKS

DevFeed: [How Salesforce efficiently right-sizes Kubernetes workloads at scale on Amazon EKS](<https://devfeed.tech/articles/how-salesforce-efficiently-right-sizes-kubernetes-workloads-at-scale-on-amazon-eks-136886.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/containers/how-salesforce-efficiently-right-sizes-kubernetes-workloads-at-scale-on-amazon-eks/>)

Author: Lei Liu

Published: 2026-11-16T20:40:26Z

Content type: article

Language: en

Sources: [Containers](<https://devfeed.tech/sources/containers.md>)

Topics: [Amazon Elastic Kubernetes Service](<https://devfeed.tech/topics/amazon-elastic-kubernetes-service.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [000](<https://devfeed.tech/tags/000.md>), [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-eks](<https://devfeed.tech/tags/amazon-eks.md>), [amazon-elastic-kubernetes-service](<https://devfeed.tech/tags/amazon-elastic-kubernetes-service.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [efficiently](<https://devfeed.tech/tags/efficiently.md>), [eks](<https://devfeed.tech/tags/eks.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [more](<https://devfeed.tech/tags/more.md>), [multi-tenant](<https://devfeed.tech/tags/multi-tenant.md>), [one](<https://devfeed.tech/tags/one.md>), [platforms](<https://devfeed.tech/tags/platforms.md>), [pod](<https://devfeed.tech/tags/pod.md>), [right](<https://devfeed.tech/tags/right.md>), [salesforce](<https://devfeed.tech/tags/salesforce.md>), [scale](<https://devfeed.tech/tags/scale.md>), [strategy](<https://devfeed.tech/tags/strategy.md>), [team](<https://devfeed.tech/tags/team.md>), [vertical](<https://devfeed.tech/tags/vertical.md>)

### AI overview

Salesforce reports more than $5 million in cumulative infrastructure savings after moving from static resource settings to workload-specific autoscaling across more than 1,400 Amazon EKS clusters. It uses VPA to adjust CPU and memory requests, HPA to change replica counts, or both when a workload needs independent vertical and horizontal scaling. For Istio, HPA tracks connected proxies while VPA adjusts each istiod pod's resources. The team describes staged, tested rollouts with rollback controls, monitoring, and safeguards such as eviction limits. Its guidance is to start VPA in Off mode, use RequestsOnly when adjusting resources, and keep VPA and HPA on distinct metrics. VPA recommendations use historical usage and are intended for steady-state optimization; sudden spikes need other protections. Planned expansions include in-place resizing and support for AI workloads.

### Source excerpt

Salesforce operates one of the largest multi-tenant Kubernetes platforms on Amazon EKS, spanning 1,400+ clusters. See how its team combined the Vertical Pod Autoscaler and Horizontal Pod Autoscaler into a multi-dimensional autoscaling strategy that right-sizes workloads at scale and drove more than $5 million in infrastructure savings.

## Microsoft 365 Outage: Testing Infrastructure Resilience

DevFeed: [Microsoft 365 Outage: Testing Infrastructure Resilience](<https://devfeed.tech/articles/microsoft-365-outage-testing-infrastructure-resilience-136345.md>)

Original publisher: [Read original article](<https://www.harness.io/blog/microsoft-365-outage-how-harness-resilience-testing-helps-identify-and-prevent-similar-infrastructure-failures>)

Author: Lavakush Biyani

Published: 2026-11-16T16:01:00Z

Content type: article

Language: en

Sources: [Harness](<https://devfeed.tech/sources/harness-blog.md>)

Topics: [Resilience](<https://devfeed.tech/topics/resilience.md>), [Chaos Engineering](<https://devfeed.tech/topics/chaos-engineering.md>), [Disaster Recovery](<https://devfeed.tech/topics/disaster-recovery.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Production Engineering](<https://devfeed.tech/topics/production-engineering.md>), [microsoft 365](<https://devfeed.tech/topics/microsoft-365.md>), [Microsoft](<https://devfeed.tech/topics/microsoft.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-powered](<https://devfeed.tech/tags/ai-powered.md>), [apis](<https://devfeed.tech/tags/apis.md>), [application](<https://devfeed.tech/tags/application.md>), [application-resilience](<https://devfeed.tech/tags/application-resilience.md>), [applications](<https://devfeed.tech/tags/applications.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [automate](<https://devfeed.tech/tags/automate.md>), [availability](<https://devfeed.tech/tags/availability.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [blast-radius](<https://devfeed.tech/tags/blast-radius.md>), [blog](<https://devfeed.tech/tags/blog.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [infrastructure-resilience](<https://devfeed.tech/tags/infrastructure-resilience.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [microsoft-365](<https://devfeed.tech/tags/microsoft-365.md>), [microsoft-365-outage](<https://devfeed.tech/tags/microsoft-365-outage.md>), [outage](<https://devfeed.tech/tags/outage.md>), [resilience](<https://devfeed.tech/tags/resilience.md>), [testing](<https://devfeed.tech/tags/testing.md>)

### AI overview

The text describes the November 12, 2026 Microsoft 365 disruption as a cascade involving Exchange Online server health-check failures, increased load on remaining infrastructure, saturation of a secure-connection process, and recovery complications when traffic returned. Microsoft investigated a network configuration change correlated with the incident; the text says updates reviewed did not confirm it as the root cause. It presents Harness Resilience Testing as a way to assess related risks: read-only Resilience Risk Insights reviews infrastructure and deployment data; Chaos Testing injects controlled faults; Load Testing can be combined with fault experiments to assess performance; topology mapping and blast-radius analysis examine dependency impacts; and disaster recovery experiments measure failover against RTO and RPO targets. The article also describes CI/CD integration with resilience checks and policy gates. These capabilities are presented as ways to identify weaknesses and validate behavior and recovery, not to predict the exact cause of an outage.

### Source excerpt

| Blog

## Scalability Testing: Techniques, Metrics, and Tools

DevFeed: [Scalability Testing: Techniques, Metrics, and Tools](<https://devfeed.tech/articles/scalability-testing-techniques-metrics-and-tools-133443.md>)

Original publisher: [Read original article](<https://testgrid.io/blog/scalability-testing/>)

Author: Yogesh Solanki

Published: 2026-11-13T13:14:58Z

Content type: article

Language: en

Sources: [TestGrid](<https://devfeed.tech/sources/testgrid-blog.md>)

Topics: [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Grafana](<https://devfeed.tech/topics/grafana.md>), [Prometheus](<https://devfeed.tech/topics/prometheus.md>), [k6](<https://devfeed.tech/topics/k6.md>)

Tags: [application](<https://devfeed.tech/tags/application.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [can](<https://devfeed.tech/tags/can.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [database](<https://devfeed.tech/tags/database.md>), [handle](<https://devfeed.tech/tags/handle.md>), [know](<https://devfeed.tech/tags/know.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [more](<https://devfeed.tech/tags/more.md>), [performance](<https://devfeed.tech/tags/performance.md>), [quality-assurance-qa](<https://devfeed.tech/tags/quality-assurance-qa.md>), [quickly](<https://devfeed.tech/tags/quickly.md>), [resources](<https://devfeed.tech/tags/resources.md>), [rise](<https://devfeed.tech/tags/rise.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [scalability-testing](<https://devfeed.tech/tags/scalability-testing.md>), [techniques](<https://devfeed.tech/tags/techniques.md>), [testing](<https://devfeed.tech/tags/testing.md>), [testing-techniques](<https://devfeed.tech/tags/testing-techniques.md>), [tools](<https://devfeed.tech/tags/tools.md>)

### AI overview

Scalability testing measures how performance changes as workload or capacity grows, including whether response times, throughput, stability, and resource use remain within defined goals. The excerpt outlines load, stress, spike, soak, volume, and capacity tests, plus testing vertical and horizontal scaling. Its workflow sets performance criteria, prepares a representative environment and realistic workloads, automates scripts, integrates checks into CI/CD, and monitors results to locate bottlenecks and guide changes to code, databases, caching, infrastructure, or scaling policies. Named tools differ by role: K6, Gatling, JMeter, LoadRunner, and Locust simulate workloads; Grafana, Prometheus, Datadog, New Relic, Dynatrace, and CloudWatch monitor system signals. The excerpt also describes TestGrid workflows using JMeter and mobile device performance signals. It does not provide the promised detailed metrics list or infrastructure table.

### Source excerpt

You know your application can handle 5,000 users. What happens at 10,000? More importantly, what happens between the two? Does throughput rise with the additional capacity? Do response times remain within your SLO? Does the database become the bottleneck? Does autoscaling respond quickly enough, or does performance deteriorate before new resources arrive? Scalability testing answers [...]

## VecDBLens: An Evaluatology-Based Framework for Vector Database Diagnosis

DevFeed: [VecDBLens: An Evaluatology-Based Framework for Vector Database Diagnosis](<https://devfeed.tech/articles/vecdblens-an-evaluatology-based-framework-for-vector-database-diagnosis-132691.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15105>)

Author: Yutong Zhou, Guoxin Kang, Wenxin Zhou, Haoyu He, Yuedong Zhu, Lei Wang, Wanling Gao, Fanda Fan, Jianfeng Zhan

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [Vectors & LLMs](<https://devfeed.tech/topics/vectors-llms.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [all](<https://devfeed.tech/tags/all.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [applications](<https://devfeed.tech/tags/applications.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [can](<https://devfeed.tech/tags/can.md>), [cause](<https://devfeed.tech/tags/cause.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [database](<https://devfeed.tech/tags/database.md>), [database-benchmarking](<https://devfeed.tech/tags/database-benchmarking.md>), [databases](<https://devfeed.tech/tags/databases.md>), [diagnosis](<https://devfeed.tech/tags/diagnosis.md>), [framework](<https://devfeed.tech/tags/framework.md>), [vector](<https://devfeed.tech/tags/vector.md>), [vector-database](<https://devfeed.tech/tags/vector-database.md>)

### AI overview

VecDBLens diagnoses vector database performance by comparing one retrieval function at a time under matched dataset, workload, parameter, and runtime conditions, then summarizing results across datasets and workloads. It uses ANOVA to examine interactions among functions, aiming to make benchmarking reproducible and identify whether representation, indexing, or query execution drives performance. The abstract reports that embedding quality upper-bounds retrieval performance, so apparent index failures may stem from representation limits; it also finds that overly aggressive index settings can alter neighborhood topology and reduce recall. The available text is an abstract, so it does not provide implementation details or quantify these effects.

### Source excerpt

arXiv:2611.15105v1 Announce Type: new Abstract: Vector databases are increasingly deployed as retrieval pipelines in which representation, indexing, and query-time execution jointly determine accuracy, throughput, and cost. Existing vector database evaluations primarily focus on indexing while treating embeddings as fixed inputs and ignoring the role of query-time execution, making it difficult to decide whether a bottleneck should be addressed by changing the embedding model, the index, or the query-processing functions. To address the above limitation, we introduce VecDBLens, an Evaluatology-based framework that treats the function under diagnosis as an evaluated object and all remaining dataset, workload, parameter, and runtime factors as matched evaluation conditions. VecDBLens attributes performance differences to individual evaluated objects through comparisons under matched evaluation conditions, summarizing their performance and stability across multiple datasets and workloads. It further analyzes interactions among objects using analysis of variance (ANOVA), thereby turning vector database benchmarking from leaderboard ranking into a reproducible diagnostic procedure. VecDBLens reveals two key insights. First, retrieval performance is fundamentally upper-bounded by embedding quality, revealing that many apparent indexing failures are therefore representation-side limitations. Second, stronger index parameterization is not always beneficial: overly aggressive configurations can reshape neighborhood topology and cause recall regression. These results challenge the common practice of evaluating vector databases primarily through indexing and show that real-world limitations arise from constraints across retrieval functions rather than any single function alone. VecDBLens provides a principled basis for identifying the true performance drivers of vector databases and for guiding their design in practical applications.

## Benchmarking Scikit-Ollama vs. Groq-Backed Scikit-LLM: Speed, Cost, and Accuracy

DevFeed: [Benchmarking Scikit-Ollama vs. Groq-Backed Scikit-LLM: Speed, Cost, and Accuracy](<https://devfeed.tech/articles/benchmarking-scikit-ollama-vs-groq-backed-scikit-llm-speed-cost-and-accuracy-125008.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/benchmarking-scikit-ollama-vs-groq-backed-scikit-llm-speed-cost-and-accuracy/>)

Author: Iván Palomares Carrascosa

Published: 2026-11-09T11:00:06Z

Content type: tutorial

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [groq](<https://devfeed.tech/topics/groq.md>), [SciKits](<https://devfeed.tech/topics/scikit.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cost](<https://devfeed.tech/tags/cost.md>), [data-privacy](<https://devfeed.tech/tags/data-privacy.md>), [dedicated-inference](<https://devfeed.tech/tags/dedicated-inference.md>), [groq](<https://devfeed.tech/tags/groq.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [local](<https://devfeed.tech/tags/local.md>), [ollama](<https://devfeed.tech/tags/ollama.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [speed](<https://devfeed.tech/tags/speed.md>), [vs](<https://devfeed.tech/tags/vs.md>)

### AI overview

The article compares local zero-shot sentiment classification with Ollama and Scikit-Ollama against a Groq cloud endpoint routed through Scikit-LLM, using a shared helper to measure execution time and accuracy. The local Llama 3 run classified 4 of 5 examples correctly; that tiny dataset is not a reliable basis for broad accuracy comparisons. The excerpt reports $0 cost for both tested options at the time of writing, near-instantaneous cloud inference, and local latency tied to GPU VRAM and token generation speed. Cloud setup is described as simpler, while local inference keeps data on the machine. Reproducing the benchmark requires Ollama and a downloaded model locally, or a Groq API key for the cloud run.

### Source excerpt

In this article, you will learn how to build a side-by-side benchmark of local and cloud LLM classifiers using Scikit-Ollama and a Groq-backed Scikit-LLM pipeline....

## MorphSched: Workload-Aware Adaptive Scheduler Generation for Linux

DevFeed: [MorphSched: Workload-Aware Adaptive Scheduler Generation for Linux](<https://devfeed.tech/articles/morphsched-workload-aware-adaptive-scheduler-generation-for-linux-124526.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.08509>)

Author: Yifan Zhang, Hengxuan Lou, Xinkui Zhao, Rengrong Xiong, Naibo Wang, Chang Liu, Guanjie Cheng, Shuiguang Deng, Jianwei Yin

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Linux](<https://devfeed.tech/topics/linux.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [also](<https://devfeed.tech/tags/also.md>), [applications](<https://devfeed.tech/tags/applications.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [build](<https://devfeed.tech/tags/build.md>), [building](<https://devfeed.tech/tags/building.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [code](<https://devfeed.tech/tags/code.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [compile](<https://devfeed.tech/tags/compile.md>), [complete](<https://devfeed.tech/tags/complete.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-os](<https://devfeed.tech/tags/cs-os.md>), [custom](<https://devfeed.tech/tags/custom.md>), [dag](<https://devfeed.tech/tags/dag.md>), [generation](<https://devfeed.tech/tags/generation.md>), [linux](<https://devfeed.tech/tags/linux.md>), [scheduler](<https://devfeed.tech/tags/scheduler.md>)

### AI overview

MorphSched generates complete Rust+BPF sched_ext schedulers tailored to workloads. It profiles workloads under Linux EEVDF to set optimization objectives, then uses a scaffold-constrained pipeline to produce implementations. A DAG-structured repair process addresses verifier, build, and runtime failures, while an evolutionary memory pool uses performance feedback to refine strategies. In 120 machine/workload/interferer configurations on a real Linux kernel, generated schedulers compiled, passed eBPF verification, loaded, and remained stable during benchmark runs without manual source edits. They improved the primary foreground metric over EEVDF by 26.5% on average and up to 87.3%; in selected comparisons, they were competitive with representative hand-written sched_ext schedulers.

### Source excerpt

arXiv:2611.08509v1 Announce Type: cross Abstract: Modern Linux scheduling increasingly requires workload-specific policies to meet the dynamic performance needs of heterogeneous applications. The recent sched_ext framework enables custom BPF schedulers, but building practical policies still requires coordinating Rust user-space control logic, BPF scheduling callbacks, generated user-kernel interfaces, and strict eBPF verifier constraints. This paper presents MorphSched, an LLM-driven system for synthesizing complete Rust+BPF sched_ext schedulers. MorphSched profiles target workloads under Linux EEVDF to construct dynamic, workload-specific optimization objectives, and uses a scaffold-constrained generation pipeline to translate scheduling intent into deployable scheduler implementations. To improve synthesis reliability, MorphSched separates implementation repair from strategy search: verifier, build, and runtime failures are handled through a DAG-structured repair process, while performance feedback is accumulated in an evolutionary memory pool to refine high-level scheduling strategies across iterations. We evaluate MorphSched across 120 machine/workload/interferer configurations on a real Linux kernel. The generated schedulers compile, pass eBPF verification, load successfully, and remain stable throughout our benchmark runs without manual edits to generated scheduler source code. Across these configurations, MorphSched improves the primary foreground metric over EEVDF by 26.5% on average and up to 87.3%. In selected comparisons, the generated schedulers are also competitive with representative hand-written sched_ext schedulers. These results show that scaffold-constrained, system-grounded LLM generation can be a practical tool for exploring workload-specific Linux scheduling policies.

## CoreWeave ARENA: A Practical Approach to Workload Evaluation

DevFeed: [CoreWeave ARENA: A Practical Approach to Workload Evaluation](<https://devfeed.tech/articles/coreweave-arena-a-practical-approach-to-workload-evaluation-121499.md>)

Original publisher: [Read original article](<https://wf.coreweave.com/blog/coreweave-arena-a-practical-approach-to-workload-evaluation>)

Author: CoreWeave

Published: 2026-11-06T00:29:52Z

Content type: article

Language: en

Sources: [Coreweave Blog Posts](<https://devfeed.tech/sources/coreweave-blog-posts.md>)

Topics: [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Grafana](<https://devfeed.tech/topics/grafana.md>), [Production Engineering](<https://devfeed.tech/topics/production-engineering.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [about](<https://devfeed.tech/tags/about.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-cloud](<https://devfeed.tech/tags/ai-cloud.md>), [ai-products](<https://devfeed.tech/tags/ai-products.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [all](<https://devfeed.tech/tags/all.md>), [arena](<https://devfeed.tech/tags/arena.md>), [blog](<https://devfeed.tech/tags/blog.md>), [can](<https://devfeed.tech/tags/can.md>), [coreweave](<https://devfeed.tech/tags/coreweave.md>), [cost](<https://devfeed.tech/tags/cost.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [like](<https://devfeed.tech/tags/like.md>), [performance](<https://devfeed.tech/tags/performance.md>), [practical](<https://devfeed.tech/tags/practical.md>), [production](<https://devfeed.tech/tags/production.md>), [run](<https://devfeed.tech/tags/run.md>), [scale](<https://devfeed.tech/tags/scale.md>), [understand](<https://devfeed.tech/tags/understand.md>)

### AI overview

CoreWeave ARENA evaluates teams' actual AI workloads on CoreWeave infrastructure over a few weeks, helping them assess performance, scaling, and cost before an infrastructure decision. CoreWeave experts provide a notebook-based environment using guided marimo notebooks, then help validate the setup and run workload-shaped tests that examine throughput, bottlenecks, and changes across configurations and scale. Evaluations can measure platform components such as AI Object Storage and LOTA. Metrics and artifacts can be logged with a preloaded Weights & Biases trial; runs are also visible in Mission Control, with logs and metrics in Grafana and an agent that can surface changes in tools such as Slack. Orchestration supports Kubernetes-native workflows on CKS or Slurm via SUNK on CKS. The source says ARENA is available to existing customers; applications for new teams were planned for Q2 2026.

### Source excerpt

CoreWeave ARENA lets you run your workload at production-like scale so you can understand performance and cost before committing to production.

## Migrating Booking.com's autocomplete workload at scale to OpenSearch

DevFeed: [Migrating Booking.com's autocomplete workload at scale to OpenSearch](<https://devfeed.tech/articles/migrating-booking-com-s-autocomplete-workload-at-scale-to-opensearch-118928.md>)

Original publisher: [Read original article](<https://opensearch.org/case-studies/migrating-booking-coms-autocomplete-workload-at-scale-to-opensearch/>)

Author: OpenSearch

Published: 2026-11-04T16:00:51Z

Content type: news

Language: en

Sources: [OpenSearch](<https://devfeed.tech/sources/opensearch.md>)

Topics: [opensearch](<https://devfeed.tech/topics/opensearch.md>), [Amazon OpenSearch Service](<https://devfeed.tech/topics/amazon-opensearch-service.md>), [zero downtime deployment](<https://devfeed.tech/topics/zero-downtime-deployment.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [apache](<https://devfeed.tech/tags/apache.md>), [autocomplete](<https://devfeed.tech/tags/autocomplete.md>), [booking](<https://devfeed.tech/tags/booking.md>), [case-studies](<https://devfeed.tech/tags/case-studies.md>), [custom](<https://devfeed.tech/tags/custom.md>), [downtime](<https://devfeed.tech/tags/downtime.md>), [engine](<https://devfeed.tech/tags/engine.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [languages](<https://devfeed.tech/tags/languages.md>), [latency](<https://devfeed.tech/tags/latency.md>), [lucene](<https://devfeed.tech/tags/lucene.md>), [maintenance](<https://devfeed.tech/tags/maintenance.md>), [migration](<https://devfeed.tech/tags/migration.md>), [opensearch](<https://devfeed.tech/tags/opensearch.md>), [relevance](<https://devfeed.tech/tags/relevance.md>), [scale](<https://devfeed.tech/tags/scale.md>), [search](<https://devfeed.tech/tags/search.md>)

### AI overview

Booking.com migrated autocomplete from its custom Lucene engine Brick to Amazon OpenSearch Service, preserving relevance across 42+ languages and rolling out without downtime. The team rebuilt active features in OpenSearch DSL using custom analyzers and edge n-grams, scoring functions, result collapsing, fuzzy matching, and parallel subqueries. A Kubernetes ingestion pipeline and daily index builds with Step Functions and OpenSearch ISM enabled automated blue/green alias switches. To meet an 85 ms p95 SLA after staging latency reached 300-500 ms, engineers flattened dynamic maps and enabled doc_values so rescoring could use in-memory columnar data, then tuned cluster storage and topology. They mirrored production traffic and compared Recall@k and Kendall's tau against Brick, with automated gates for incomplete index updates. A phased language and traffic rollout reached 4,000 requests per second within the SLA and removed bare-metal and NFS dependencies. The new platform also enabled experimentation with hybrid search and intent prediction.

### Source excerpt

Booking.com migrated its autocomplete workload from Brick, a custom Apache Lucene engine, to OpenSearch. The team preserved search relevance across 42+ languages while meeting strict latency requirements and completing the transition without downtime. The migration reduced infrastructure maintenance and established a foundation for more advanced search experiences driven by traveler intent. The post Migrating Booking.com's autocomplete workload at scale to OpenSearch appeared first on OpenSearch.

## Reliability Stress Testing of RAG Retrieval under Query-Relevant Corpus Poisoning

DevFeed: [Reliability Stress Testing of RAG Retrieval under Query-Relevant Corpus Poisoning](<https://devfeed.tech/articles/reliability-stress-testing-of-rag-retrieval-under-query-relevant-corpus-poisoning-117453.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.04264>)

Author: Janet Sung, Kainan Zhou, Gangzhen Qian, Hang Xiao, Lu Yi

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [information retrieval](<https://devfeed.tech/topics/information-retrieval.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>)

Tags: [admission](<https://devfeed.tech/tags/admission.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [poison](<https://devfeed.tech/tags/poison.md>), [query](<https://devfeed.tech/tags/query.md>), [rag](<https://devfeed.tech/tags/rag.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [stress](<https://devfeed.tech/tags/stress.md>)

### AI overview

Across 623 SciFact and NFCorpus queries, five query-targeted poison documents per query reduced classical retriever nDCG@10 by 40.2-61.5% in isolated-target tests. Full-query anchoring drove most BM25 degradation; appended payload text also shifted LSA and Hybrid rankings. In a pretrained BGE-small validation, query-only poisons achieved 100% target exposure on both datasets, while adding payload text had little effect on SciFact nDCG and slightly reduced NFCorpus exposure. Partial-query attacks remained effective on long SciFact queries: two random terms produced 39.5% Hybrid target exposure. Multi-target contamination caused cross-target interference. Overlap-based admission removed full-query poisons but could sharply reduce clean retrieval utility; MMR did not change exposure in the tested setup. These results concern retrieval-layer admission, not downstream instruction following.

### Source excerpt

arXiv:2611.04264v1 Announce Type: new Abstract: External text can affect a retrieval-augmented generation (RAG) system only after the retriever admits it to the model context. We stress this admission step while varying document content, attacker query information, per-target poison budget, and contamination scope. Across 623 Benchmarking Information Retrieval (BEIR) queries from SciFact and NFCorpus, five query-targeted documents per query reduce classical-retriever nDCG@10 by 40.2-61.5% in isolated-target stress. Full-query anchoring explains most Best Matching 25 (BM25) degradation, whereas appended text also shifts latent semantic analysis (LSA) and Hybrid rankings. A pretrained BGE-small validation reaches 100% target exposure for query-only P=5 on both datasets; adding the injection payload changes SciFact nDCG only from 0.269 to 0.271 and lowers NFCorpus exposure slightly to 98.8%, indicating that query anchoring remains the dominant retrieval-side factor for this dense model. Query length materially changes the interpretation of partial-information attacks: on 162 long SciFact queries, two random terms cover 19.9% of eligible terms and still yield 39.5% Hybrid target exposure. Multi-target contamination adds cross-target interference. Overlap-based admission can remove full-query poisons but may sharply reduce clean retrieval utility, and maximal marginal relevance (MMR) leaves exposure unchanged under the tested configuration. The study measures retrieval-layer failures, not downstream instruction following.

## Capacity Planning for Resource-Constrained Clinical LLM Serving

DevFeed: [Capacity Planning for Resource-Constrained Clinical LLM Serving](<https://devfeed.tech/articles/capacity-planning-for-resource-constrained-clinical-llm-serving-115135.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01152>)

Author: Yi Tang, Geoffrey Martin, Yiye Zhang, Yifan Peng

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [scheduling](<https://devfeed.tech/topics/scheduling.md>)

Tags: [biomedical](<https://devfeed.tech/tags/biomedical.md>), [bottlenecks](<https://devfeed.tech/tags/bottlenecks.md>), [budget](<https://devfeed.tech/tags/budget.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [capacity-planning](<https://devfeed.tech/tags/capacity-planning.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fleet](<https://devfeed.tech/tags/fleet.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm-serving](<https://devfeed.tech/tags/llm-serving.md>), [load](<https://devfeed.tech/tags/load.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [resource](<https://devfeed.tech/tags/resource.md>), [routing](<https://devfeed.tech/tags/routing.md>), [routing-policy](<https://devfeed.tech/tags/routing-policy.md>), [serving](<https://devfeed.tech/tags/serving.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

In a controlled four-GPU clinical NLP serving setup, fleet composition accounted for more variation than the evaluated routing policies in time to first token, latency, and long-context coverage across 24 configurations. All-small fleets gave the best responsiveness and throughput but substantially lower long-context coverage; adding larger-model capacity improved coverage at the cost of serving efficiency. Heterogeneous fleets offered intermediate results but could bottleneck when large-model capacity was scarce. In a focused bottleneck evaluation, capacity-aware routing reduced mean latency and raised throughput with a small reduction in long-context coverage. These findings support prioritizing capacity allocation and using routing to manage demand within provisioned capacity, within the measured regime.

### Source excerpt

arXiv:2611.01152v1 Announce Type: new Abstract: Deploying large language models (LLMs) under a fixed accelerator budget requires jointly deciding how computational capacity is allocated across model sizes and how incoming requests are routed across that capacity. Although prior work has extensively studied LLM routing and serving efficiency, the relative importance of fleet composition and routing under resource constraints remains less understood. We investigate this question using a controlled four-GPU serving environment with heterogeneous clinical NLP workloads spanning short factual verification, biomedical semantic matching, and long-context clinical summarization. We systematically vary fleet composition, routing policy, and offered load while holding the workload trace and hardware budget fixed. Across 24 configurations, descriptive main-effect shares attribute more variation to fleet composition than to the evaluated routing policies in time to first token, latency, and long-context coverage. All-small fleets provide the best responsiveness and throughput but substantially lower long-context coverage, whereas larger-model capacity improves coverage at the cost of serving efficiency. Heterogeneous fleets provide intermediate operating points but can introduce strong-model bottlenecks when large-model capacity is scarce. In a focused evaluation of this bottleneck, capacity-aware routing reduces mean latency and increases throughput while incurring a small reduction in long-context coverage. Within this measured regime, the results support prioritizing capacity allocation, with routing managing demand within the provisioned capacity.

## Serving Agentic Robot Fleets: Workload Characterization, KV-Cache Thrashing, and Admission Control

DevFeed: [Serving Agentic Robot Fleets: Workload Characterization, KV-Cache Thrashing, and Admission Control](<https://devfeed.tech/articles/serving-agentic-robot-fleets-workload-characterization-kv-cache-thrashing-and-admission-control-113797.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01404>)

Author: Sida He

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>)

Tags: [admission](<https://devfeed.tech/tags/admission.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [cache](<https://devfeed.tech/tags/cache.md>), [characterization](<https://devfeed.tech/tags/characterization.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [robot](<https://devfeed.tech/tags/robot.md>), [robot-fleets](<https://devfeed.tech/tags/robot-fleets.md>), [serving](<https://devfeed.tech/tags/serving.md>)

### AI overview

The measurements show that robot-fleet serving faces different capacity limits in its vision-language brain and VLA policy. Across 80 LIBERO-PRO episodes profiled on H800 GPUs, brain contexts reached 215k tokens; the median GPT-6-Astra request took 7.78 seconds, versus a 1.0-second gap between environment steps. One VLA GPU met a 1% starvation target for one robot with a 0.25-second action deadline, and supported at least 32 robots with a 1.0-second deadline. In sustained vLLM brain replay, raising concurrency from 20 to 32 cut throughput from about 2.4 to 0.6 requests per second as repetitive prefill displaced generation. A static active-episode cap restored 2.24-2.25 requests per second and increased completed trace-program throughput from 1.30 to 5.00-5.55 episodes per minute. These results show that capacity planning and admission control depend on the workload and action deadline.

### Source excerpt

arXiv:2611.01404v1 Announce Type: cross Abstract: Agentic robots combine a vision-language brain, a tool-using harness, and a vision-language-action (VLA) policy. Serving a fleet of these agents introduces two coupled resource constraints: the brain repeatedly revisits long, growing key-value (KV) cache states, while the VLA executes action chunks under tight control deadlines. We characterize 80 LIBERO-PRO episodes generated by GPT-6-Astra, RPent, and pi0.5 (1,862 brain turns and 1,582 VLA inference chunks), profile both layers on H800 graphics processing units (GPUs), and replay the resulting agent programs on a vLLM serving engine. Brain contexts reach 215k tokens; the median Astra request takes 7.78 s, while the intervening environment-step gap is only 1.0 s. Under a 0.25 s action-chunk deadline, one VLA GPU meets the 1% starvation target for one robot; under a 1.0 s deadline, its measured capacity reaches at least 32. In sustained brain replay, increasing concurrency from 20 to 32 robots reduces throughput from about 2.4 to 0.6 requests/s as repetitive prefill displaces generation. A static active-episode cap restores 2.24-2.25 requests/s and raises completed trace-program throughput from 1.30 to 5.00-5.55 episodes/min. Our measurements expose the workload-dependent shift between action deadlines and brain KV residency, and establish practical principles for capacity planning and admission in sustained agentic robot fleets.

## LLM Fine-Tuning Workloads: Characterization, Challenges, and Opportunities for Serverless AI

DevFeed: [LLM Fine-Tuning Workloads: Characterization, Challenges, and Opportunities for Serverless AI](<https://devfeed.tech/articles/llm-fine-tuning-workloads-characterization-challenges-and-opportunities-for-serverless-ai-111746.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.40983>)

Author: Radu Nicolae, Daniele Lotito, Alexandru Iosup

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [characterization](<https://devfeed.tech/tags/characterization.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [failure](<https://devfeed.tech/tags/failure.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [llm-fine-tuning](<https://devfeed.tech/tags/llm-fine-tuning.md>), [moe](<https://devfeed.tech/tags/moe.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [roce](<https://devfeed.tech/tags/roce.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

An analysis of 30,920 LLM fine-tuning experiments finds that feasibility and performance vary sharply with workload and deployment. Failure rates rise from 52% for batch sizes 16-64 to 95% at batch size 1,024, while throughput grows logarithmically with batch size. Scaling is near-linear within a node; for inter-node runs, RoCE raises efficiency to 0.72 versus 0.26 without it. The reported system-level RoCE optimization reaches up to 2.9x improvement, compared with 1.7x for the application-level Fast MoE optimization. The study covers four GPU types and setups from 1 to 32 GPUs, and its dataset includes both successful and failing configurations.

### Source excerpt

arXiv:2610.40983v1 Announce Type: new Abstract: Serverless execution of Large Language Models (LLMs) is becoming important, reflecting their wide societal use. An increasingly present stage in LLM pipelines is fine-tuning, which specializes trained models and improves inference performance. Serverless operation of LLM fine-tuning workloads shifts operational concerns from the user to the provider. For these new workloads, providers face important operational complexities, such as performance issues, resource waste, and even service failure due to misconfiguration. However, optimizing such workloads is still at an early stage, and, relevant to this work, relatively little is currently known about how LLM fine-tuning workloads run in practice. In this work, we address the knowledge gap by characterizing the execution of LLM fine-tuning workloads on clusters equipped with different GPUs, running LLM models at various scales. We analyze IBM's LLMFineTuningBench, a large-scale, diverse profiling dataset of 30,920 fine-tuning experiments that records both successful and failing configurations. We provide quantitative insight into throughput (expressed in tokens/s), scaling, failure, and potential for performance optimization for four commonly used GPU types, in setups of 1 to 32 GPUs. We find that (1) job feasibility (run or fail) depends on workload properties, with failure ratios rising from 52% for small batch sizes (16-64) up to 95% for the largest (1,024); (2) throughput grows logarithmically with batch size; (3) scaling is near-linear when the workload is deployed intra-node, and optimizations such as RoCE substantially improves efficiency for inter-node deployment (e.g., 0.72 vs. 0.26 efficiency without RoCE); and (4) system-level optimizations can outweigh application-level ones (RoCE, up to 2.9$\times$, vs. Fast MoE, 1.7$\times$).

## GAUSS: A Workload-Aware Analytical and Monte-Carlo Simulator for LLM Inference Serving

DevFeed: [GAUSS: A Workload-Aware Analytical and Monte-Carlo Simulator for LLM Inference Serving](<https://devfeed.tech/articles/gauss-a-workload-aware-analytical-and-monte-carlo-simulator-for-llm-inference-serving-107999.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37989>)

Author: Srikant Bharadwaj, Ankur Mallick, Spyridon Mastorakis, Anjaly Parayil, Dhruv Deshmukh, Victor Ruehle

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [scheduling](<https://devfeed.tech/topics/scheduling.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [batching](<https://devfeed.tech/tags/batching.md>), [continuous-batching](<https://devfeed.tech/tags/continuous-batching.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [decode](<https://devfeed.tech/tags/decode.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [inference](<https://devfeed.tech/tags/inference.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [prefill](<https://devfeed.tech/tags/prefill.md>), [routing](<https://devfeed.tech/tags/routing.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [serving](<https://devfeed.tech/tags/serving.md>), [simulator](<https://devfeed.tech/tags/simulator.md>), [slos](<https://devfeed.tech/tags/slos.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [time-to-first-token](<https://devfeed.tech/tags/time-to-first-token.md>), [ttft](<https://devfeed.tech/tags/ttft.md>)

### AI overview

GAUSS estimates the full time-to-first-token (TTFT) and time-between-tokens (TBT) latency distributions and sustainable request rate for a workload, serving engine, and accelerator. It combines calibrated, phase-specific prefill and decode latency surfaces with a Lindley-recurrence queueing model for stochastic arrivals, using ensemble Monte Carlo sampling rather than replaying one trace. The authors say this approach can estimate latency tails that trace-driven simulators may under-sample and evaluate undeployed workloads, routing policies, and hardware. They describe applications to operating-point selection, traffic segmentation, and capacity questions for shortest-remaining-processing-time scheduling. The supplied text is an abstract and does not provide evaluation details or quantitative results.

### Source excerpt

arXiv:2610.37989v1 Announce Type: new Abstract: Large language model (LLM) inference has become an operational workload that fleets must provision, monitor, and bill against latency service-level objectives (SLOs). A single throughput number cannot answer the questions operators actually face, because generative serving couples two phases with very different service characteristics, continuous batching reshapes the effective service process, and user-visible tail latency can degrade long before aggregate throughput saturates. We present GAUSS, a workload-aware analytical and Monte-Carlo simulator that predicts the full distributions of time-to-first-token (TTFT) and time-between-tokens (TBT), together with the sustainable request rate, for a given workload, engine, and accelerator. GAUSS pairs calibrated, phase-specific latency surfaces for prefill and decode with a Lindley-recurrence model of queueing under stochastic arrivals and estimates performance by ensemble Monte-Carlo sampling rather than by replaying a single trace. This lets GAUSS reason about workloads, routing policies, and hardware that have never been deployed, and to capture the latency tails that trace-driven simulators systematically under-sample. We describe the model and show how it supports operating-point selection, traffic segmentation, and shortest-remaining-processing-time scheduling capacity questions that trace-driven simulators cannot address.

## Dell XPS 16 Creator Edition Review: RTX Spark N1X Holds Full Speed in a 17.8mm Chassis

DevFeed: [Dell XPS 16 Creator Edition Review: RTX Spark N1X Holds Full Speed in a 17.8mm Chassis](<https://devfeed.tech/articles/dell-xps-16-creator-edition-review-rtx-spark-n1x-holds-full-speed-in-a-17-8mm-chassis-106869.md>)

Original publisher: [Read original article](<https://www.storagereview.com/review/dell-xps-16-creator-edition-review-rtx-spark-n1x>)

Author: Brian Beeler

Published: 2026-10-29T21:02:44Z

Content type: article

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [dell](<https://devfeed.tech/topics/dell.md>), [NVIDIA RTX](<https://devfeed.tech/topics/nvidia-rtx.md>), [M.2 NVMe](<https://devfeed.tech/topics/m-2-nvme.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>), [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-developer](<https://devfeed.tech/tags/ai-developer.md>), [aluminum](<https://devfeed.tech/tags/aluminum.md>), [battery](<https://devfeed.tech/tags/battery.md>), [chip](<https://devfeed.tech/tags/chip.md>), [dell](<https://devfeed.tech/tags/dell.md>), [end-user](<https://devfeed.tech/tags/end-user.md>), [first](<https://devfeed.tech/tags/first.md>), [hdmi](<https://devfeed.tech/tags/hdmi.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [n1x](<https://devfeed.tech/tags/n1x.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [oled](<https://devfeed.tech/tags/oled.md>), [panel](<https://devfeed.tech/tags/panel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [review](<https://devfeed.tech/tags/review.md>), [rtx](<https://devfeed.tech/tags/rtx.md>), [rtx-spark](<https://devfeed.tech/tags/rtx-spark.md>), [usb4](<https://devfeed.tech/tags/usb4.md>), [workstation](<https://devfeed.tech/tags/workstation.md>), [xps](<https://devfeed.tech/tags/xps.md>)

### AI overview

The Dell XPS 16 Creator Edition sustains full RTX Spark N1X performance in a 17.8mm chassis, with the review finding no throttling during a 30-minute loop. Its 64GB configuration trails the HP OmniBook Ultra 16 by about 5% in multi-core CPU performance, runs local AI workloads within 2-8% where its 30.5GB GPU memory can fit them, and leads the tested N1X notebooks in single-core performance and memory bandwidth. The 3.2K Tandem OLED and ports--including three USB4 ports, HDMI 2.1b and a full-size SD reader--stand out; service access is unusually straightforward, with replaceable USB-C port modules. The main tradeoff is storage: the $4,799.99 1TB configuration uses a Gen4 SSD, roughly half the OmniBook's Gen5 sequential speeds. Its 64GB memory cannot run the 35B coding-agent tier; Dell offers 128GB and a 2TB Gen5 SSD only in the $6,999.99 configuration.

### Source excerpt

The Dell XPS 16 Creator Edition is the first XPS built on NVIDIA's RTX Spark N1X, and Dell has put the chip in what it calls the thinnest high-performance XPS it has made: a 17.8mm CNC aluminum chassis with a 3.2K Tandem OLED touch panel, a 99Wh battery, three USB4 ports, HDMI 2.1b, and a The post Dell XPS 16 Creator Edition Review: RTX Spark N1X Holds Full Speed in a 17.8mm Chassis appeared first on StorageReview.com.

## Nutanix AHV Earns NVIDIA Certification for AI Workloads

DevFeed: [Nutanix AHV Earns NVIDIA Certification for AI Workloads](<https://devfeed.tech/articles/nutanix-ahv-earns-nvidia-certification-for-ai-workloads-107664.md>)

Original publisher: [Read original article](<https://www.nutanix.com/blog/nutanix-ahv-earns-nvidia-certification>)

Author: Luke Congdon

Published: 2026-10-29T08:00:00Z

Content type: article

Language: en

Sources: [Nutanix](<https://devfeed.tech/sources/nutanix.md>)

Topics: [nutanix](<https://devfeed.tech/topics/nutanix.md>), [AI Platform](<https://devfeed.tech/topics/ai-platform.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [accelerated](<https://devfeed.tech/tags/accelerated.md>), [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-adoption](<https://devfeed.tech/tags/ai-adoption.md>), [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [ai-factories](<https://devfeed.tech/tags/ai-factories.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-operations](<https://devfeed.tech/tags/ai-operations.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [blog-category](<https://devfeed.tech/tags/blog-category.md>), [certification](<https://devfeed.tech/tags/certification.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-environments](<https://devfeed.tech/tags/cloud-environments.md>), [cloud-platform](<https://devfeed.tech/tags/cloud-platform.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-accelerated](<https://devfeed.tech/tags/gpu-accelerated.md>), [nutanix](<https://devfeed.tech/tags/nutanix.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

Nutanix AHV has been certified under NVIDIA's Certified Hypervisors program, which validates virtualization platforms for representative AI and accelerated-computing workloads. The source says testing covers compute performance, memory efficiency, data-path optimization and LLM inference behavior. AHV's certification demonstrates its ability to expose hardware topology and apply key GPU-computing optimizations, supporting AI workloads on a single host or across multiple hosts for distributed inference. Nutanix presents this as a way to evaluate and operationalize AI within its existing cloud operating model; the claims apply to supported configurations.

### Source excerpt

Nutanix AHV earns certification under NVIDIA's Certified Hypervisors program, delivering validated GPU-accelerated AI performance on Nutanix Cloud Platform.

## AOS-Bench: Can LLM Agents Manage Dynamic Workloads?

DevFeed: [AOS-Bench: Can LLM Agents Manage Dynamic Workloads?](<https://devfeed.tech/articles/aos-bench-can-llm-agents-manage-dynamic-workloads-103038.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.33178>)

Author: Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su, Hongning Wang

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [scheduling](<https://devfeed.tech/topics/scheduling.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [all](<https://devfeed.tech/tags/all.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [bench](<https://devfeed.tech/tags/bench.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [best](<https://devfeed.tech/tags/best.md>), [bug](<https://devfeed.tech/tags/bug.md>), [bug-fix](<https://devfeed.tech/tags/bug-fix.md>), [build](<https://devfeed.tech/tags/build.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [comes](<https://devfeed.tech/tags/comes.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-agents](<https://devfeed.tech/tags/llm-agents.md>), [manage](<https://devfeed.tech/tags/manage.md>)

### AI overview

AOS-Bench measures whether coding-agent systems can manage a stream of verified tasks under shared CPU and memory limits, comparing stream delivery with each system's one-at-a-time results. Across five model-harness pairs, heavy workloads reduced success by 13-23 points; on medium workloads, systems close in isolated success diverged substantially in stream delivery. Policy interventions linked much of the gap to task intake: serial dispatch left 73% of heavy-workload tasks unclaimed. A fixed concurrency of two performed within four points of the best agent-managed policy on medium and heavy workloads. Median CPU use stayed below 30% of provisioned cores across policies, even while tasks waited.

### Source excerpt

arXiv:2610.33178v2 Announce Type: replace Abstract: Coding agents are moving from handling one request at a time to serving as persistent assistants to which users hand a stream of work: a build, a data-processing job and a bug fix may all be pending on the same machine at once. Such an agent must decide when to take on new tasks, how to delegate and overlap them, and how to stay within the machine's CPU and memory. Existing benchmarks cannot measure this, because they evaluate each task in isolation. We introduce AOS-Bench, which evaluates an agent system as a workload manager: verified tasks arrive over time and must be delivered under shared CPU and memory limits, and the same tasks run one at a time serve as a per-system reference. Across five model--harness pairs, isolated task success does not carry over evenly. On heavy workloads every system loses 13 to 23 points. On medium workloads, systems span 11 points in isolated success but 26 points in stream success, and two systems 0.6 points apart in isolation deliver 54\% and 68\% of the same tasks. Execution-policy interventions trace much of this gap to task intake: serial dispatch leaves 73\% of heavy-workload tasks unclaimed. Yet more elaborate management does not necessarily help. A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads, and under every policy median CPU use stays below 30\% of the provisioned cores while tasks wait. These results identify workload management as a distinct capability for agents, and AOS-Bench as a way to measure it.

## Speeding up Log-Sum-Exp: Kernel Fusion at the Memory Wall, Integer Arithmetic at the Compute Wall

DevFeed: [Speeding up Log-Sum-Exp: Kernel Fusion at the Memory Wall, Integer Arithmetic at the Compute Wall](<https://devfeed.tech/articles/speeding-up-log-sum-exp-kernel-fusion-at-the-memory-wall-integer-arithmetic-at-the-compute-wall-101608.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.33647>)

Author: Lingyun Yao, Martin Andraud, Niki Loppi, Andrea Pilzer, Anji Liu, Guy Van den Broeck, Martin Trapp

Published: 2026-10-27T04:00:00Z

Content type: news

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [cpu](<https://devfeed.tech/topics/cpu.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [3](<https://devfeed.tech/tags/3.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [accelerate](<https://devfeed.tech/tags/accelerate.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bottlenecks](<https://devfeed.tech/tags/bottlenecks.md>), [call](<https://devfeed.tech/tags/call.md>), [compute](<https://devfeed.tech/tags/compute.md>), [computing](<https://devfeed.tech/tags/computing.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cpus](<https://devfeed.tech/tags/cpus.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [exp](<https://devfeed.tech/tags/exp.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-wall](<https://devfeed.tech/tags/memory-wall.md>), [up](<https://devfeed.tech/tags/up.md>)

### AI overview

The paper reports two drop-in kernels for faster Log-Sum-Exp (LSE): FuseLSE uses a single-pass GPU kernel to reduce launch overhead and high-bandwidth memory traffic, evaluating exp and log on the GPU's Special Function Unit; IntLSE replaces LSE computation with approximated integer arithmetic for CPUs, where exp and log are costly. Benchmarks across workload sizes and hardware platforms report GPU speedups over PyTorch of about 10-11x for small workloads and 6x for larger ones. On CPUs, IntLSE is reported to run about 3-6x faster than FuseLSE. These are abstract-level reported results; no further benchmark details are supplied.

### Source excerpt

arXiv:2610.33647v1 Announce Type: new Abstract: While the Log-Sum-Exp (LSE) function is a foundational numerical primitive for stable log-domain computation in modern machine learning, its fast and efficient computation on generic platforms (\ie, CPUs and GPUs) has been underexplored, potentially creating execution bottlenecks for large workloads. On GPU, the de facto implementation torch.logsumexp dispatches a chain of nine sub-kernels per call, incurring redundant High Bandwidth Memory (HBM) traffic and launch overhead. On CPU, exp and log operations are hardware-hungry, limiting the computing efficiency. To address these bottlenecks, we propose two drop-in kernels to significantly accelerate LSE computation on both GPUs and CPUs: FuseLSE, a single-pass fused kernel that evaluates LSE's exp and log functions on the GPU's Special Function Unit (SFU); and IntLSE, which further replaces the LSE computation with approximated integer arithmetic for CPU computation. Our experiments, validated by benchmarks across workload sizes and hardware platforms, show that both kernels are ${\sim}10-11\times$ faster than PyTorch at small workloads and ${\sim}6\times$ faster on larger ones on GPUs, where the speed-up is attributed to reduced launch overhead and HBM traffic. On CPU, where no SFU is available, LSE computation dominates the cost, allowing IntLSE to gain ${\sim}3-6\times$ execution speed over FuseLSE.

## Workload-Aware Monitoring for AI-RAN

DevFeed: [Workload-Aware Monitoring for AI-RAN](<https://devfeed.tech/articles/workload-aware-monitoring-for-ai-ran-97343.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.31627>)

Author: Babak Azkaei, Georgios Exarchakos, Kishor Chandra Joshi

Published: 2026-10-23T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [gpu monitoring](<https://devfeed.tech/topics/gpu-monitoring.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [dispatch](<https://devfeed.tech/tags/dispatch.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>)

### AI overview

Workload-aware monitoring compares radio timings only after associating workload descriptors with processing paths, timing boundaries, and operating context. Across 16 over-the-air downlink experiments, dispatching transport blocks below 4,096 bytes to the CPU and larger blocks to the GPU produced lower mean host-callback durations than direct GPU dispatch: 29.66% lower without a synthetic GPU co-runner and 22.93% lower with one. Grouping attributed the observed separation to small blocks; large-block callback-duration ranges overlapped. The results suggest complementary CPU/GPU dispatch as a candidate, not a demonstrated general improvement.

### Source excerpt

arXiv:2610.31627v1 Announce Type: new Abstract: AI-RAN platforms are expected to share CPU and GPU resources between radio workloads and AI services. Aggregate telemetry reports operating conditions, but timing changes may reflect different workloads, processors, or measurement boundaries. We propose workload-aware monitoring that associates radio-work descriptors with processing paths, timing boundaries, and operating context before comparing like with like. In 16 over-the-air downlink experiments, hybrid dispatch (CPU for transport blocks (TBs) below 4,096 bytes and GPU otherwise) has 29.66% and 22.93% lower mean host-callback durations than direct GPU dispatch, without and with a synthetic GPU co-runner, respectively. Grouping locates the separation in small TBs, while large-TB callback-duration ranges overlap. These observations identify a candidate for complementary CPU/GPU use.

## VMware Migration Testing Is Critical to Completing Your Exit

DevFeed: [VMware Migration Testing Is Critical to Completing Your Exit](<https://devfeed.tech/articles/vmware-migration-testing-is-critical-to-completing-your-exit-90886.md>)

Original publisher: [Read original article](<https://www.verge.io/blog/vmwareexit/vmware-migration-testing/>)

Author: George Crump

Published: 2026-10-20T15:17:21Z

Content type: article

Language: en

Sources: [VergeIO](<https://devfeed.tech/sources/vergeio.md>)

Topics: [data migrations](<https://devfeed.tech/topics/data-migrations.md>), [cloud-migration](<https://devfeed.tech/topics/cloud-migration.md>), [Performance Testing](<https://devfeed.tech/topics/performance-testing.md>)

Tags: [2-2](<https://devfeed.tech/tags/2-2.md>), [2026](<https://devfeed.tech/tags/2026.md>), [22](<https://devfeed.tech/tags/22.md>), [about](<https://devfeed.tech/tags/about.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [critical](<https://devfeed.tech/tags/critical.md>), [exit](<https://devfeed.tech/tags/exit.md>), [first](<https://devfeed.tech/tags/first.md>), [image](<https://devfeed.tech/tags/image.md>), [migration](<https://devfeed.tech/tags/migration.md>), [migration-testing](<https://devfeed.tech/tags/migration-testing.md>), [snapshots](<https://devfeed.tech/tags/snapshots.md>), [sql-server](<https://devfeed.tech/tags/sql-server.md>), [takes](<https://devfeed.tech/tags/takes.md>), [testing](<https://devfeed.tech/tags/testing.md>), [vergeos](<https://devfeed.tech/tags/vergeos.md>), [vergeos-26-2](<https://devfeed.tech/tags/vergeos-26-2.md>), [virtual-data-center](<https://devfeed.tech/tags/virtual-data-center.md>), [vmware](<https://devfeed.tech/tags/vmware.md>), [vmware-exit](<https://devfeed.tech/tags/vmware-exit.md>), [vmware-migration](<https://devfeed.tech/tags/vmware-migration.md>), [vmwareexit](<https://devfeed.tech/tags/vmwareexit.md>)

### AI overview

VergeOS 26.2 combines migration speed improvements with snapshots and isolated Virtual Data Centers (VDCs) to test workloads before production cutover. In the vendor's lab, transfers ran 2.2 times faster than on VergeOS 26.1.8; VergeFS shared-disk support also enables migrations of Microsoft Failover Clustering workloads. The original VMware image is retained, and snapshots take under a second with little capacity impact, allowing tests to be reset without copying the data again. Assigning each workload its own VDC isolates tests, lets administrators cap CPU and GPU use, and supports parallel testing from identical snapshots without network conflicts. For clustered applications, testers can validate node failover and shared-disk behavior before approving cutover. The VDC that passes testing can become production with its VMs and configuration intact. These are capabilities and performance claims described by VergeIO; the excerpt provides no independent validation.

### Source excerpt

VMware migration testing decides whether your exit finishes. VergeOS keeps the VMware image, takes instant snapshots and isolates each workload in a VDC. The post VMware Migration Testing Is Critical to Completing Your Exit appeared first on VergeIO.

[Next page](<https://devfeed.tech/topics/performance-testing.md?cursor=WyIyMDI2LTEwLTIwVDE1OjE3OjIxKzAwOjAwIiwgImE3NzYyY2Y3LTA4NDEtNGYwYS1iNDQ4LTk1N2NmYjg3ZDNhNCJd>)