# Why Other Requests Change Your LLM's Output

DevFeed: [Why Other Requests Change Your LLM's Output](<https://devfeed.tech/articles/why-other-requests-change-your-llm-s-output-110132.md>)

Original publisher: [Read original article](<https://blog.dailydoseofds.com/p/why-other-requests-change-your-llms>)

Author: Avi Chawla

Published: 2026-10-31T20:21:59Z

Content type: article

Language: en

Sources: [Daily Dose of Data Science](<https://devfeed.tech/sources/daily-dose-of-data-science.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [llamaindex](<https://devfeed.tech/topics/llamaindex.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>)

Tags: [a2a](<https://devfeed.tech/tags/a2a.md>), [ag-ui](<https://devfeed.tech/tags/ag-ui.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [agent-tasks](<https://devfeed.tech/tags/agent-tasks.md>), [alibaba](<https://devfeed.tech/tags/alibaba.md>), [api](<https://devfeed.tech/tags/api.md>), [arena](<https://devfeed.tech/tags/arena.md>), [artifacts](<https://devfeed.tech/tags/artifacts.md>), [attention](<https://devfeed.tech/tags/attention.md>), [backend](<https://devfeed.tech/tags/backend.md>), [batch](<https://devfeed.tech/tags/batch.md>), [batching](<https://devfeed.tech/tags/batching.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [harness](<https://devfeed.tech/tags/harness.md>), [harness-engineering](<https://devfeed.tech/tags/harness-engineering.md>), [llm](<https://devfeed.tech/tags/llm.md>), [update](<https://devfeed.tech/tags/update.md>)

## AI overview

Identical prompts can produce different outputs at temperature 0 because GPU kernels change the order of floating-point reductions according to batch size; near-tied token scores can then select different next tokens, causing completions to diverge. The forward pass remains repeatable for an identical batch, but production continuous batching changes batch composition across decoding steps. Batch-invariant kernels use a fixed reduction order to make outputs independent of batch composition. Thinking Machines Lab reports that this made all 1,000 sampled Qwen3-235B completions identical. The tradeoff is lower throughput: SGLang's cited benchmarks show typical slowdowns of 25-45%; vLLM's option is in beta, and DeepSeek's DeepInfer option supports NVIDIA GPUs only. For reproducible evaluations or RL rollouts, the text recommends enabling the relevant engine flag and treating batch composition as part of the experimental setup.

## Source excerpt

+ an update on Harness Engineering 2.0.