# When Log Parsing Removes What Matters: Detecting Failures in Distributed GPU Workloads

DevFeed: [When Log Parsing Removes What Matters: Detecting Failures in Distributed GPU Workloads](<https://devfeed.tech/articles/when-log-parsing-removes-what-matters-detecting-failures-in-distributed-gpu-workloads-107356.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.14009>)

Author: Boyuan Cui, Yuyang Zeng, Yifang Tian, Hans-Arno Jacobsen

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [anomaly-detection](<https://devfeed.tech/tags/anomaly-detection.md>), [bench](<https://devfeed.tech/tags/bench.md>), [communication](<https://devfeed.tech/tags/communication.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [detection](<https://devfeed.tech/tags/detection.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [execution](<https://devfeed.tech/tags/execution.md>), [failure](<https://devfeed.tech/tags/failure.md>), [fault-injection](<https://devfeed.tech/tags/fault-injection.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [llm](<https://devfeed.tech/tags/llm.md>), [logs](<https://devfeed.tech/tags/logs.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [out-of-memory](<https://devfeed.tech/tags/out-of-memory.md>), [parsing](<https://devfeed.tech/tags/parsing.md>)

## AI overview

GF-Bench is a PyTorch framework and dataset for evaluating failure detection in distributed GPU workloads. Its current release contains 220 labeled two-GPU runs covering normal behavior, CUDA out-of-memory fail-stop faults, and persistent-straggler fail-slow faults. On parsed logs, DeepLog, LogAnomaly, LogBERT, and LogPrompt with in-context learning or chain-of-thought prompting did not reliably detect both fault classes; the highest overall F1 was 0.667. For LogPrompt, using raw logs raised ICL's overall F1 to 0.983, but required 9.0 times the average input tokens and 8.9 times the per-run cost. The results suggest parsing can lower token use and inference cost while discarding execution context needed for accurate detection. The framework is designed to extend to other workloads and fault types.

## Source excerpt

arXiv:2610.14009v2 Announce Type: replace-cross Abstract: Large AI models increasingly rely on distributed GPU execution, yet it remains unclear whether established anomaly detection methods transfer effectively to this setting. We introduce GF-Bench, a PyTorch-based framework for generating controlled failures in distributed GPU workloads, and a dataset produced with it. The framework supports extensible workload and fault-injection logic; based on this design, the current release targets failures visible in multi-GPU communication logs and provides 220 labeled two-GPU runs covering normal behavior, CUDA out-of-memory fail-stop faults, and persistent-straggler fail-slow faults. Using GF-Bench, we evaluate three sequence-based methods (DeepLog, LogAnomaly, and LogBERT) and the large language model (LLM)-based LogPrompt with in-context learning (ICL) and chain-of-thought (CoT) prompting strategies on parsed logs, and additionally evaluate LogPrompt on raw logs. On parsed inputs, no method reliably detects both failure classes, and the highest overall F1 score is 0.667. Raw inputs raise the overall F1 score of ICL-LogPrompt to 0.983, at 9.0x its average input-token usage and 8.9x its per-run cost. These results reveal a trade-off between effectiveness and efficiency: parsed inputs reduce LLM token use and inference cost but can discard the distributed execution context required for accurate failure detection. GF-Bench provides a reproducible basis for studying cost-efficient anomaly detection methods and can be extended to additional GPU workloads and failure types.