# SteelAgent: LLM-Orchestrated, Physics-Informed Steel Property Prediction with Audited Generalization and Calibrated Uncertainty

DevFeed: [SteelAgent: LLM-Orchestrated, Physics-Informed Steel Property Prediction with Audited Generalization and Calibrated Uncertainty](<https://devfeed.tech/articles/steelagent-llm-orchestrated-physics-informed-steel-property-prediction-with-audited-generalization-and-calibrated-uncertainty-120540.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06953>)

Author: Aleksandr Volkov, Roman Sultimov, Mikhail Kuzin, Yury Maximov

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Calibrated uncertainty](<https://devfeed.tech/topics/calibrated-uncertainty.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Feature Engineering](<https://devfeed.tech/topics/feature-engineering.md>)

Tags: [boosting](<https://devfeed.tech/tags/boosting.md>), [calibrated-uncertainty](<https://devfeed.tech/tags/calibrated-uncertainty.md>), [compliance](<https://devfeed.tech/tags/compliance.md>), [cond-mat-mtrl-sci](<https://devfeed.tech/tags/cond-mat-mtrl-sci.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [errors](<https://devfeed.tech/tags/errors.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [features](<https://devfeed.tech/tags/features.md>), [inverse-design](<https://devfeed.tech/tags/inverse-design.md>), [laboratory](<https://devfeed.tech/tags/laboratory.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [llm](<https://devfeed.tech/tags/llm.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [physics](<https://devfeed.tech/tags/physics.md>), [physics-informed](<https://devfeed.tech/tags/physics-informed.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [property](<https://devfeed.tech/tags/property.md>), [regression](<https://devfeed.tech/tags/regression.md>), [steel](<https://devfeed.tech/tags/steel.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [uncertainty](<https://devfeed.tech/tags/uncertainty.md>)

## AI overview

SteelAgent combines metallurgical descriptors and machine-learning models with a language-model interface for steel property prediction, uncertainty reporting, and inverse design. In SteelBench tensile-strength evaluations, gradient-boosting performance depended sharply on the held-out population: pooled out-of-fold R² dropped from 0.859 on a random split to 0.113 when steel families were held out. Nominal 90% prediction intervals achieved 91.0% empirical coverage under random splitting but 37.6% on held-out sources. Physics-derived features also had mixed effects, raising R² by 0.2534 in a 17-family holdout analysis while lowering it by 0.358 in a separate four-grade laboratory analysis. The authors report group-level errors and a synthetic regression experiment on calibration deterioration, and argue that property estimates or compliance scores should be accompanied by the evaluation population. These findings are specific to the reported evaluations.

## Source excerpt

arXiv:2611.06953v1 Announce Type: new Abstract: Steel property models can perform well on random test splits even when they extrapolate poorly to unfamiliar alloys. SteelAgent couples metallurgical descriptors and machine-learning models with a language-model interface for property prediction, uncertainty reporting and inverse design. We examine how the choice of held-out unit changes the conclusions drawn from SteelBench tensile-strength evaluations. For gradient boosting, pooled out-of-fold $R^2$ fell from 0.859 on a random split to 0.113 when steel families were held out. Empirical coverage of nominal 90% prediction intervals ranged from 91.0% under random splitting to 37.6% on held-out sources. The effect of physics-derived features depended on the evaluation. They increased gradient-boosting $R^2$ by 0.2534 in a 17-family holdout analysis and decreased it by 0.358 in a separate four-grade laboratory analysis. We report group-level errors and a synthetic regression experiment illustrating how calibration can deteriorate on held-out groups. These results motivate reporting the evaluation population alongside a property estimate or compliance score.