# ArtifactArena: Evaluating Models by What They Build in the Physical World

DevFeed: [ArtifactArena: Evaluating Models by What They Build in the Physical World](<https://devfeed.tech/articles/artifactarena-evaluating-models-by-what-they-build-in-the-physical-world-99842.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.06511>)

Author: Kushagra Tiwary, David Mayo, Nikhil Behari, Xiangzhou Sun, Abdulrahman Alabdulkareem, Isaac Galatzer-Levy, Boris Katz, Brian Cheung

Published: 2026-10-26T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Robotics Simulation](<https://devfeed.tech/topics/robotics-simulation.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>)

Tags: [artifacts](<https://devfeed.tech/tags/artifacts.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bot](<https://devfeed.tech/tags/bot.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [frontier-models](<https://devfeed.tech/tags/frontier-models.md>), [functional](<https://devfeed.tech/tags/functional.md>), [gameplay](<https://devfeed.tech/tags/gameplay.md>), [hardware-software-co-design](<https://devfeed.tech/tags/hardware-software-co-design.md>), [physical-design](<https://devfeed.tech/tags/physical-design.md>), [platform](<https://devfeed.tech/tags/platform.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [robots](<https://devfeed.tech/tags/robots.md>), [simulator](<https://devfeed.tech/tags/simulator.md>), [testbed](<https://devfeed.tech/tags/testbed.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

## AI overview

ArtifactArena evaluates models by the robots they engineer for a simulated arena. It uses three harnesses to assess zero-shot design, verifier-guided refinement, and open-ended physical design, refining robot artifacts using text descriptions, physics simulator feedback, or gameplay data. Artifacts compete in head-to-head tournaments that produce Elo rankings. The platform and tournament infrastructure are presented as an open testbed for ongoing community submissions. This overview is based on the supplied abstract.

## Source excerpt

arXiv:2610.06511v2 Announce Type: replace-cross Abstract: To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We introduce ArtifactArena, an open-ended platform where models face a physically grounded hardware-software co-design challenge: engineering fully functional robots to compete in a simulated arena. We evaluate a frontier model's zero-shot, verifier guided refinement, and open-ended physical design capabilities through three harnesses that refine their bot artifacts based on text descriptions, physics simulator feedback, and gameplay data. We benchmark these capabilities with an Elo ranking of frontier models derived from head-to-head tournaments between their artifacts. By releasing this framework and tournament infrastructure for ongoing community submissions, we establish a living, non-saturating testbed to continuously measure the expanding limits of open-ended intelligence in the physical world. Please visit https://artifactarena.ai for more information.