# ScarfBench: Benchmarking AI Agents for Enterprise Java Framework Migration

DevFeed: [ScarfBench: Benchmarking AI Agents for Enterprise Java Framework Migration](<https://devfeed.tech/articles/scarfbench-benchmarking-ai-agents-for-enterprise-java-framework-migration-7269.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ibm-research/scarfbench>)

Author: Raju Pavuluri; Rahul Krishna; Srikanth Govindaraj Tamilselvam; Bridget M; Ashita Saxena; George Safta; Advait Pavuluri; Michele Merler

Published: 2026-06-30T18:32:50Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [migration](<https://devfeed.tech/topics/migration.md>), [benchmarking](<https://devfeed.tech/topics/benchmarking.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Software Engineering](<https://devfeed.tech/topics/software-engineering.md>), [coding](<https://devfeed.tech/topics/coding.md>), [Jakarta EE](<https://devfeed.tech/topics/jakarta-ee.md>), [Quarkus](<https://devfeed.tech/topics/quarkus.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [Refactoring](<https://devfeed.tech/topics/refactoring.md>), [configuration](<https://devfeed.tech/topics/configuration.md>), [agent observability](<https://devfeed.tech/topics/agent-observability.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [applications](<https://devfeed.tech/tags/applications.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [coding](<https://devfeed.tech/tags/coding.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [java](<https://devfeed.tech/tags/java.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [migration](<https://devfeed.tech/tags/migration.md>), [modernization](<https://devfeed.tech/tags/modernization.md>), [open](<https://devfeed.tech/tags/open.md>), [software](<https://devfeed.tech/tags/software.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [spring](<https://devfeed.tech/tags/spring.md>), [validation](<https://devfeed.tech/tags/validation.md>)

## AI overview

ScarfBench is an open benchmark for evaluating AI agents on enterprise Java framework migrations across Spring, Jakarta EE, and Quarkus. It measures whether migrated applications build, deploy, and preserve behavior, and reports that current agents achieve less than 10% behavioral success on the benchmark.

## Source excerpt

Recent advances in coding agents have sparked excitement around AI-assisted modernization. But an important question remains: Can AI agents reliably modernize real-world enterprise applications? Existing software engineering benchmarks have demonstrated impressive progress in bug fixing and code generation, but framework migration presents a fundamentally different challenge.