# From Evaluation to Evolution: Benchmarking Image Editing and Evolving Image Models

DevFeed: [From Evaluation to Evolution: Benchmarking Image Editing and Evolving Image Models](<https://devfeed.tech/articles/from-evaluation-to-evolution-benchmarking-image-editing-and-evolving-image-models-112042.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.41210>)

Author: Jun He, Canmiao Fu, Ying Zhang, Zhipeng Huang, Chenyu Mu, Yazhen Xie, Chen Li, Jing Lyu, Weijia Li, Xiang Zhang

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [image editing](<https://devfeed.tech/topics/image-editing.md>), [Image](<https://devfeed.tech/topics/image.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>)

Tags: [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [image](<https://devfeed.tech/tags/image.md>), [image-editing](<https://devfeed.tech/tags/image-editing.md>), [models](<https://devfeed.tech/tags/models.md>)

## AI overview

WeGenEditBench evaluates image editing at the level of individual requirements, using shared checklists and visual evidence before aggregating scores for execution, preservation, quality, and task-specific correctness. It contains 2,740 instances across 148 task types, including single-image, text-intensive, and multi-reference editing. The accompanying WeGenEvoStudio workflow diagnoses failures, builds retrieval-grounded training data, adapts models with supervised learning, and re-evaluates them. Among 15 configurations, GPT-Image-2 leads single-image editing (4.076 Overall) and Seedream-5.0-Pro leads text-intensive editing (4.291). In targeted evolution experiments, FLUX.2 [dev] rises from 1.083 to 3.517 Overall on the report-selected Top-5 single-image targets; Qwen-Image-Edit-2511 gains 13.2%. These are reported targeted results, not evidence of uniform gains across tasks.

## Source excerpt

arXiv:2610.41210v1 Announce Type: cross Abstract: Real-world image editing requires coordinating multiple visual transformations with content preservation and reference constraints, yet aggregate scores can readily obscure unmet local requirements. Effective evaluation must therefore identify local failures and translate them into priorities for subsequent model improvement. We introduce WeGenEditBench, comprising 2,740 evaluation instances and 148 task types across single-image, text-intensive, and multi-reference editing. Pre-generated checklists shared across models decompose each request into atomic requirements; item-level deductions and visual evidence are established first and then summarized into scores for edit execution, preservation, visual quality, and task-specific correctness. To support the experience-acquisition component of recursive self-improvement (RSI), we develop WeGenEvoStudio, a unified multi-agent harness for RSI-oriented learning that integrates model-conditioned failure diagnosis, retrieval-grounded data construction, supervised adaptation, and re-evaluation. Among 15 evaluated configurations, GPT-Image-2 leads single-image editing with an Overall score of 4.076, while Seedream-5.0-Pro leads text-intensive editing at 4.291. Evolution experiments across four generation and editing benchmarks demonstrate substantial targeted gains: on their report-selected Top-5 single-image targets, FLUX.2 [dev] improves Overall from 1.083 to 3.517, while Qwen-Image-Edit-2511 gains 13.2%. Together, our benchmark and harness provide a foundation for developing more reliable image models through interpretable evaluation and feedback-driven learning.