# MoI-Bench: A Benchmark for Image Generation and Editing on Mobile Devices

DevFeed: [MoI-Bench: A Benchmark for Image Generation and Editing on Mobile Devices](<https://devfeed.tech/articles/moi-bench-a-benchmark-for-image-generation-and-editing-on-mobile-devices-103295.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37599>)

Author: Kailai Feng, Haoxiang Rao, Yuxiang Wei, Bo Chen, Junsong Li, Yang Pan, Chengkun Cai, Jagmohan Chauhan

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Image](<https://devfeed.tech/topics/image.md>), [image editing](<https://devfeed.tech/topics/image-editing.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [devices](<https://devfeed.tech/tags/devices.md>), [generation](<https://devfeed.tech/tags/generation.md>), [image](<https://devfeed.tech/tags/image.md>), [mobile](<https://devfeed.tech/tags/mobile.md>)

## AI overview

MoI-Bench evaluates mobile image generation and editing with 64 subtasks and 1,150 samples, pairing 16 operations with eight visual target categories to expose specific capability gaps. Its MoI-Eval uses task-specific criteria for instruction following, visual quality, and detail preservation, then profiles quality and capability across operations, targets, and criteria. The authors also measure mobile deployment costs and present a Quality-Deployability map rather than relying on a single model score. Their reported results suggest current on-device models can generate or edit efficiently, while broader task coverage merits more attention; remaining editing priorities include identity preservation and text rendering. The evidence provided is an abstract, so it does not include detailed model-by-model results.

## Source excerpt

arXiv:2610.37599v1 Announce Type: cross Abstract: Developing on-device image models requires considering practical user needs, task types and device resource constraints. However, existing benchmarks provide limited guidance on identifying capability gaps between on-device models and server-side models. We introduce MoI-Bench, a benchmark grounded in mobile user needs, comprising 64 subtasks and 1150 samples. It defines 16 operations, such as removal and replacement, and eight visual target categories covering content such as portraits and text. These form two orthogonal axes whose applicable combinations define specific tasks, such as removing facial makeup. To better guide model improvement, we further propose MoI-Eval, which introduces task-specific criteria covering instruction following, visual quality and detail preservation. We further construct sample-level evaluation questions and aggregate their scores into overall quality and capability profiles along the operation, target and criteria axes. We evaluate representative on-device and server-side models and measure mobile deployment costs. Results show that this decoupled Operation x Target design provides finer-grained guidance for improving compact models. For example, it highlights the need to prioritize specific capability gaps (e.g., text-replacement) over broad application-level goals (e.g., improving text editing). Unlike conventional benchmarks that rank models using a single score, we introduce a two-dimensional Quality-Deployability capability map to assess models for practical mobile use. The map shows that current on-device models can achieve efficient generation or editing but expanding their task coverage deserves greater attention than further improving deployment efficiency. Further progress requires improving editing quality, particularly identity preservation and text rendering, while broadening task coverage.