# Predicting Human-Gain Curves under Local Proxy Optimization from Repeated Ratings

DevFeed: [Predicting Human-Gain Curves under Local Proxy Optimization from Repeated Ratings](<https://devfeed.tech/articles/predicting-human-gain-curves-under-local-proxy-optimization-from-repeated-ratings-112027.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.41114>)

Author: Moonwon Choi, Kisung Nam, Seunggeun Lee

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [human feedback](<https://devfeed.tech/topics/human-feedback.md>), [dpo](<https://devfeed.tech/topics/dpo.md>), [lora](<https://devfeed.tech/topics/lora.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [branch](<https://devfeed.tech/tags/branch.md>), [bridge](<https://devfeed.tech/tags/bridge.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [changes](<https://devfeed.tech/tags/changes.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [correlation](<https://devfeed.tech/tags/correlation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [d](<https://devfeed.tech/tags/d.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [dpo](<https://devfeed.tech/tags/dpo.md>), [error](<https://devfeed.tech/tags/error.md>), [local](<https://devfeed.tech/tags/local.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [proxy](<https://devfeed.tech/tags/proxy.md>), [under](<https://devfeed.tech/tags/under.md>)

## AI overview

PACE uses repeated human ratings to estimate how human-perceived quality may change as a fixed proxy is optimized gently. It models the local gain at KL radius d as A√d + Bd, with A capturing first-order proxy-human alignment and B its leading second-order correction. The authors report lower held-out gain-prediction error than A-only, zero-gain, and reliability-only baselines across several evaluation datasets and a GPT-4o judge proxy experiment. PACE is competitive with flexible proxy-conditional calibration given the same pre-pool information; a LoRA-DPO experiment offers a modest link to actual optimization. Source coefficients help transfer across datasets, while target calibration improves predictions. The diagnostic is limited to local, one-sided optimization along the positive tilt branch of a fixed item pool.

## Source excerpt

arXiv:2610.41114v1 Announce Type: new Abstract: Optimizing a proxy objective can improve human-perceived quality, but the resulting gain can vary with optimization strength. Average proxy-human correlation alone does not reveal how human-perceived quality changes as optimization strength increases. We propose PACE (Proxy Alignment and Curvature Expansion), a pre-optimization diagnostic that uses repeated human ratings to predict the human-gain curve induced by a fixed proxy. The method models proxy optimization as a KL-regularized exponential tilt. At KL radius $d$, the local gain is approximated by $A\sqrt{d}+Bd$, where $A$ measures first-order proxy-human alignment and $B$ captures the leading second-order correction along the proxy-optimization path. We estimate $A$ and $B$ from repeated human ratings on calibration items and test the resulting gain-curve prediction on held-out evaluation items. Across SummEval, analysis-eligible WMT MQM clusters, an OpenMEVA-MAGS ROC/WP repeated-rating slice, and a GPT-4o judge proxy experiment, PACE consistently reduces held-out gain-prediction error relative to $A$-only, zero-gain, and reliability-only baselines. With the same pre-pool information, it is competitive with flexible proxy-conditional calibration, and a LoRA-DPO experiment provides a modest bridge to actual optimization. Transfer experiments across datasets and evaluation slices show that source coefficients are useful priors, but target calibration further improves prediction. The resulting diagnostic is local and one-sided: it characterizes gentle optimization along the positive tilt branch of a fixed item pool.