# What Does LOL Measure? Hidden Label Shifts in Low-Light Enhancement Benchmarks

DevFeed: [What Does LOL Measure? Hidden Label Shifts in Low-Light Enhancement Benchmarks](<https://devfeed.tech/articles/what-does-lol-measure-hidden-label-shifts-in-low-light-enhancement-benchmarks-108649.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.38121>)

Author: Adityan Thirumani Srinivasan

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [image editing](<https://devfeed.tech/topics/image-editing.md>), [color](<https://devfeed.tech/topics/color.md>), [Image](<https://devfeed.tech/topics/image.md>), [reproducibility](<https://devfeed.tech/topics/reproducibility.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-iv](<https://devfeed.tech/tags/eess-iv.md>), [enhancement](<https://devfeed.tech/tags/enhancement.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [light](<https://devfeed.tech/tags/light.md>), [reporting](<https://devfeed.tech/tags/reporting.md>), [shift](<https://devfeed.tech/tags/shift.md>), [test](<https://devfeed.tech/tags/test.md>), [train](<https://devfeed.tech/tags/train.md>)

## AI overview

An audit of 18 checkpoints from eight low-light enhancement methods finds train-test label shifts in LOL-v1 and LOLv2-Real that affect benchmark scores and method rankings. LOL-v1's 15 test labels have the same luminance as corresponding LOLv2 labels but 1.48x higher saturation; LOLv1 training labels match the LOLv2 rendering. LOLv2-Real test labels are 14% darker than training labels, and all 15 LOL-v1 test inputs recur in LOLv2-Real, including 12 in its training set. Test-only tuning yields a median residual of 0.37 dB on LOL-v1 and 0.92 dB on LOLv2-Real, versus 0.00 dB on unshifted LOLv2-Syn; test-set checkpoint selection can inflate a from-scratch model's score by 0.9-2.0 dB. Removing released hand-set test-time settings changes which method ranks first. The authors provide reporting recommendations.

## Source excerpt

arXiv:2610.38121v1 Announce Type: cross Abstract: LOL-v1 and LOLv2-Real are the standard benchmarks for supervised low-light image enhancement, and several leading methods ship hand-set test-time adjustments with their evaluation scripts. We audit both benchmarks with 18 public checkpoints of eight methods under one metric implementation that reproduces every published number to within 0.09 dB, and find that their test labels differ from their training labels. (i) The 15 LOL-v1 test labels are rendered differently: the same captures appear in LOLv2 with labels of identical luminance but 1.48x lower saturation, and LOL-v1's own training labels match the LOLv2 rendering; we confirm this on the official release. (ii) The LOLv2-Real test set is the contiguous block of images 690-789, whose labels are 14% darker than the training labels (Cliff's delta = -0.45, p < 1e-12) with no comparable shift in the input statistics we examined; methods trained without any labels score up to 5.5 dB higher on test than on train. (iii) All 15 LOL-v1 test inputs reappear in LOLv2-Real, 12 in its training set. The released test-time settings are consistent with compensating for these shifts. Separating what can be calibrated on training data from what only test-set tuning finds, the test-only residual has median 0.00 dB on LOLv2-Syn, which has no shift, but 0.37 dB on LOL-v1 and 0.92 dB on LOLv2-Real (up to 3.9 dB). Under the shift, test-set checkpoint selection inflates a from-scratch model by 0.9-2.0 dB, and removing the released settings changes which method ranks first. We give concrete reporting recommendations.