# Patching the Protocol: A Taxonomy of Pre-Registration Repairs in LLM-Proposer Experiments

DevFeed: [Patching the Protocol: A Taxonomy of Pre-Registration Repairs in LLM-Proposer Experiments](<https://devfeed.tech/articles/patching-the-protocol-a-taxonomy-of-pre-registration-repairs-in-llm-proposer-experiments-116187.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.04338>)

Author: Peter Chen, Ni Zhang, Siyuan Zhang

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [trust](<https://devfeed.tech/topics/trust.md>)

Tags: [case-study](<https://devfeed.tech/tags/case-study.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [data](<https://devfeed.tech/tags/data.md>), [defects](<https://devfeed.tech/tags/defects.md>), [design](<https://devfeed.tech/tags/design.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [impact](<https://devfeed.tech/tags/impact.md>), [incentive-design](<https://devfeed.tech/tags/incentive-design.md>), [llm](<https://devfeed.tech/tags/llm.md>), [patching](<https://devfeed.tech/tags/patching.md>), [production-scheduling](<https://devfeed.tech/tags/production-scheduling.md>), [program](<https://devfeed.tech/tags/program.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [repair](<https://devfeed.tech/tags/repair.md>), [research](<https://devfeed.tech/tags/research.md>), [rules](<https://devfeed.tech/tags/rules.md>), [software](<https://devfeed.tech/tags/software.md>), [study](<https://devfeed.tech/tags/study.md>), [taxonomy](<https://devfeed.tech/tags/taxonomy.md>), [timing](<https://devfeed.tech/tags/timing.md>), [trust](<https://devfeed.tech/tags/trust.md>)

## AI overview

In three LLM-proposer experiments spanning production scheduling, incentive design, and task assignment, each protocol needed one defect-repair patch. None changed a numbered judgment criterion, its threshold, or the scoring rule, or altered collected data; two changed pre-registered design parameters before data collection and recorded a rationale. The paper proposes classifying patches by defect type (environmental, feasibility, or correctness) and timing (pre-data or post-data), plus a verifiable non-impact claim for post-data repairs and a nine-item checklist. Every checklist item appeared in at least one patch event, which the authors describe as use, not proof that the checklist is sufficient. The abstract emphasizes auditability as central to credibility.

## Source excerpt

arXiv:2611.04338v1 Announce Type: cross Abstract: Pre-registration is increasingly proposed as a trust mechanism for machine-learning research involving LLM-generated content, but its canonical form assumes a frozen protocol can be executed unchanged. Software-intensive studies violate this assumption: the protocol is partly implemented, and implementations carry defects. We report a three-case study of protocol repairs ("patches") from a program where an LLM proposes executable rules that a programmatic verifier gates, across production scheduling, incentive design, and task assignment. Each protocol required exactly one defect-repair patch: three events in total. None modified a numbered judgment criterion, its threshold, or the scoring rule, and none altered already collected data; two changed pre-registered design parameters before any data were collected, each with a recorded rationale. We propose a two-axis taxonomy of patches (defect class: environmental / feasibility / correctness, x timing: pre-data / post-data), introduce a verifiable non-impact claim for post-data repairs, and distill a nine-item checklist; all nine items were exercised by at least one patch event -- an observation of use, not a validation of sufficiency. The credibility of verifier-bound, LLM-proposer studies depends less on whether patches occur than on whether they are auditable.