# Toward an Evidence-Based Template for Agent Instruction Files: An Empirical Study of Adoption, Consistency, and Maintenance

DevFeed: [Toward an Evidence-Based Template for Agent Instruction Files: An Empirical Study of Adoption, Consistency, and Maintenance](<https://devfeed.tech/articles/toward-an-evidence-based-template-for-agent-instruction-files-an-empirical-study-of-adoption-consistency-and-maintenance-113525.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02383>)

Author: Mehedi Hasan Shanto, Md. Anaytul Islam, Muhammad Asaduzzaman, Alioune Ngom

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Claude Code](<https://devfeed.tech/topics/claude-code.md>), [GitHub Copilot](<https://devfeed.tech/topics/github-copilot.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>)

Tags: [adoption](<https://devfeed.tech/tags/adoption.md>), [agent](<https://devfeed.tech/tags/agent.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [evidence-based](<https://devfeed.tech/tags/evidence-based.md>), [files](<https://devfeed.tech/tags/files.md>), [maintenance](<https://devfeed.tech/tags/maintenance.md>), [study](<https://devfeed.tech/tags/study.md>), [template](<https://devfeed.tech/tags/template.md>)

## AI overview

Across 1,000 GitHub repositories, 42.2% of instruction-bearing repositories in the higher-popularity cohort maintained multiple agent instruction formats; in 386 manually examined pairs of corresponding sections, 16.1% gave divergent guidance. The study also found that 40.4% of analyzed files had only one recorded modification event, with maintenance patterns varying by format. From category prevalence and addition ratios, the authors derived a template with ten Required, four Gap, and two Recommended categories. In an exploratory experiment, explicit template guidance raised structural category coverage from 18.7% to 82.8%. The AGIT-Instruct benchmark evaluates generation by structural coverage and content alignment; among four configurations, retrieval-augmented generation had the highest full-file ROUGE-1 F1 (0.459). Manual review found that generated categories fully preserved developer intent in 19.4% of instances and partially preserved it in 61.2%.

## Source excerpt

arXiv:2611.02383v1 Announce Type: new Abstract: AI coding agents rely on repository-level instruction files to understand project conventions, development workflows, and constraints. However, different agents support different instruction formats, and limited empirical evidence exists about how these files are adopted, what information they contain, whether coexisting formats provide consistent guidance, and how they are maintained. We conduct an empirical study of 3,387 instruction files from 1,000 GitHub repositories across four formats: AGENTS.md, CLAUDE.md, GitHub Copilot instructions, and Cursor rules. We find that 42.2% of instruction-bearing repositories in the higher-popularity cohort maintain multiple formats. Among 386 manually examined pairs of corresponding sections, 16.1% provide divergent guidance. Our maintenance analysis further shows that 40.4% of the analyzed files have only one recorded modification event and that maintenance patterns differ across formats. Based on category prevalence and category addition ratios (CAR), we derive an evidence-based template containing ten Required, four Gap, and two Recommended categories. In an exploratory generation experiment, explicit template guidance increases structural category coverage from 18.7% to 82.8%. We further introduce AGIT-Instruct, a benchmark for evaluating instruction-file generation through structural coverage and content alignment. Across four generation configurations, retrieval-augmented generation achieves the highest full-file ROUGE-1 F1 of 0.459. A manual assessment of 278 generated category instances finds that 19.4% fully preserve and 61.2% partially preserve developer-written intent. Our findings provide empirical guidance for organizing and maintaining agent instruction