# Scaling llms.txt with a Hierarchy of Files

DevFeed: [Scaling llms.txt with a Hierarchy of Files](<https://devfeed.tech/articles/scaling-llms-txt-31084.md>)

Original publisher: [Read original article](<https://www.mintlify.com/blog/scaling-llms-txt>)

Author: Kyan Yang

Published: 2026-09-01T00:00:00Z

Content type: article

Language: en

Sources: [Mintlify Blog](<https://devfeed.tech/sources/mintlify-blog.md>)

Topics: [Documentation](<https://devfeed.tech/topics/documentation.md>), [navigation](<https://devfeed.tech/topics/navigation.md>), [Website](<https://devfeed.tech/topics/website.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [documentation](<https://devfeed.tech/tags/documentation.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [files](<https://devfeed.tech/tags/files.md>), [generation](<https://devfeed.tech/tags/generation.md>), [navigation](<https://devfeed.tech/tags/navigation.md>)

## AI overview

Mintlify rebuilt llms.txt generation as a hierarchy of files to keep large documentation sites within the 100,000-character limit while preserving access to pages. In tests across 200 tasks on ten documentation sites, the improved version completed tasks 52.1% faster and used 44.8% fewer tokens.

## Source excerpt

Large documentation sites were hitting Mintlify's 100,000-character llms.txt limit and omitting pages. We rebuilt generation as a hierarchy of files so agents can reach every page without loading the entire index.