# What September shows: three findings from the radar Topic: [Research](https://btlabs.dev/en/ai-discovery-radar) > llms.txt just above 13%, usage reservations and files for AI agents below one percent: three findings from the radar, with denominator and working. In September 2026, roughly one observable domain in seven or eight served an llms.txt. Files declaring usage reservations towards AI stayed below one percent, and files for AI agents came in at about one per thousand at most. Those are the three observations the published data of the [ai-discovery radar](https://btlabs.dev/en/ai-discovery-radar) can actually carry. Here they are with denominator, measurement date and working — and with what they do not say. How the radar counts the adoption of machine-readable files is explained in the [method report](https://btlabs.dev/en/posts/method-report-ai-discovery-radar); we won't repeat it here. Just this much: what counts is whether a file can be retrieved and is formally valid. Not whether any system reads it. ## The basis Both data files come from the same run, 1 to 3 September 2026, computed under ruleset 0.3.2: - **Panel:** 1,000 fixed domains, 853 of them reachable. The series that is measured the same way every month. - **Monthly run:** 10,667 domains (block a with 9,667 plus the panel), 9,206 of them reachable. A larger base, narrower intervals. Every route — that is, every path such as /llms.txt — has its own denominator n: only the hosts where we could actually look. A host that turns the request away, forbids it in robots.txt or does not answer is left out. In the panel, n therefore ranges from 641 to 761; in the monthly run from 6,725 to 8,221. Every percentage below comes with its 95% interval, and every one can be recomputed from the numerator and denominator in the files. ## Observation 1: llms.txt sits just above 13% — in two separate samples In September, 13.45% of the observable panel domains served an llms.txt: 90 of 669 (interval 11.1–16.2%). An llms.txt is a text file at the root of a website that tells language models, in short form, which content matters. It is an industry proposal, not an adopted standard ([LLMs.txt in the glossary](https://btlabs.dev/en/glossary/llms-txt)). The figure holds up because it repeats. The panel is part of the monthly run. Take it out and you are left with the 9,667 domains of block a — where 845 of 6,404 observable hosts served an llms.txt: 13.19% (12.4–14.0%). Two separate draws from the same frame, the same picture: roughly one domain in seven or eight. Two details from the full monthly run: - The long version /llms-full.txt was served by 303 of 7,061, or 4.29%. - The location is unambiguous: 935 hits at the root (/llms.txt), 2 in the /.well-known/llms.txt folder (of 7,046). What the figure does not mean: that 13% of "websites" are ready for AI. The sample is drawn from rankings of popular domains (Tranco and the Chrome UX Report), not from the whole web. And a file that exists says nothing about whether it is read, kept current, or achieves anything. ## Observation 2: usage reservations in a dedicated file stay below one percent Three files declare, in machine-readable form, what content may be used for. In the September monthly run, all three stayed below one percent of observable domains: - /rsl.xml, licence terms for the use of content: 53 of 6,795 = 0.78% (0.6–1.0%) - /.well-known/tdmrep.json, a reservation against text and data mining: 28 of 7,282 = 0.38% (0.3–0.6%); valid declarations via HTTP header are included - /ai.txt, content withheld from AI training: 19 of 7,053 = 0.27% (0.2–0.4%); in the /.well-known/ai.txt folder it was 4 of 7,090 For comparison, each with its own denominator: 6,966 of 8,221 served a robots.txt, 84.73%. Each of the three reservation files is more than a hundred times rarer. A ranking among the three small values, however, is not something the data supports, because every row has its own denominator. In the panel alone there are 2 to 5 hits per file. That is enough for "rare", not for a share worth quoting. Only the monthly run carries this observation. Worth keeping in mind: robots.txt, ai.txt and TDMRep are signals, not technical locks. What a [TDM reservation](https://btlabs.dev/en/glossary/tdm-opt-out-tdmrep) does and does not do is in the glossary. ## Observation 3: files for AI agents sit between zero and about one per thousand All nine routes aimed explicitly at AI agents and automated clients — agent cards, MCP descriptions, ai-catalog.json, plugin manifests and x402 — were at 0.11% or below in the monthly run. A selection: - /.well-known/ai-plugin.json: 8 of 7,038 = 0.11% - /.well-known/agent-card.json: 5 of 7,194 = 0.07% - /.well-known/mcp.json: 5 of 7,181 = 0.07% - /.well-known/ai-catalog.json: 2 of 7,190 = 0.03% - /.well-known/x402.json: 0 of 7,167 General interface files such as the API catalogue (0.29%) or the OAuth descriptions (about 5%) are not counted here, because they do not serve AI agents specifically. This is where it becomes concrete why a zero in the panel is not an absence. In the September panel, ai-catalog.json stood at "0.0%", measured on 675 hosts. The 95% interval reaches up to 0.6%: with 675 hosts you cannot say more than "at most about six in a thousand". The run ten times the size narrows that to 0.0–0.1%. Rare, but not nothing. It works the other way too. /.well-known/agent.json stands at 0.15% in the panel, and that is exactly one hit. A single host moves a panel share by about 0.15 percentage points — for rare files, the panel is too small to give shares. ## What the data cannot tell you Four questions come to mind, and the published files answer none of them: - **Is it growing?** The panel was measured in August too, but under the older ruleset 0.3.1. For all 33 routes measured in both months, the intervals overlap. The first regular month-on-month comparison comes with the October run. - **Are operators blocking AI crawlers in robots.txt?** The radar counts whether a robots.txt exists, not what it says. How many of them address the [crawlers](https://btlabs.dev/en/glossary/crawler) of AI providers is not in the published data. - **What about South Tyrol?** The sample is drawn partly from top lists for Italy, Germany, Austria and Switzerland, but figures per country are not published. Nothing follows from it for businesses in South Tyrol. - **Does it help?** Availability is not use. Whether AI systems fetch, read or follow these files is not something the radar measures. ## What it means for your business The numbers work against two lines you hear a lot. "Everyone has that by now" is not even true of robots.txt (84.73%), and after it and the sitemap (53.92%) the list drops below 14%. "Nobody uses that" rarely holds either: in the large run, only 4 of the 34 routes had no hit at all. What the numbers do not do: tell you which file you need. That depends on what you want to tell machines. A table of contents for language models is one thing, a reservation against AI training another, an offer to agents a third. Adoption is no yardstick for that — in either direction. Disclosure: btlabs Core ships several of the measured files out of the box, among them llms.txt, ai-catalog.json and a TDM reservation. Why we built them in rather than bolting them on later is in [What is already built in](https://btlabs.dev/en/posts/what-is-already-built-in). That is exactly why this post only contains numbers anyone can recompute from the files. If you read one of these numbers differently from us, write to us via the form on the radar page. We are happy to run the numbers with you. ## Sources and status Measurement window 1–3 September 2026, ruleset 0.3.2, frame 2026-Q3. Data files panel-2026-09 and monat-2026-09-block-a in the [public repository](https://github.com/flober81/ai-discovery-radar/tree/main/data), dataset DOI [10.5281/zenodo.22178282](https://doi.org/10.5281/zenodo.22178282), data licensed CC BY 4.0. The block-a-without-panel values are our own: numerator minus numerator, denominator minus denominator, because under the ruleset the panel is a subset of the monthly run. Their intervals are Wilson intervals, as in the radar. --- Source: https://btlabs.dev/en/posts/radar-september-2026-findings Last-Modified: 2026-10-02T07:00:02.507Z Languages: [de](https://btlabs.dev/llms/de/posts/radar-september-2026-befunde) · [it](https://btlabs.dev/llms/it/posts/radar-settembre-2026-risultati) See also: [llms.txt](https://btlabs.dev/llms.txt) · [ai.txt (Policy)](https://btlabs.dev/ai.txt) · [identity.json](https://btlabs.dev/identity.json)