# A bad llms.txt is worse than none at all > A missing llms.txt costs nothing — a sloppy one fails in PageSpeed Insights. What the Lighthouse audit checks, and the five things to get right. Not having an llms.txt costs you nothing. Having a sloppy one costs you a red mark in Google's own auditing tool. So if you ship the file, ship it properly — or don't ship it at all. ## What changed in 2026 Lighthouse, the tool behind PageSpeed Insights, now has a [category for agentic browsing](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt). It holds three audits, and one of them inspects your [llms.txt](https://btlabs.dev/en/glossary/llms-txt). It runs in the default configuration — so for anyone who drops a site into PageSpeed Insights. Your client, your competitor, you. In the same year, Google says on the search side: we do not support llms.txt. Gary Illyes put it that way at Search Central Live in July 2025, and there are no plans to change it. John Mueller was asked whether Google's own llms.txt was a recommendation. His answer: "to be direct, no." Both are Google. Search denies the route, the browser scores it. ## What the audit actually checks This is the part most articles get wrong, so here it is precisely: - **File does not exist (404)** → no penalty, the audit is marked not applicable - **Server errors out or does not respond** → score 0 - **File present, no heading** → fail - **File present, shorter than 50 characters** → fail - **File present, no Markdown link** → fail The heading must be a real Markdown heading with a hash, at the very top. The length limit is what the tool itself calls "suspiciously short". The report then reads: "llms.txt does not follow recommendations." Read that first line again. **Absence is not penalised.** If you have no llms.txt, this audit does not touch you. If you have one, you get measured. That is the whole point of this article, and it is the place where we ourselves were wrong once: an earlier note of ours claimed the opposite. ## The auditor is young — and it makes mistakes Being measured would be fine if the yardstick held still. It does not. Three examples from Lighthouse's public issue tracker: In June 2026, [llms.txt files generated by the WordPress plugin Yoast failed](https://github.com/GoogleChrome/lighthouse/issues/17051). The cause was an invisible control character at the start of the file. Nothing about the content was wrong. Millions of WordPress sites were affected. Two weeks later, [files that matched the specification failed](https://github.com/GoogleChrome/lighthouse/issues/17082) — depending on which Chrome version ran the check. And since late August 2026 there is an [open bug where PageSpeed Insights reports a missing heading and missing links](https://github.com/GoogleChrome/lighthouse/issues/17194) although both are in the file. Nothing dramatic follows from this, but something practical does: if you ship an llms.txt, you are measured against a yardstick that is still moving. If you don't, you are not. ## Who actually fetches the file We measure who requests the file on our own site — across roughly three months, only the two llms paths. The picture in broad strokes: around seven hundred fetches, fewer than one in ten from a crawler whose origin can be technically confirmed. Googlebot and GPTBot demonstrably fetch it — regularly, but in modest numbers. The largest block is requests with no identifiable sender. Right behind that: llms.txt checking tools, half a dozen different ones. From one major AI provider, not a single fetch in that window. So there were more checking tools out there than AI providers. Four limits, so the number does not carry more than it can. This is **one** site over **one** window — a local finding, not a statement about the web. Fetching is not reading; nobody can see from the outside what happens to the file afterwards. Our site writes about llms.txt itself and therefore attracts checking tools disproportionately. And "unverified" does not mean "forged" — for some providers, origin cannot be verified at all. What the numbers do support is still clear: the file gets checked. Whether it gets read, we do not know. ## What this means for you **Having no llms.txt is a perfectly legitimate state.** No tool penalises it, no search engine demands it. If you don't have one and don't need one: done, move on. **If you do have one**, check five things: a heading with a hash at the very top, at least one real Markdown link, more than 50 characters of content, no invisible control character at the start, and reachable without a redirect chain. Then run it through PageSpeed Insights once. **If a plugin generates it for you**, check the result anyway. The Yoast case shows that one tool can produce a file another tool rejects — without you doing anything wrong. And treat the file as what it demonstrably is today: a signal to checking tools. Not a pipe to a language model. What comes after the file we have described elsewhere: [llms.txt is only step one](https://btlabs.dev/en/posts/llms-txt-is-only-step-one). ## The bigger line A format rarely wins because the document behind it is good. It wins because a tool emits it or scores it. Until 2026, llms.txt had no such carrier. Now it has one — and that carrier is an auditor, not a reader. The same tool, incidentally, also looks for a second route, /.well-known/ai-catalog.json. In our [ongoing measurement series](https://doi.org/10.5281/zenodo.22178282) it sits at zero per cent adoption. Whether an auditor alone is enough to move a format off zero, we will see in October. --- Source: https://btlabs.dev/en/posts/check-llms-txt-bad-file Last-Modified: 2026-09-21T13:58:02.115Z Languages: [de](https://btlabs.dev/llms/de/posts/llms-txt-pruefen-schlechte-datei) · [it](https://btlabs.dev/llms/it/posts/verificare-llms-txt-file-fatto-male) See also: [llms.txt](https://btlabs.dev/llms.txt) · [ai.txt (Policy)](https://btlabs.dev/ai.txt) · [identity.json](https://btlabs.dev/identity.json)