llms.txt File Mistakes to Avoid: Fixes That Keep AI Crawlers Reading Your Site
An llms.txt file is a small text file that tells AI systems what your site is about and which pages matter most. Done well, it gives an AI reader a clean map of your best content, in plain text, without forcing it to guess from navigation menus and design elements. Done badly, it creates confusion, sends AI readers to the wrong pages, or says nothing useful at all. Most mistakes in this file are simple, and most fixes take only a few minutes once you know what to look for.
This article covers the mistakes that show up most often, and the fix for each one. It is a supporting guide, not a replacement for the setup steps. If you have not created the file yet, start with the guide to adding an llms.txt file to your website, then come back here to check your file against the list below.
Mistake 1: Putting the file in the wrong place
The file belongs at the root of your site, where an AI reader expects to find it. A file buried in a blog folder, a media folder, or a page builder folder may never be found, even if the content inside is excellent. Check the live address directly in a browser. If the file does not load at the root address, move it and test again. Do not rely on a redirect to fix a wrong location, because a redirect adds a failure point and can hide the problem during a later site move.
While you are there, check that the file loads as plain text and that it does not show an error page, a login screen, or a redirect to your homepage. Any of those results means the file is not doing its job, no matter how good the draft on your computer looks. Test the live file, not only the local copy.
Mistake 2: Copying your homepage into the file
Your homepage is written for human visitors. It uses short slogans, design cues, and calls to action that make sense in context. An llms.txt file needs a different kind of writing. It should state what your business does, who it serves, and which pages explain each topic in full. Copying homepage slogans into the file wastes the one clear summary an AI reader will use to judge your site.
Write the summary as if you are explaining your site to a careful colleague who has never seen it. Name your main services in plain words. Name the audience in plain words. Then list the pages that prove each point, with a short description of what each page covers. If a page does not help an AI reader understand your site, leave it out. A focused file beats a long file that lists everything.
Mistake 3: Listing pages that no longer exist
Dead links in an llms.txt file do real damage. They waste the attention of an AI reader, they signal that the site is not maintained, and they can lead to a summary based on missing content. Check every link in the file at least once a month, and every time you delete, rename, or merge a page. A link checker helps, but a manual click on the most important links is still worth doing, because a checker may miss a page that loads with the wrong content.
When you remove a page, remove it from the file in the same update. Do not leave it in place while you decide what to do with it. The same rule applies to pages you no longer want AI systems to highlight, such as an outdated offer or a service you no longer provide. A clean file reflects the site you have today, not the site you had last year.
Mistake 4: Fighting your own robots rules
An llms.txt file does not override other access rules on your site. If a page is blocked in your robots file, or hidden behind a login, listing it in llms.txt will not make it available to an AI reader. Worse, it creates a mixed signal that can confuse the systems you are trying to help. Before you list a page, confirm that a public visitor can open it without signing in, and that it is not blocked by a rule you set for another reason.
Check the file itself for the same problem. If your site blocks text files, blocks unknown crawlers by default, or serves the file only to certain visitors, fix that access rule first. The file is only useful when the readers it is meant for can actually read it.
Mistake 5: Writing vague descriptions for every page
A description that could fit any page helps no one. A line that says a page covers useful tips, great services, or helpful information tells an AI reader nothing about what is inside. Write each description to answer a simple test. After reading the description, could a stranger say what the page teaches, who it is for, and what decision it helps with. If not, rewrite it.
Use the same plain language on the page itself. If the description promises a comparison, the page should contain a comparison. If the description promises steps, the page should contain steps. A mismatch between the file and the page breaks trust, and trust is the whole point of giving AI systems a clean map of your content.
Mistake 6: Never updating the file
A site changes, and an llms.txt file that never changes slowly becomes a record of an old site. New services appear, old services close, and your best pages change as you publish stronger work. Set a simple update habit. Each time you publish a major page, ask whether it belongs in the file, and whether a weaker page should come out. Each month, read the file from top to bottom and remove anything that no longer reflects your work.
Keep the file short enough to maintain. A file with a huge list is hard to review, hard to keep accurate, and easy to abandon. A file with your core pages, described well, is easy to review and easy to trust. For a broader view of how this fits into AI visibility work, read the LLM SEO strategy for small business, which puts the file in context with content, citations, and brand consistency.
Mistake 7: Overclaiming what you do
An llms.txt file is not an advertisement, and AI readers treat overclaiming as a warning sign. Avoid claims that you are the best choice for everyone, that a method works in every case, or that a result is certain. State what you do, who it suits, and what limits apply. A business that describes its real scope clearly is easier to recommend for the right question, and easier to skip for the wrong one. That is a good outcome, because a wrong recommendation helps no one.
Check names, services, and contact details with the same care. A wrong service name, an old phone number, or a closed location in the file can end up in an AI answer. Small errors in a trusted file travel further than small errors on a single page, so proofread the file as if it will be quoted.
A simple test routine
Use this routine after every change. Open the live file and read it as a stranger. Click the first three links and confirm each one loads the page described. Check that the summary matches your current services. Check that no blocked or deleted page remains. Then save the date of the check in your notes, so the next check is not left to memory.
If you find a mistake, fix it in the live file and in any draft you keep elsewhere. Keeping two versions, one live and one on your computer, is a common cause of repeat mistakes. Make the live file the only version that matters, and update it directly.
Questions and Answers
Does an llms.txt file guarantee that AI systems will cite my site?
No. The file helps AI systems understand and find your best pages, but citation still depends on the quality, clarity, and trustworthiness of the pages themselves. Treat the file as a map, not a promise.
How many pages should an llms.txt file list?
List only the pages that help an AI reader understand your site and answer real questions about it. For most small businesses, a focused list of core pages works better than a long list of every page on the site.
What should I do first if my llms.txt file is not working?
Check the live file at the root address first. Confirm it loads as plain text, that the links work, and that the pages listed are public and current. Fix location, access, and dead links before you rewrite any descriptions.