🤖
Robots.txt Generator
Generate a valid robots.txt with allow/disallow rules, crawl-delay and a sitemap reference — no syntax mistakes.
robots.txt
🔒 This tool runs entirely in your browser. Your data is never uploaded.
Frequently asked questions
What does robots.txt do?＋
It tells search-engine crawlers which paths they may or may not crawl, and can point to your sitemap.
Does it stop a page being indexed?＋
Not reliably — robots.txt controls crawling, not indexing. To keep a page out of results, use a noindex meta tag instead.
Where does the file go?＋
At the root of your site, served as /robots.txt.