A robots.txt file tells crawlers which paths they may fetch. Choose a preset, add your rules and copy the result.
SEO tools
Robots.txt Generator
Build a clean robots.txt file with crawl rules, a sitemap line and common presets for WordPress sites.
Loading tool…
Privacy: this tool runs entirely in your browser. What you type is not uploaded or stored.
Frequently asked questions
Does robots.txt hide pages from Google?
No. It controls crawling, not indexing. A blocked URL can still appear in results if other sites link to it. Use a noindex directive to keep a page out of search.
Where do I upload the file?
At the root of your domain, for example example.com/robots.txt. WordPress can serve a virtual one; an SEO plugin can edit it.