Home / Web & SEO / robots.txt Generator & Tester
robots.txt Generator & Tester
Build a robots.txt with rule groups and sitemaps, block AI crawlers in one tick, and test which URLs are allowed.
Free, no uploads, no sign-up. Runs 100% in your browser.
How robots.txt Generator & Tester works
Generated in your browser from what you type or drop. Nothing is fetched from or sent to your site. The tester follows the standard: the longest matching rule wins and Allow wins a tie.
Input: Text. Output: TXT.
How to use robots.txt Generator & Tester
- Set user agents and paths.
- Add your sitemap.
- Download robots.txt.
- Test URLs in the Test tab.
Tips
- Nothing is fetched from your site, so these tools also work for staging sites and pages behind a login.
- Test changes on a staging copy of your site first.
robots.txt Generator & Tester FAQ
- Are my files uploaded?
- No. This tool runs entirely in your browser tab. Files are read from your disk into memory and the result is generated locally.
- Is it free? Are there limits or watermarks?
- Free and open source (MIT). There are no file-size caps, daily quotas, accounts or watermarks.
- Does it work offline?
- Yes, once the page (and any engine it needs) has loaded, it keeps working without a connection thanks to the service worker cache.
Related tools
- XML Sitemap Generator: Make sitemap.xml from a list of URLs or your HTML files, with dates and priorities, split at 50,000 URLs.
- llms.txt Generator: Write an llms.txt file that gives AI assistants a clean map of your site, docs and key pages.
- Schema Markup Generator: Make JSON-LD structured data for FAQ, Article, Product, Local Business, Event, Recipe, How-to and more.
- Sitemap Checker: Check a sitemap for broken dates, wrong priorities, duplicates and mixed hosts, and find pages missing from it.