Robots.txt Generator

Create a properly formatted robots.txt file for your website. Configure crawl rules, block specific bots, and set sitemap references with ease.

5 / 5 free uses remaining today

Quick Presets

Choose a preset to quickly configure your robots.txt, or build custom rules below.

Add rules to control which bots can access which parts of your site.

User-Agent

Rule Type

Path

Sitemap URL

Crawl Delay (seconds)

robots.txt2 lines

User-agent: *
Allow: /

Generate a robots.txt that helps search engines properly crawl and index your website for better visibility and rankings.

Use preset configurations or build custom rules with the intuitive visual editor. No need to remember the robots.txt syntax.

Output follows the official Robots Exclusion Protocol standard, ensuring compatibility with all major search engines and crawlers.

Loading tool...