Robots.txt Generator
Generate robots.txt with allow/disallow rules, user-agents, crawl-delay, and sitemap.
Rules
Disallow paths
/admin/
Allow paths
Output
Live Audit (Fetch & Compare)
Frequently Asked Questions
Can I generate separate rules per bot?
Yes, add multiple user-agent sections with their own allow/disallow.
Will it validate conflicts?
We warn if the same path is both allowed and disallowed.
Is sitemap required?
Optional, but recommended for discovery.