🤖

robots.txt Generator

Local processing · no uploads

Visually build robots.txt with multiple rules and sitemaps.

Multiple groups, each bound to a User-agent. Produces a standard robots.txt.

Rule groups 1

User-agentCrawl-delay (seconds, optional)Allow (one path per line)Disallow (one path per line)

Sitemap URLs (one per line)

User-agent: *
Disallow: /admin
Disallow: /private

Sitemap: https://example.com/sitemap.xml

Related tools