Build a valid robots.txt with allow/disallow rules, crawl-delay and sitemap reference.
Fetching live data…
The Robots.txt Generator is a free, in-browser tool that builds a valid robots.txt file from simple options — without you having to remember the exact directive syntax. You choose which user-agents to target, add allow and disallow paths, set an optional crawl-delay, and point search engines to your sitemap. The tool outputs a clean, correctly formatted file you can save to the root of your domain. Everything runs locally in your browser; nothing is uploaded.
robots.txt in your site's root directory.* or name a specific bot./admin/ or /cart/ from being crawled.Sitemap: line pointing to your XML sitemap.To block your admin area but allow everything else and declare a sitemap, the tool outputs: User-agent: * — copy it straight into your root file.
Disallow: /admin/
Allow: /
Sitemap: https://example.com/sitemap.xml
Yes — completely free, with no signup and no limits. It runs in your browser and never stores your input.
It must live at the root of your domain, at https://example.com/robots.txt. After uploading, confirm the rules behave as intended with the Robots.txt Tester and make sure your sitemap resolves with the Sitemap Finder & Validator.
Yes — a stray Disallow: / can block your whole site from being crawled. Always double-check before deploying. Our robots.txt guide explains each directive and common pitfalls in detail.
Guides, tips and tutorials hand-picked for this tool — go from running a check to actually fixing it.
SEO is how you make a website rank higher in search engines so it earns free, organic traffic. This complete beginner's guide explains what SEO is, how search engines work, the four pillars (keyword research, on-page, technical and off-page), GEO for AI search, and a simple step-by-step roadmap you can start today.
Read article →robots.txt controls what crawlers can access. One wrong line can deindex your whole site — here is how to get it right.
Read article →llms.txt is a simple Markdown file that gives AI models a clean, accurate summary of your site or product. Here is what it is, what to include, and how to create one.
Read article →DNS records are the hidden settings that route a domain's traffic, email and verification. Here is what A, AAAA, CNAME, MX, TXT and NS records do, how to read them, and how to check each one for free.
Read article →Updating old content is often the highest-ROI SEO task. Here is a simple refresh workflow.
Read article →Clean, readable URLs help users and search engines. Here is how to structure them.
Read article →Create a valid llms.txt (and optional llms-full.txt) so AI models — ChatGPT, Perplexity, Google AI Overviews — can understand and cite your site. Add your site name, summary and key URLs, then copy or download.
Open tool →Generate clean, copy-ready title, description, robots, Open Graph and Twitter meta tags.
Open tool →Generate WWW/non-WWW, HTTP→HTTPS and 301 redirect rules for Apache .htaccess.
Open tool →Produce valid JSON-LD structured data for Organization, Website, Article or LocalBusiness.
Open tool →Create trackable campaign URLs with utm_source, medium, campaign, term and content parameters.
Open tool →Convert any title into a clean, lowercase, hyphenated, SEO-friendly URL slug (handles Turkish chars).
Open tool →