Guides › Platforms

WordPress robots.txt: default rules and how to edit

If there's no physical file, WordPress serves a virtual robots.txt. The core default is:

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Sitemap: https://example.com/wp-sitemap.xml

Editing it

• Yoast SEO: Tools › File editor. Rank Math: General Settings › Edit robots.txt.
• Or upload a physical robots.txt to the web root. A physical file overrides the virtual one.

Watch out for

• Settings › Reading › "Discourage search engines" makes WordPress send noindex (newer versions) or add Disallow: / (older versions). Untick it after launch.
• Don't block /wp-content/ or /wp-includes/, because Google needs CSS and JS to render pages.
• Core sitemaps live at /wp-sitemap.xml. Yoast and Rank Math replace them with /sitemap_index.xml.

Related