Quick summary
Blocking the SemrushBot in WordPress requires editing your robots.txt file via a dedicated plugin to prevent the crawler from accessing your site. Adding a Disallow directive for SemrushBot and clearing your cache ensures the rule takes effect immediately.
Steps
- Head over to Plugins in the left sidebar of your WordPress dashboard.
- Select Add New from the dropdown menu.
- Search for Robots.txt in the plugin marketplace.
- Select and install a robots.txt editor plugin, such as Robots.txt Editor.
- After installation, navigate to Settings > Reading in the left sidebar.
- To block the SemrushBot, type User-agent: SemrushBot followed by Disallow: / in the robots.txt editor.
- Finally, clear the cache by going to Toolbar > WPRocket > Clear Cache to apply the changes.
