How to remove sitemap from robots.txt

In order to simplify the settings pages, we removed the option to remove the sitemap URL from the robots.txt.
Very few people have used the option, and placing the URL there is in accordance with Google’s recommendations.
We do have a filter hook to remove it again.

add_filter( 'aioseop_robotstxt_sitemap_url', '__return_false' );

• This reply was modified 5 years, 5 months ago by Michael Torbert.

Thanks. Some of us have private (but no login) sites, e.g., family, where we want to cache the site’s pages. Members get access by knowing the page URL.

Some caching plugins, e.g., w3tc, use the sitemap as a guide in the caching process.

Even though we may tell search engines to not index a site, individuals or malicious entities may use the sitemap to find private pages.

The two things that AISEO has recently done that makes this more difficult are:
1) hide the ability to rename the sitemap to something not obvious.
2) publish the sitemap path/name in the robots.txt file.

The filters are appreciated, but it would be nicer and more helpful to have left in place easy ways to change the sitemap name and omit the sitemap from the robots.txt file. Another option would be to add a module to your Feature Manager that offers those two options. That way they would be out of sight for most users but available for advanced users.

If a page is marked as private, it shouldn’t show up in the sitemap.

hide the ability to rename the sitemap to something not obvious

Changing the sitemap filename is an edge case, so in order to simplify the settings page for 99.999% of users, we removed this setting in version 2.6, about a year ago and replaced it with an API filter hook. https://semperplugins.com/documentation/aiosp_sitemap_filename/

Similarly, 99.999% of users aren’t going to want to remove the sitemap’s URL from the robots.txt, which is the Google recommendation, so we removed that option in 3.0. It has been replaced by a filter hook you can use as well. https://semperplugins.com/documentation/aioseop_robotstxt_sitemap_url/

I hear your feedback that you’d rather have kept the options, but we’re trying to get out of the habit of including options that barely get used and only add confusion for most users.
Adding these filter hooks is very easy, even for a non-coder. https://semperplugins.com/documentation/how-do-i-use-your-api-code-examples/