Robots.txt quastion Yoast vs Rank Math

Ok, so i notice something very wired.

I notice Rank Math (compare to Yoast) Rank Math put this lines on the robots.txt:

Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

before long time ago Yoast was put this line as well, but i ask ChatGPT and i understand today WordPress do it by default (ChatGPT answer):

Rank Math includes the lines:

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

This is done to ensure that search engines are blocked from crawling the /wp-admin/ directory, which contains sensitive information about the backend of a WordPress site. At the same time, it allows access to admin-ajax.php, a file that's necessary for certain functionalities like handling AJAX requests (used for dynamic elements on the site).

Yoast, on the other hand, does not include these lines by default because WordPress itself includes similar rules in the core robots.txt file or meta tags when the wp-admin directory is set up correctly. Yoast assumes that WordPress’s native mechanisms handle this aspect.

2. Default Philosophy
Rank Math tends to have a more security-conscious default configuration, explicitly blocking certain areas to avoid any potential crawl issues.
Yoast relies on the default WordPress behavior and assumes that webmasters using their plugin are more likely to customize the robots.txt file based on their specific needs.

Now my quastion is, why you put it by default in the Rank Math plugin?

Maybe better to remove it? if WordPress core robots.txt do it by default, is better to keep the robots.txt cleaner much as we can, and “/wp-admin/admin-ajax.php” is important for API REST POINT.

So not better remove this 2 lines from the robots.txt and avoid trouble? keep the robots.txt much cleaner?

Waiting for your recomendation.

Regards,

Nadav.

The page I need help with: [log in to see the link]