additional useragent added

the following is the robots.txt of my website:

User-agent: * Disallow: /wp-admin/

Allow: /wp-admin/admin-ajax.php

User-agent: * Disallow: /wp-content/uploads/wpo-plugins-tables-list.json

Please help me remove ” User agent * ” in the last line.

• This reply was modified 1 year, 7 months ago by janardhan.

@janardhanjp It is not possible to remove single line. You can remove all the lines added by WP-Optimize by adding following code in your theme’s functions.php file

add_action('after_setup_theme', 'ab123_remove_robots_txt_entry');
function ab123_remove_robots_txt_entry() {
	if (function_exists('WP_Optimize')) {	
		remove_filter('robots_txt', array(WP_Optimize(), 'robots_txt'), 99, 1);
	}
}

• This reply was modified 1 year, 7 months ago by Venkat Raj.
• This reply was modified 1 year, 7 months ago by Venkat Raj.

the following is th Robots.txt as on today:

User-agent: *

Disallow: /wp-admin/

Allow: /wp-admin/admin-ajax.php

User-agent: *

Disallow: /wp-content/uploads/wpo-plugins-tables-list.json

the robots text analysis is telling like this,

Results of robots.txt analysis

1 error

lines used

User-agent: *???? Multiple ‘User-agent: *’ rules found

So there are two User Agent:*

So pl try to remove the last but one line in robots text

that is remove User Agent:* before

Disallow: /wp-content/uploads/wpo-plugins-tables-list.json

which is repetitive

@janardhanjp At this moment there is no option to remove that single line. To implement this, could you please let us know which robots.txt analyzer throws error? Thanks

Yandex Site diagnostics says,

Robots.txt contains errors. This can lead to incorrect crawling and indexing of your site.

Check?the robots.txt file.

User-agent: *

Disallow: /wp-content/uploads/wpo-plugins-tables-list.json

WP Optimize was added by me recently. After that I found this error message in Yandex.

Thanks Venkat Raj. I have added the code given by you in functions.php. And the lines added by WPoptimize are gone. Thank you very much.