Stop internal search spam

Viewing 2 replies - 1 through 2 (of 2 total)
  • Plugin Author Anh Tran

    (@rilwis)

    Hi @alriksson,

    Thanks for bring it out to our attention. Regarding internal site search, this is what we’ve done:

    • Blocking search engine bots to crawl them in the robots.txt (similar to what Yoast does for it internal site search cleanup) => so we don’t waste the server resource by serving these pages to bots.
    • And in case bots still crawl the pages (just because they can ignore the robots.txt), then we add robots “noindex” to all internal site search results page => so these pages are not showing in the SERP.

    If that’s not enough, with the redirection module, you can also create a custom redirect that can redirect all internal site search requests to a “black hole” page and then block them (as suggested in the Yoast’s issue).

    Thread Starter Rookie

    (@alriksson)

    The last one would be great to create a black hole and you log all ips going there so we can block them on edge or server level. Log user-agent + ip

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Stop internal search spam’ is closed to new replies.