• Hi, if I post in the wrong forum please let me know,

    Here is my issue. my website gets a lot of hits from search engine with the URL like this:

    140.205.69.138 tripwinego.com – [25/Oct/2022:21:26:14 +0000] “GET /cn/category/news-opinion-analysis/?_page=2/page/12//page/6//page/3//page/4//page/13//page/3//page/5//page/5//page/4//page/19/ HTTP/1.1” 200 29283 “-” “YisouSpider” | TLSv1.2 | 0.966 0.971 0.972 MISS 0 NC:000000 UP:SKIP_CACHE_SET_COOKIE

    Obviously, the URL is wrong, but it consumes a lot of resources on the server, and I try to avoid that. My questions are:

    1. Does anyone knows why the search engine keep crawling URLs that don’t exist (?_page=2/page/12//page/6//page/3//page/4//page/13//page/3//page/5//page/5//page/4//page/19)
    2. Can I block these URLs? Currently, I am using Cloudflare to block these URLs, it works but I would prefer not to use another external service.

    Thanks for your help.

    • This topic was modified 2 years, 1 month ago by Jan Dembowski. Reason: Moved to Fixing WordPress, this is not an Developing with WordPress topic

    The page I need help with: [log in to see the link]

Viewing 2 replies - 1 through 2 (of 2 total)
Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Block URLs with Rule’ is closed to new replies.