/events-calendar/?ajaxCalendar hit 1000’s times by bots, eats bandwidth

  • Hi. I’ve been hit by bots eating all our bandwidth for the 3 or 4th time and each time it takes the site down.

    The bots are hitting the ajaxCalendar link in the calendar page and generating 1000’s of requests for a small site on cheap hosting. We have seen 18K hits in a day for a site that has 30 pages and normally sees a few hundred hits a month!

    Looking in the AWSStats accesslogs I see many many entries like that below. this batch had 6K hits in 4 hours until I blocked the bot in htaccess. 

    [07/Dec/2023:17:03:32	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1181698652&mo=12&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:34	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1286001320&mo=11&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:36	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1286001320&mo=1&yr=2024 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:39	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=2118010445&mo=9&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:41	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=2118010445&mo=11&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:43	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=2118010445&mo=12&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:44	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1753992886&mo=10&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:47	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1753992886&mo=8&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:48	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1753992886&mo=12&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:51	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=22108654&mo=10&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:53	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=22108654&mo=12&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:55	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1201090799&mo=9&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:03:57	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1201090799&mo=12&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:04:00	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1201090799&mo=11&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:04:04	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=553390698&mo=12&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:04:05	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=553390698&mo=10&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)
[07/Dec/2023:17:04:07	+0100]	GET /events-calendar/?ajaxCalendar=1&calendar_size=%E2%80%9Dlarge%E2%80%9D&id=1325864577&mo=10&yr=2023 HTTP/1.1	200	Mozilla/5.0 ... (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)

    Is there a way to stop the bots following the ajaxCalendar link from in EM or can you add a nofollow to the link. Anything that will stop these bots killing our site.

    thanks Colin

    WP 6.4.2
    EM 6.4.6.4

    The page I need help with: [log in to see the link]

Viewing 14 replies - 1 through 14 (of 14 total)
  • Plugin Support angelo_nwl

    (@angelo_nwl)

    Hello,

    I’m going to let the Devs know about this.

    podvodnik

    (@podvodnik)

    Actually I have the same problem. Its an amazon server from Singapore, which wants connect to Event Manager every 4 minutes.

    It started mid December.

    WP 6.4.2.
    EM 6.4.6.4.

    I’ve got exactly the same problem, to the extent that my page is unresponsive, because it’s on a small shared webhosting plan and maxing out CPU usage.

    I banned some of the worst offenders via .htaccess, activated softaculous bot protection and use the blackhole for bad bots plugin. First measure helped, so I can’t verify if the others would be useful.

    I am not experiencing this same problem but it is the facebook and bing bots. They are excessively accessing the ajaxCalendar.

    Has there been any progress on this issue?

    I am limiting bots via WordFence but I am not sure this will be enough.

    awaremitten

    (@awaremitten)

    @angelo_nwl any news on this one?

    I’ve got the same problem. Any news about this???

    This is happening us as well.

    Does anyone have a solution?

    /calendar/?id=1438129239&ajaxCalendar=1&mo=5&yr=2024

    The id is changing every time on GET requests, which is causing the bots to get caught in a loop. The bots can’t tell they shouldn’t be following that link

    Would blocking GET requests to that URL work? I believe when used as intended they are supposed to be ajax POST requests.

    • This reply was modified 5 months, 1 week ago by Seagull87.
    Plugin Author Marcus

    (@msykes)

    Hello Everyone,

    We need to evaluate this a little more but meantime we’ve made some solutions that’ll help a lot we hope.

    We’ll remove the id= querystring and we can add it on via JS, that’ll help immediately and will be out in today’s update.

    Additionally, we’re adding a calendar_nav_nofollow parameter you can add to your shortcode. We’ll elaborate further on this in a following release, but you can also make that a default value set to try by hooking into the em_calendar_get_default_search filter, the array key is in there.

    If you have multiple calendars spread out around your site, I’d recommend adding that nofollow argument and leave maybe one without it.

    Ugh, same issue, from Facebook. Tried adding the shortcode parameter, no luck (in fact it looks like it might have gotten worse).

    Version:?6.4.9
    WP 6.5.5?

    We’ve had the facebook spider blocked via useragent for over a week now and they are still constantly requesting the calendar with the id= query string. I assume things won’t calm down until facebook have exhausted all the links they’ve already gathered. Hopefully they shouldn’t be picking up any more new links now thanks to the last update.

    Chiming in to +1 this thread. I’ve had this issue happening for a while to the point that it was preventing real users to access my websites, and only today I realized it was because of the bots constantly querying the ajaxCalendar endpoint. After blocking both Amazonbot and Facebook’s bot user agents on Nginx, the issue seems to be solved and both CPU and bandwidth consumption dropped gigantically. Would recommend going this way if you can afford to block both bots.

    danifly

    (@danifly)

    I have the same issue with Facebook bot, across all 5 of the sites I have using Event Calendar. They are using 12GB of bandwidth a month and more! Need a fix asap. We have the latest version so fix above doesn’t seem to have worked.

    • This reply was modified 4 months ago by danifly.
    Plugin Author Marcus

    (@msykes)

    We need to work closer with people on this, ideally with more direct feedback regarding access logs that may contain semi-sensitive info, as we need to know what page they are shifting from/to. You can get in touch via our contact form for privacy.

    We’re working directly with some pro customers to mitigate this, we not allowed to ask you here for FTP/login info as per the .org forum policies.

    That said, so far the customers with this issue are on slightly older versions and we’re pending feedback after updating. What’s important (so far as per my findings) is removing links with id=… from getting crawled, because that ID is unique and therefore each page load creates a unique url and therefore endless navigation back/forth in the calendars.

    For example, our demo site is on the latest versions and there’s no id in the navigation URLs anymore. Robots won’t necessarily respect the nofollow tag (they may follow it and not index it, but that’s still a hit on your server). Our demo site also was sluggish recently, I think the cause may well have been the same, but recently (coincidentally after fixing this in recent updates) load is down to optimal again.

    @flagship890 may have a point here too:

    I assume things won’t calm down until facebook have exhausted all the links they’ve already gathered. Hopefully they shouldn’t be picking up any more new links now thanks to the last update.

    edit your robots.txt file takes about 24 hours but this worked for us.

    User-agent: *
    Disallow: /wp-admin

    User-agent: Amazonbot # Amazon’s user agent
    Disallow: /calendar/ # disallow this directory

    User-agent: * # any robot
    Disallow: /calendar/ # disallow this directory

    User-agent: meta-externalagent
    Disallow: /calendar/ # Disallow a specific directory

    you may need to tweak the disallow rules specific to your situation but this immediately solved our issue with the bots rather than blocking it out right which would still consume bandwidth and logs.

Viewing 14 replies - 1 through 14 (of 14 total)
  • You must be logged in to reply to this topic.