• Resolved bbceg

    (@bbceg)


    This morning (06:17 UTC) Stripe started making thousands of webhook requests to our server and eventually overwhelmed it. To give you an idea, we had over 6,500 requests from Stripe in under half an hour. All webhook requests were for the same order (which, after a couple of failed payments due to invalid card details, succeeded). At the time we were running version 5.0.0 of the Stripe plugin. I have since upgraded WordPress, all plugins and updated to PHP 7.4 (from Ubuntu’s 7.2) so the status report is not the same as it was when the problem started.

    I’ve been in contact with Stripe and they say that Woocommerce generates all of the Stripe webhook requests and I wondered if there is somewhere I can see this happening in Woocommerce or on the server. I did not have webhook logging enabled but I have now switched this on.

    The pending webhook requests are still attempting to clear since the server cannot process them fast enough before it falls over (at which point the requests are queued for retry and the cycle continues).

    Might a possible fix for the backlog of requests be to delete the webhook endpoint and recreate this?

    Any thoughts welcome.

Viewing 6 replies - 1 through 6 (of 6 total)
  • I’ve been in contact with Stripe and they say that Woocommerce generates all of the Stripe webhook requests

    That’s not exactly accurate. The webhooks that Stripe sends to your site are, indeed, a result of various payment requests that are sent from your site to Stripe. You can configure which events webhooks will be sent for on your Stripe dashboard. We have a bit of documentation on that here: https://docs.woocommerce.com/document/stripe/#webhooks

    Might a possible fix for the backlog of requests be to delete the webhook endpoint and recreate this?

    No.

    Thread Starter bbceg

    (@bbceg)

    Thanks @wbrubaker. Any ideas how the problem would have manifested itself and where it may have originated from?

    Incidentally, deleting the webhook and recreating it did appear to prevent the remaining requests from firing (they were due at 03:30 UTC and there are no corresponding entries in the NGINX access log). Any ideas why that would be?

    Plugin Support Saravanan S, a11n

    (@simplysaru)

    Hey there!

    Any ideas how the problem would have manifested itself and where it may have originated from?

    This could have been due to some malicious code running or a plugin conflict or due to fatal error etc. It would be hard to pinpoint this without looking at the site, logs, etc. Do you see any fatal errors that coincide this incident?

    Incidentally, deleting the webhook and recreating it did appear to prevent the remaining requests from firing (they were due at 03:30 UTC and there are no corresponding entries in the NGINX access log)Any ideas why that would be?

    I am not certain that these are logged in Nginx by default. Do you see them in the stripe logs at WooCommerce > Status > Logs ?

    Thread Starter bbceg

    (@bbceg)

    Hi @simplysaru

    Thanks for the response. There are no errors showing in the error log around the time of the first Stripe log entry. The NGINX access log does show more than 6500 entries similar to this one in a sub 30 minute window:

    {STRIPE IP} - - [05/Jul/2021:07:17:50 +0100] "POST /?wc-api=wc_stripe HTTP/1.1" 200 5 "-" "Stripe/1.0 (+https://stripe.com/docs/webhooks)"

    The first 450 succeeded before the web server was overwhelmed (after 3 minutes or so) and then the response code changed as the requests timed out or errored.

    Any additional thoughts welcome.

    Plugin Support nicw.a11n

    (@nicw)

    Hi @bbceg

    In view of the sensitive nature of things like Stripe logs, and because we’d really like to help get to the bottom of this, please contact us at WooCommerce.com > My Account > Support. You may need to create an account before you can access that page.

    Please include a link to this forum thread, so that we can keep track of what’s already been done.

    We will be able to help you further there.

    Thread Starter bbceg

    (@bbceg)

    Thanks @nicw that’s been done. I’ll leave this thread open for now in case we can supply a resolution for anyone else coming across this issue in the future.

Viewing 6 replies - 1 through 6 (of 6 total)
  • The topic ‘Thousands of webhook requests for a single order and overwhelming server’ is closed to new replies.