• Resolved Nikoi

    (@iokinpardo)


    Hi team, does AI Engine support “Implement retries with exponential backoff” technique ? I think this is super important as this technique embraces the fact that cloud resources might intermittently be unavailable for more than a few seconds for any reason. In our case, OpenAI models have their RPM limits, that once reached the reponse received is bad and it stops for a few short period.

    So I consider that for a minimal good UX something like this should be applied to solve high volume concurrent users, or maybe there is something Im not aware of than can manage those limits situations for users. I know there exists an excerption hook that handles AI errors, so the idea from AI Engine is to be able to retry the call for X attempts during Y time to solve it automatically.

    Microsoft Azure is sugesting applying this technique.

    Thanks in advance.??

Viewing 1 replies (of 1 total)
  • Plugin Author Jordy Meow

    (@tigroumeow)

    Hi,

    This is not implemented, and I will never do this on the server-side, as WordPress, PHP, and even hosting services have all limitations in terms of number of requests performed simultaneously and timeout rules/policies.

    However, it could be implemented on the JS side, on different modules. That could be integrated in the chatbot for example, I don’t see any issue with that ??

    Can you vote for that feature here? https://trello.com/c/z93IGt72/133-exponential-backoff-for-the-chatbot-on-the-js-side. I have only mentioned the integration for the chatbot for now, but it could also be done for the forms easily.

Viewing 1 replies (of 1 total)
  • The topic ‘Retries with exponential backoff’ is closed to new replies.