• Hi Murali,

    We’ve come across a critical issue since moving to HTTP v1 from Legacy Firebase.

    We use PNFPB REST API to send mobile app (not web app) notifications for BuddyPress private messages. Since updating to HTTP v1 we noticed that:

    Messaging became unreliable.

    • E.g. David would attempt to message Ana, but the request would timeout. To David, it would appear that the message had never been sent. However, Ana actually received the message multiple times. In our Android app, the message would be received twice. In the iOS version, the message would continue to be received repeatedly until the app was shut down.
    • This issue seemed to only occur for specific users (i.e. it was not impacting every user). This made it harder for us to test/isolate the problem, and was concerning as we didn’t know how widespread the issue was.
    • In Postman, we could see that the server executed the send request very quickly, but was very slow to reply back to Postman that the message had been sent, causing it to timeout.

    Our server CPU usage was significantly higher than it usually would be (e.g. 80%, when it normally sits between 5-10%).

    • E.g. four or five users being online would cause significant CPU spikes, which we had not encountered previously.
    • This caused our servers to become very slow in responding, with timeouts for some users, as above.
    • Our server has more than enough capacity to handle a much higher number of active users, and regularly does so.

    These issues were resolved immediately once we moved back to Legacy Firebase, when using version 1.89 of PNFPB; or by rolling the plugin back to version 1.69, where we’re able to continue using HTTP v1 Firebase. We rolled it back to 1.69 as we knew this version had worked for us previously. We haven’t tested other versions to know when this issue started.

    Given Legacy Firebase is being retired on the 21st of June, could these issues be looked into as soon as possible?

    Thanks.

Viewing 10 replies - 1 through 10 (of 10 total)
  • Plugin Author Murali

    (@murali-indiacitys)

    Private messages/friendship request/accept in httpv1 firebase is almost similar to legacy firebase because it will send notifications only to recipient firebase subscription tokens (individually). I think you are using only these 3 options right? If you use other options like activity/comments… then httpv1 version will send notification default internal group in firebase server (not based on firebase subscription tokens in legacy). Due to this difference, if you use httpv1 then for every site it requires complete different separate project under which web,android,ios apps needs to be created.

    if we use same credentials across different websites, corresponding webview apps using httpv1 then for activity/comment … notifications duplicate notifications can occur because httpv1 activity/comments/other options are based on internal group in firebase server (Except for pvt msg/friendship request/accept)

    Today i tested android app using 1.89 version of this plugin, i got notification.

    Different between legacy and httpv1 in sending notifications is, legacy uses server key but httpv1 uses gets oauth token for every request/call to httpv1 routine based on service account json credentials which you uploaded in admin area and using that token, it will send details to firebase to send notification. This oauth notification routines which i am using are default google’s firebase php routines, this will provide addition security for push notifications.

    I will look into your issue and i will let you know in few days. If you send me screenshot console log of your site or any other relevant screenshots or video to support email id mentioned in admin area, then it will be more helpful

    Thread Starter Nifty

    (@niftythree)

    Hi Murali,

    Thanks for your quick response.

    We’re only using private messages; we’re not using other options, such as activity, friendship requests, comments, etc.

    The user receiving the private message will still get a notification, but the requests are significantly slower when using version 1.89, compared to version 1.69 of PNFPB. We’ve done some rollback testing today to try and identify the version where the issue of slow request times started. From this testing, it appears that the issue first started in version 1.76.

    Please see the below screenshots from Postman when replying to an existing message thread using the BuddyPress REST API endpoint: /wp-json/buddypress/v1/messages
    Both of these screenshots were taken after approx. 50 messages were sent in the thread, to simulate two users chatting (this is a regular scenario we encounter).

    PNFPB version 1.89 using HTTP v1 Firebase
    Requests take a significant amount of time. If the server load is higher, requests will take longer and can occasionally timeout. We saw similar slow request times when testing versions 1.76, 1.80 and 1.87, for example.

    PNFPB version 1.69 using HTTP v1 Firebase
    The request time is significantly quicker with this setup, compared to using PNFPB version 1.89. We saw similar request times when testing versions 1.70 – 1.75.

    From our testing, there does not appear to be an issue with either versions 1.69 or 1.89 using Legacy Firebase (i.e. request times are consistently less than 1 second).

    Thanks.

    Thread Starter Nifty

    (@niftythree)

    Hi @murali-indiacitys

    Do you have any updates on this?

    Thanks.

    Plugin Author Murali

    (@murali-indiacitys)

    I compared the versions, only difference, earlier there is separate logic for desktop and for webview. Since httpv1 notification api has options for both mobile app and desktop, single logic is used to process all the tokens. I did load testing by repeating same token more than 10 times like userid with more than 10 devices and i sent private messages repeatedly it worked without problem. For every private message there will be separate token and it will send notifications to all devices for that particular userid using same oauth token. In last week’s latest version i have also removed some unwanted logic while subscribing for notifications using httpv1. Till now i did not face any problem

    Thread Starter Nifty

    (@niftythree)

    Hi Murali,

    Thank you for the update. We really can’t identify why v1.69 has worked well for us, but other versions haven’t.

    However, we have found that this issue is predominantly for users with a large number of different device tokens, (e.g. 12+, although we have some test users that have over 50+ tokens, where the issue becomes even more prominent). If we manually delete old tokens, then private message speeds improve significantly and server load decreases. i.e. the more tokens a user has, the slower private messaging becomes, and it increases server load. When a user has a large number of different device tokens, is there a way the process can be improved to prevent the significant impact on performance?

    Are you able to test the issue by sending message replies in an existing thread using the BuddyPress REST API, to a user who has a large number of different device tokens (e.g. 20), and then manually removing most of these tokens? Each time we’ve repeated this test, we’ve noticed a significant difference in performance.

    This would be doing POST /wp-json/buddypress/v1/messages

    If needed, please see the reference handbook for arguments here: https://developer.buddypress.org/bp-rest-api/reference/private-messaging/#start-a-new-messages-thread-or-reply-to-an-existing-one

    Thanks.

    Plugin Author Murali

    (@murali-indiacitys)

    Hi

    If private messages are sent repeatedly …more frequently then it will be like live chat. If private messages are used like chat then it will overload the firebase server because oauth token and api connecting time extra , It is better to use push notifications every 1 or 2 hours not like chat application. I will try to add new scheduling option for private messages like how scheduling option for activities, so that every 2 or 4 hours, general push notification can be sent saying
    “New messages are available in your inbox” with link to private messages section according to scheduled. If we use private messages like live chat with more frequent messages then it needs to get oauth token as well as it needs to connect to firebase often to send push notification, it will affect websites as well as Firebase , users will get more and more push notifications often. I will add scheduling option for private messages with general title and general content like i said above,so that when ever new unread private messages are available for particular user then it will send notification like reminders to check their inbox or private message page for new messages that will be better option for private messages.

    If we follow above scheduling method to send general message then we can send it to a device group instead of sending to individual ids and device group will be updated with user id subscription tokens whenever there is new messages sent within the scheduled timeframe. So that it will reduce overload in server also.

    • This reply was modified 9 months ago by Murali.
    Thread Starter Nifty

    (@niftythree)

    Hi Murali,

    Thank you. Our users frequently use it like a live chat. What would you advise we do in this situation (apart from scheduling notifications)? We’d like the notifications for new messages to come through in a timely manner to encourage user engagement with our app. Would OneSignal be better in this case from your point of view?

    Thanks.

    Plugin Author Murali

    (@murali-indiacitys)

    It is your choice, if you think Onesignal WordPress plugin is suitable then please contact them, I do not think Onesignal plugin supports BuddyPress, due to this i added Onesignal as additional provider in this plugin apart from Firebase to support BuddyPress related push notifications. Firebase is always better and free , logic is coded as per Google Firebase httpv1 notification documentation, for one to one notification we need to use loop to send notification to individual tokens otherwise we need to create several device groups in Firebase for every user so that notification will be sent to all devices in single api call, inorder to avoid creating device group for every user, I am using loop to process notifications which is same as in 1.69 version. In legacy all device tokens are sent in single Firebase api call in an array but in httpv1 that is removed, if we want to send it to individual user devices, httpv1 api has option to have only one device token(not api oauth token) per call otherwise we need to create device group for every user in Firebase, I will try to check how many device groups can be created if there is no limit then it can be used but i find looping through device tokens is better for private messages.

    • This reply was modified 8 months, 4 weeks ago by Murali.
    Plugin Author Murali

    (@murali-indiacitys)

    If you want to use onesignal using this plugin then for private messages, this plugin will call onesignal api only one time with recipient userid and onesignal will take care of delivering notifications to device tokens associated with particular userid. Device tokens from onesignal will not be stored in PNFPB WordPress table instead it will add one record for every user with user id saying onesignal in device token and it will be used by PNFPB javascript to communicate with Onesignal to store User id from WordPress as external id in Onesignal database. If you refer onesignal dashboard under subscription page, WordPress user id will be stored as external id (in the settings of subscription records page enable external id column to be visible in onesignal dashboard from icon in top 3-lines icon). External id is stored using this plugin using onesignal api when user signed up for Onesignal notification after logging. In order to use onesignal plugin with PNFPB, please install onesignal wordpress plugin enter onesignal dashboard and in PNFPB plugin enable Onesignal option to send notification for BuddyPress related private messages.

    Thread Starter Nifty

    (@niftythree)

    Hi Murali,

    Thank you for your continued support and the information, we appreciate it. ??

    That all sounds promising. We will have a look at it and see if it’s better suited for our use.

    Thanks again.

Viewing 10 replies - 1 through 10 (of 10 total)
  • The topic ‘Critical issue since moving to HTTP v1 from Legacy Firebase’ is closed to new replies.