Service Disruption – 24/02/2014

Hello, this explains why it wasn’t working for me. I am due to make a website live in the next day or so. Will it be fixed by then? If not I may have to find a different plugin which is a shame as I like this one.

This is really very bad – how soon it will be working fine again? and whats the guarantee this will not be happening in the future???

thanks God my site was on dev.

It should be up and working again by tonight. I won’t stop until it is live and fully functioning again.

This move was planned as it separates the API from the website, onto its own Linode VPS. Once it is configured and running, I can then create a snapshot of the server and create more on demand. Then I can have them behind a load balancer.

This has several benefits:

1) I can spread the load between multiple servers, reducing the extreme number of connections required per server.
2) I can spawn new servers quickly if demand requires or if other servers suffer problems.
3) I can easily move this setup to another service (e.g. AWS) if needs be.

Update on the progress…

I almost have the server configured as needed and I am beginning to test the API code to ensure it is working as expected in the new environment. There are a few small changes required so far and I am working through them as I discover them.

Just wanted to say thank you for your efforts, Peter (and for alerting us to the situation here). This is an excellent plugin. It sucks that it’s down right now, but it sounds like you have everything in hand. I’m going to relay the information you gave us here to my client and let them know it should be up and running by tomorrow. Thanks again!

Me too I wanna thank you for your great plugin and for keeping us updated. Much appreciated. Keep up the good work!

Hear hear!! An excellent plugin and thanks for keeping us updated. Keep it up!

Thank you for all the support, I really appreciate it.

I have just got the backup working, so switched the DNS back to that temporarily. So as it propogates the plugin should start working again.

I have the API running on the new VPS but have to fix some problems which have cropped up from the change in server setup / configuration. As soon as I am happy it is working I will switch the DNS back to the new server and it should seamlessly switch between.

The backup is up and functioning well, serving around 40 requests/second at the moment and picking up all the time.

The new server is almost ready, I need to make some more server config tweaks and setup the load balancer and then I will switch the DNS tonight and it should seamlessly move between them.

There may be the odd issue over the next 24 hours due to the DNS changes taking time to propogate. It should be fully working for everyone by tomorrow morning.

I will continue to keep a close eye on it and take action where needed. I will follow up tomorrow with a detailed description of what happened and what I will be doing in the future.

Hello, thank you for the support and keeping us up to date. Great plugin! Quick question do I need to re download the plugin?

Hi Ibbdesigns,

You don’t need to re-download the plugin, saving the plugins settings page should force it to refresh data. This should work, but might not due to the DNS changing. It should be back to normal by the morning.

You can read a little more about what happened and what we are doing to avoid it happening again in the future here: https://kebopowered.com/service-disruption-02-2014/

It is a little brief but hopefully gets the message across. I am very sorry to all our users for the disruption to the service and am committed to avoiding this ever happening again.

Hi Peter,

Just wanted to let you know that yesterday was working OK but today it’s giving me the “Sorry, no Tweets were found.” message.

Thanks again!

Hi Jordi,

Thank you for letting me know. I am sorry you are still experiencing problems.

From what I can see (I only log whether the API responded properly or sent a particular error, no details), it is not functioning any differently on the new server, but there could be something about the setup which is causing issues.

There are a couple of things to test, firstly if you save the plugin settings page it will force the plugin to attempt to refresh the Tweet data. Next, if that still isn’t working would you be able to remove and try to re-connect the Twitter account to the plugin please? As this uses the API too so would help to narrow down the possibilities.

Yeah I already tried both; saving settings and resetting the whole connection, no luck.

Does it let you re-add the connection? Or does it fail to forward you to Twitter?