Partial Exports Running Multiple Servers on AWS Behind a Load Balancer

davedi — new to Woo and long absent from WordPress programmer here. I’m diving back into both by trying to answer other people’s support questions. Given my lack of recent hands on experience I’m not sure if I’ll be able to give you a total answer, but I hope I’ll be able to help.

The problem here appears to be how the export’s been implemented. When you click the Export button, WooCommerce doesn’t do all the work in a single request. Instead, your browser’s making several ajax requests over and over again, each one doing part of the export. When the export’s complete, WordPress hands you the finished file. This sort of thing is typical in web applications, and is usually done to avoid timeouts in large order exports.

So, (from one point of view) the problem here is your load balancer is setup to send users along to **any** of your front-end servers. So, those multiple ajax requests each do part of the export on an individual server. This is a common problem when scaling out a web application that was designed for use on a single server to multiple servers.

One solution to try is configuring your load balancer to use something like Sticky Sessions. This is a common feature in modern load balancers, and helps ensure a user is sent to the same frontend web server every-time.

Another solution that’s common is to configure your AWS architecture such that every request to wp-admin is served by a single, dedicated box. The idea here is that admin traffic is usually light enough to be handled by a single web server.

Hope that helps!

Hello Dave, Alan

Should I add option “Don’t use progressbar” and get results in single request( if this option is checked)?

But it might not work for 300+ orders, because webserver usually sets max execution time to 30 seconds.

Hello Dave, Alan

please, email me [email protected]
I’ll provide the version with new button “Export[w/o progressbar]”

thanks, alex

“Export[w/o progressbar]” added in 1.2.1

Hi Alex.

Sorry for the delayed response… gmail put these emails into the “social” bucket and I just saw your last response.

Thanks for working in the “single request” feature. I will try the new plug-in with multiple servers and let you know.

Understood about the 300+ orders. One workaround could be using an S3 bucket (if configured in your settings) to store the chunks of files instead of the local server.

Thanks again,
Dave

Hi alanstorm. Thanks for your suggestions. I am sure both would work to solve the export issue. In fact, I just enabled sticky sessions and that did work consistently.

We went away from sticky sessions early on when we were running just a few smaller servers that were failing during traffic spikes because (it seemed) the bulk of the traffic did not spread to the new servers in a timely fashion. Perhaps now, running larger and more servers, sticky sessions will not be an issue.

Algo.plus, I will try your new version in case sticky sessions are an issue for us. Any issues with temporarily setting the max execution time in your script higher (via a configurable plugin setting) when using the no progress bar option?

Thanks again to you both.
Dave

hi Dave

I can add following line
set_time_limit(0);
but I’m not sure if it will work on your server

thanks,alex

Hi algol.plus.

Thanks for your help on this issue. I wanted to let you know how it went with version 1.2.1:

I tried version 1.2.1 and have a couple issues, but **I think (see issue #1 below)** I am getting all of my orders in the export file.

The issues:

1) The CSV file that is produced does not appear to have line breaks. When I export, I get the header and all the orders on one line.

2) Even though I choose CSV as my export format (and that is what comes out) I am always prompted by my browser to save a “.xls” file. If I forget to change the file extension (or if somebody does not know to do this) MS excel can’t open the file.

2a) I also noticed the “file extension issue” in the pro version (Advanced Order Export For WooCommerce, Version: 1.1.13) of the plugin that I purchased as well.

Thanks again for your help.
Dave

hi Dave

1) What software do you use to open csv ? Windows Excel?
Can you send sample file to [email protected] ?

2) Can you switch formats ( csv -> xls -> csv) ?
Will these actions change filename mask ?

2a)yes, Pro and free use same UI forms/elements

thanks, alex

Hi Alex.

Sorry for the delayed reply…

1) I use Microsoft Excel to open the CSV files, but I also see a single line in notepad and notepad++. I am running these on a Windows machine.

I’d have to clean up (remove a lot of customer data) before sending you a sample… I can tell you that the CSV file that is not working does not have Carriage Returns or Line Feeds ( “\r\n” ) after each record. Let me know if you would still like the file and I will clean it up.

2) I can change the “xls” extension to “csv” when I save (the working file) and that does open up in Excel as you would expect a CSV file to open. So that is a work around, but some people may not know to do that or just click too quickly and wonder why the file is not opening properly.

If you end up adding the time out, I think it would be good if it were configurable just so people have the option and can avoid “hanging” their server. At this time, I don’t know that you need to as the time out (at least not for me) as I am able to easily export hundreds of records in just a few seconds.

Thanks again for all of your efforts.

Dave

hi Dave

1) Can you create simple profile (mark only order id and order date)
?
2-3 rows is enough , my email is [email protected]

2) No need to rename files.
Can you click on (*) and switch Csv -> Xls -> Csv formats?

thanks, alex