Not Equally Distributing Visitors Among Variations

Where are you hosting your site and do you have caching turned on?

It is hosted through WP Engine and yes its a high traffic site so it has caching, here is their public statement on WPE caching.

Next is our EverCache system. Again this is part of our “secret sauce,” so we can’t divulge everything. But, what we can say is that we run the most sophisticated caching system in the world, which we’re able to do because we support only WordPress. This makes common pages (like home pages and feeds) load in 30ms (not including the time to make it all the way to the end user’s browser), protects the back-end system from most traffic and specific traffic from popular plug-ins and 3rd-party services, and yet is tied back into your WordPress system (via our system plugin) so that when the cache needs to be refreshed (e.g. post is updated) we refresh immediately so there’s never stale data. There’s also a lot of special logic protecting your database from various bots and spiders, which for larger sites can represent 50% of the site’s traffic.

We use Wpengine as well!

You will need to contact WPengine support and request them to add a no-cache exception to the file /wp-content/plugins/landing-pages/modules/module.redirect-ab-testing.php

This will prevent the wpengine cache from serving only 1 variation.

We setup preventions for w3 total cache and WP super cache but can’t override the wpengine settings on our end.

The other way to send folks into the different variations, if you don’t want to turn off cache from wpengine for some reason would be sharing the two different urls.

landing page 1a: https://yoursite.com/go/landing-page?lp-variation-id=0
landing page 1b: https://yoursite.com/go/landing-page?lp-variation-id=1
etc.

Those will send folks to the exact A or B variation.