lundi 10 août 2015

Using javascript to A/B test landing pages, Math.random() not random?

I have a javascript ad that runs on multiple websites. The ad is for an app, and when clicking the ad the user is sent to a landing page where js on that landing page determines whether to show A) landing page A, B) landing page B) or C) redirect the user to app store/play store.

The ad is choosing landing pages dynamically, by setting the parameter utm_content={landingA, landingB, nolanding}, so there are basically three choices:

  • Directly to store (50%)
  • Landing page (50%)
    • Landing A (50%)
    • Landing B (50%)

The way this is chosen is by pure js and the code is like this:

if(Math.random() < 0.5) nolanding;
else {
 if (Math.random() < 0.5) landingA;
 else landingB; 
}

When the user then clicks the ad, he or she is sent to the landing page, and based on the utm_content parameter's value, the landing page is showing A / B or redirect to store without showing anything.

In Google Analytics I made a pie chart to just test if the traffic is distributed 50%, 25%, 25%, but that does not seem to be the case.

enter image description here

As you ca see from the attached image, it does not have the expected distribution. So, my question is:

Is the problem that the Math.random() is not as random as I expect? I've looked into some other questions discussing that issue, but it cannot be that inaccurate?

Or is it Google Analytics not being able to track the utm_content parameter correctly? Could it be an issue with the redirect to stores going too fast, so GA can't track the page view?




Aucun commentaire:

Enregistrer un commentaire