Evaluate your Website for Split Testing

The graph below shows the total number of visitors required (Y axis) to reach a statistically relevant test result based on a starting Conversion Rate of 0.5%, 0.75% and 1.0% (X axis) for an A-B test (testing 2 versions of 1 variable on your web page) producing a 25% improvement in the conversion rate or a 50% improvement.

Eyeball your website's present conversion rate on the x-axis and using either the 25% (blue) or 50% (red) inmprovement line (whichever you think will most closely apply to your situation), you'll see the approximate amount of traffic you will require for a reliable test (divide the traffic # by your avg. daily traffic for a time estimate in months).

Your necessary sample size for statistical relevance is inversely proportional to your

  • level of traffic
  • starting conversion rate
  • the conversion rate improvement for your test

    • In other words, the more traffic you have AND the higher your control (starting) conversion rate AND the more improvement your test produces, the shorter the amount of time needed to run your test to a statistically relevant conclusion.

    Therefore, it's obvious that you need to choose your test variables wisely- going for the biggest conversion rate improvement variables first before fine tuning with less important variables.

    Even if your current traffic and conversion rate yields 4 months to complete a test, that equates to almost a 100% improvement in your website's conversion rate over the course of a year (assuming a 25% improvement with each of the three tests you run during that year). And that's not to be sneezed at!

    		 
     

    A little glossary help:

    Statistical relevancy:

    For true reliability, tests should run to a 90% relevancy level, which is used in the graph's computations. All ez-TEST purchases include a link to a statistics site that will easily compute your relevancy level for you.

    Conversion Rate:

    (Sales/Visitors)*100

    Use sales or subscriptions, whichever you're measuring.

    Use UNIQUE visitors, i.e. a specific visitor is only counted once, not every time they visit your site.