The Forgotten Split Test – a 3 Year Test!

Posted by David Whitehouse

I was writing a post for the Dave Naylor blog about Google Analytics and Website Optimizer when I thought “have I ever done a split test?”, so I logged into Google Website Optimizer briefly and noticed the following test from 9th May 2007! And it was still running!!!

A graph showing conversion rates changes in a 3 year split test

I’m not entirely sure what is happening over time, but it seems as those my initial test increased conversions by around 33%. But it seems over time that the effectiveness of this has decreased. The test in question was for an old site, originally designed by SEO Doofus, when he was at 9XB (sorry to point the finger Carps).

Here is the original:

And here is the alternative which performed significantly better (awful design I know – but it shows how it was a user interface problem rather than a “style” problem):

So this awfully designed changed actually increased conversions coming through the home page by 35%!!!  So can anyone explain the convergence of the split test over time? I’m really perplexed!!!

Feedback Form