Home » Internet » Internet Marketing

Split Testing Software -- What You Need To Know.

Apr 10, 2008
You've probably heard that split testing elements of your website can improve your conversion rates. And that's true. But there are many kinds of split testing out there. And there are many ways to set up split tests on your website.

I can't discuss every feature of every testing tool out there, but I can give you a major overview of the different kinds of split testing software.

First, there's a difference between multivariate and single variable testing.

With single variable testing you just pick one element on your page to test. Take your headline, for example. If your current headline says "Are you having trouble with tooth decay?", you might want to see how "Do your kids have tooth decay?" works. In a single variable test, you just find a way to rotate each of those headlines on your page, and keep track of how many sales each generates.

Paul Hancox has a simple split testing tool. And Google has one that's free, though it takes a bit to set up all your accounts to use Google's tool. And the reporting is not real-time.

But simple split testing can be discouraging. When you test a new idea against your control version, sometimes the control version wins. In fact, it often wins. You can get a string of disappointing tests all in a row. What would it be like if you could test 10 things at once? Well, you'd be bound to find at least one or two ideas that led to improvement. And what if you could test all those ideas simultaneously with the same amount of traffic it takes to test one idea with simple split testing? Well, it sounds like magic, but that's exactly what multivariate testing can do for you.

So there's the main distinction between single variable testing and multivariate testing. But once you get into the realm of multivariate testing the options seem to multiply again. One difference among multivariate tools is between tools that are hosted on someone else's server, and ones that you install on the same server as your webpages. Hosted solutions can be slower, and more expensive, but you don't have to install any software.

Hosted solutions include Google's Website Optimizer, and Vertster. Installed solutions include the Split Test Accelerator and Kaizen Track.

Taguchi testing is only one kind of multivariate testing. There are also "full factorial" tools, and random independent rotators. When you run a Taguchi test, you run only a small fraction of the possible combinations of your factor levels. You might, for instance, run only 18 pages to test 4374 combinations. This is made possible by orthongonal arrays. This allows you to find the best combination with out running through all of them. You do have to be careful about your factors interacting negatively with each other, but if you take care as you design your test, you can avoid the bulk of the problem with these. The Split Test Accelerator, Vertster, and Kaizentrack are Taguchi testing tools.

Full factorial tools allow you to cycle through all the combinations for your factors. So if you have 3 headlines, 3 offers, 3 P.S. statements, and 3 images you're testing, you will cycle through 81 pages. One advantage of full factorial testing is that you don't have to worry as much about interaction effects. Good and bad interactions should show up in the data. However, if you want to receive this advantage, you have to run many times the traffic through your test as you would with a Taguchi test. You also can't run as many combinations. Google's website optimizer is a full factorial tool.

Some multivariate tools randomly serve the options for each factor independently from what is served for the other factors. This can lead to more flexible test design, but also to more interaction effects and other statistical noise, leading to longer tests.

Now that's just the overview. There are many other features to compare and questions to ask. Does a tester allow you to exclude bots and spiders? Does it allow real time access to data? What kind of reports does it show? What kind of information does it track for each visitor, and is the visitor-level data available for viewing. Etc.
About the Author
Dominic DeLong has used multivariate testing tools for years to improve landing pages.
Please Rate:
(Average: Not rated)
Views: 201
Print Email Share
Article Categories