View Full Version : A/B split test our home page?

10-07-2012, 03:16 PM
How do I set this up? I want to change a few things on our home page and test out how well they work.

10-08-2012, 02:06 PM
The easiest way is to make a new article or category page that looks how you want your homepage to look then run the test from the content>experiments section in Analytics using your existing hompage as the A version and your new page as the B version.

I haven't done this since they moved website optimiser in to Analytics, there were a few quirks with GWO and Volusion, so you might have to tinker a little.

Let me know how you get on.

10-08-2012, 04:18 PM
This would imply that you can split test the inside page content, but not the Top bar and left nav?

10-09-2012, 02:11 AM
That's right, this was possible with the old GWO multivariate tests and an awful lot of jquery but with the new content experiments I don't know of a way it can be done.

10-09-2012, 06:15 AM
(noting that I am not a programer)....
In the product pages, we have control over the meta tags. Is it possible to use this control to insert commands that prevent a whole bunch of stuff from loading on the page? This would then provide complete control of a page to use as the "b" in the split test?

10-10-2012, 01:52 PM
You could create a category/ product page and throw in some CSS/js to manipulate the appearance of the top and left nav, however it won't carry through to other pages on the site so the user experience might be a bit odd.

If you go down this route, I suggest using a reduction in bounce rate as your goal as your changes will only really effect the landing page.

11-01-2012, 08:25 PM
Look into Optimizely.

01-02-2013, 12:45 AM
I use Visual Website Optimizer, it works for me, and it's a pretty good interface. I've heard Optimizely works well too and is perhaps simpler though not as powerful.

03-14-2013, 08:44 AM
I use Visual Website Optimizer, it works for me, and it's a pretty good interface. I've heard Optimizely works well too and is perhaps simpler though not as powerful.

I've used both for years, and I'm also a certified VWO partner. I've found that Optimizely is much more powerful. It allows you to use historical data for goals, manually activate tests via a javascript API, and use regular expressions for URLs.

03-22-2013, 10:47 AM
well you have to do it correct if you're going to do it.

Begin broad, test two EXTREMELY different designs. If one wins, make two more designs more like the winner, as you pick out your winners get more and more narrow until you're really just changing colors. Then do it all over again.


1) Don't act on data that is not statistically significant. This is important to prevent from making the wrong decisions
2) Lose your emotional attachment with your designs in favor of revenue
3) Begin testing broadly different designs, narrow down, do it again.

This is science, not your preference, what you think "pops" etc. Do this by the math and you'll be glad you did.

03-26-2013, 09:55 AM
Color me slow, cabin-fevered, or whatever, but what is an A/B split test?

03-26-2013, 10:27 AM
It's a method where you have two versions of something on your site, so you can gauge how it impacts customers and conversions.

Do customers like BIG COLORFUL BUTTONS? :)

Or do they like sad, grey buttons :(

Do customers click more if your link is at the TOP of a page?

Or do they click more if it's at the bottom of your page.

You keep performing these tests, and then apply them. Many, many sites do this and you may not even be aware of it.

03-26-2013, 11:54 AM
Highly recommend doing testing of your site. Customers, unfortunately, do not usually act the way you think they would :-)

Through testing, you can figure out how they act on your site. And each site is unique, so something that may work on another site, won't on yours.

03-26-2013, 01:15 PM
Color me slow, cabin-fevered, or whatever, but what is an A/B split test?

A/B split testing is one of the core methods of improving your site's conversion rate. When you built your site, it was generally a creative process (tangent: Although it should no longer be a creative process which is why there are so many poorly designed sites out there. People are attracted to "pretty" sites and low-cost designs instead of designing from a conversion perspective) where you usually discuss what could happen on your site. Once you launch your site, you have your analytics which answers the question "is it working?"

A/B split testing can be effectively used to find out how closely you're matching the visitor's expectations and allow you to test ideas that will help increase engagement and therefore increased revenue into your pocket. It is much more than changing colors and buttons.

For example, I know if I look at most of your sites your product pages focus on features instead of benefits. Remember:

A feature is what your product or service does
A benefit is what your product does for them

So you could do a test of your main product...page A would be your traditional list of features and then page B would be a list of features. The software would randomly show page A or B to your visitors measuring engagement and/or goals (e.g. buying). After time, the software would declare a winner and you could measure that page B drove 20% more sales than page A (for example).

I love this quote "A man who goes into a hardware store to buy a quarter-inch drill bit does not need a quarter-inch drill bit -- he needs a quarter-inch hole"

What does your customer need? A/B split testing can give you those answers...

05-24-2013, 11:15 AM
has anyone tried the Google method? I see you need to paste additional code onto the original version of the page only.... just wondering how the did this. ( they ask for the code to be in the head tag, which would put it on all pages via template..) guessing some Java is needed so it only triggers on the home page ( if a/b testing home page)

06-07-2013, 02:20 AM
I just saw this and figured it was worth adding to the thread:


In essence it's a method of triggering a different bit of jQuery for each variation rather than redirecting to a separate page, so if you can script it you can test it.

If I'm reading it right it would let us do sitewide tests, I'll have a play with it and see what can be done.

06-07-2013, 09:01 AM
Just set up an experiment for a client using the method above, it works really well and was much easier to set up than I expected.
We're testing whether or not to have a massive pop up on the homepage.

With the old Website Optimiser you had to define a goal url, with this method you can still do that, but you can also use it to test the impact of changes on metrics like time on site, pages per visit, or in the case of the homepage pop up, we're testing the effect on bounce rate.

03-23-2014, 08:54 PM
I know this is old, but I having been thinking of radically changing one Product Category and most of it's Subcategories. Looking at both of the tools (VWO and Optimizely) I am not sure if that would be possible for not. It is going to affect so many pages that I am not sure if either would do it or not and if so, would I be able to set it up myself? (Without signing up at least)

Based on my GA this category should be doing better than it does but I don't know if it is set up in the best way possible. I would need to change the main category page and eight subcategories with all their products. After reading so much about not changing without testing I feel like to need to test it against my present settings. I have also had some comments about not being sure where to find some items and that is also a reason I want to test a new organization.

I do not however want to make it worse! Are there other AB testing tools too to look at now?

03-25-2014, 02:20 PM
In Visual Website Optimizer you can set up tests, so they affect multiple pages through a wild card, since category pages have a certain structure. www.yoururl.com/*_s* to run a test that impacts all your category pages for example.

For testing a new category structure, there is no great way to do it unfortunately. You can set up a dummy htlm page that has your new structure in it, and replace a section on a category page with it. This way you can compare how the old version did compared to the new one in an A/B Test.

The trick is you have to define what is success.

A key is have a great hypothesis, before you start testing.

03-25-2014, 07:29 PM
I was thinking I could make a new hidden category and subcategory pages and then then having the same products in both just arranged under different subcategories. I was wondering if I could direct half of my traffic to the hidden page and half to the live page? Or do you have to set up the test page on the A/B testing site?

03-26-2014, 09:29 AM
Yes, you can also do it this way.

In VWO, on the landing page, you set up an A/B Test with version A no change, and version B linking to the hidden sub-categories.

03-26-2014, 08:45 PM
Thanks ritchey, I will have to work on setting up my page and sign up for a trial to run an A/B test!