Initial Test Results
Recently, I started running a few tests on my blog. Some initial results are in, so I want to share them with my readers.
I’ve been testing out my opt-in form. (It’s the one up there in the top right corner of this website.) Originally, I was running a simple split test using Aweber. However, as you might know, split testing is a slow and tedious process. Also, Aweber only allows you to split test while using their javascript code which makes it difficult to customize other parts of the opt-in form. There are several reasons why I don’t like split testing, but I won’t go into all of them.
Originally, my opt-in form was rather basic. It had a description, name field, e-mail field, and a submit button. But, since I wasn’t allowed to customize more specific variables in Aweber, like the submit button color for example, I had to make some changes.
So, I decided to run a multi-variable test using MuVar 2007. I use that software to test my sales pages, so why not test an opt-in form?
I decided to add some variables… here they are:
Area (the area around the form)
Headline
Description
Name
Email
Button (submit button)
Privacy (the privacy statement below the form)
I first started running a test over a month ago, but later realized I set it up wrong which skewed the data. So… I started over and here’s what has been working better so far for me (more subscribers):
Variable: Headline
I tested a few headlines for the opt-in form. One version displayed “Subscribe Now!”, another displayed “Subscribe!” and yet another displayed nothing at all. What worked best out of those three versions?
NOTHING!
That’s right, showing no headline at all for my opt-in form gave me the best conversion.
Version: 1 - Subscribe Now!
Conversion Ratio: 0%
Version: 2 - Nothing
Conversion Ratio: 3.03%
Version: 3 - Subscribe!
Conversion Ratio: 0.69%
Variable: Description
The description variable had three versions:
- Subscribe to my blog update list to be notified via e-mail when I post a new blog entry:
- Nothing
- Insert your first name and e-mail to receive an e-mail each time I submit a new entry!
Here are the results:
Version: 1
Conversion Ratio: 0%
Version: 2 - Nothing
Conversion Ratio: 1.78%
Version: 3
Conversion Ratio: 2.14%
Hmm…the second version is interesting. I wouldn’t have expected someone to sign up with no description. Of course, if someone read a previous post which gave reason to enter their name and e-mail address into the form, that could have triggered it. Or, maybe someone was just curious?
Variable: Name
Two versions tested here (both were Arial, Size 1, Bold):
Version: 1 - Enter Your First Name:
Conversion Ratio: 1.33%
Version: 2 - First Name:
Conversion Ratio: 2.09%
Variable: E-mail
Version: 1 - Primary E-mail Address:
Conversion Ratio: 1.77%
Version: 2 - E-mail:
Conversion Ratio: 1.24%
Variable: Submit Button
This was interesting…
Version: 1 - Button: “Subscribe Now” Border: 5px edit
Conversion Ratio: 0%
Version: 2 - Red Border
Conversion Ratio: 2.78%
Version: 3 - Blue Border
Conversion Ratio: 0%
Version: 4 - Grey Border
Conversion Ratio: 3.33%
Version: 5 - Dark Red Border
Conversion Ratio: 0%
Version: 6 - Dark Blue Border
Conversion Ratio: 0%
Why did only a red border, and a grey border work best!? Well, we will never know the answer for sure, but I have a few guesses. My original subscribe button before doing any testing was a button with a grey border. Therefore, maybe more people signed up because they were more familiar seeing it more often? Or maybe because grey just fits in better with the theme of the website? As for red… it stands out! Some people have said that it acts like a big red stop sign. It really does stand out, so it makes sense to me.
As for the text on the submit button, I didn’t have any other versions. All of the above versions display “Subscribe Now”.
Variable: Privacy
Should you show a privacy statement? Yes? No? Apparently it doesn’t matter much (at least at this point in the test):
Version: 1 - I respect your trust and privacy. I will NEVER share your email address with anyone at any time.
Conversion Ratio: 1.58%
Version: 2 - Nothing
Conversion Ratio: 1.69%
 - Aaron Brandon
I had just been considering how embedding a muvar based test in another page might work.
Good to see the iframe idea in production somewhere.
Thanks for sharing the results. Very interesting!
Comment by Wayne — November 29, 2007 at 8:40 pm
Hi Aaron,
I understand the workings of MuVar, but I’m not sure how you used it on your blog to test the subscription form.
Here’s an idea for you, first dibs on a copy if you make it.
How about an in depth tutorial on all the different ways you can use MuVar in different situations, like this blog and subscription form for example.
I know there have been times when I have tried to use MuVar and hit some snags, so a bit more info on top of what JB offers with the program would be great.
Thanks Aaron, you are an inspiration.
Ian
Comment by Ian — November 29, 2007 at 9:07 pm
I noticed that your results match Glyphius on everything but “nothing.”
I wish Glyphius could measure nothing better.
Most split-testing results can be predicted by Glyphius… in my experience.
Comment by Chuck — November 29, 2007 at 9:26 pm
Hats off to you Aaron!
MuVar on the opt-in form. Woooweee. That’s cool. I didn’t even know you could do this.
There surely is a small product in showing others how to do this. Or at least a video for your blog. This is a big answer to exactly the problem of aweber’s split-test.
I just hope I can get this working on my blog.
Comment by Martin Russell — November 29, 2007 at 10:31 pm
This is why you are a success, and why you will continue to grow. You are one of the few that not only talk about testing, you actually test! Furthermore, you analyze the results scientifically. Good going.
Comment by James Alenteal — November 29, 2007 at 11:12 pm
Hi Aaron.
Have you tried to test the ultimate submit button? This was proposed by “conversion doctor” Eric Graham.
http://www.conversiondoctor.com/conversion-blog/2007/the-ultimate-submit-button-revealed-putting-all-the-pieces-together/
In all my tests, especially at opt-in pages generated a substantial improvement in my rates.
Just my 2 cents.
Francisco Moriones
Comment by Marketing Reviews — November 30, 2007 at 2:24 am
Thanks for sharing your results…and the muvar idea. I have been considering “muvaring” my whole blog but I think I’ll start small and just muvar the opt-in box first. Thanks for the idea…just in time for a new month of testing. There’s always a solution to challenges and this is the solution to the limitations of aweber.
Comment by Robert — November 30, 2007 at 2:24 am
Conversion Testing, Joint Ventures, & Ethics….
Hi there, good to meet you! If you’re new here, you may want to subscribe to my RSS feed over on the right-hand column. Thanks for visiting, and I hope my information helps you!I like to point out things that I think are helpful, and I found two tod…
Trackback by James Alenteal's Conversion Rate Clinic — November 30, 2007 at 6:35 am
[...] Aaron Brandon Aaron posted some interesting preliminary data on a test he's been doing on split testing an optin form. This is actually something I'm planning on doing myself once I get my copy of MuVar [...]
Pingback by End of Week Roundup (11-30-07) — November 30, 2007 at 12:07 pm
Francisco, I did add the “Ultimate Submit Button” as a new variable version in MuVar immediately after I watched Eric’s recent video:
http://www.ultimateconversionbooster.com/aaron
It didn’t make make it into my initial test results because I added it so recently. Maybe I’ll share the results with everyone once there’s enough data.
By the way, Eric’s taking down that free video really soon! He’ll be turning it into a product… So, if you haven’t seen it yet, jump on it…and more importantly - take action. When it comes to conversion, Eric is the man!
- Aaron
Comment by Aaron Brandon — November 30, 2007 at 2:51 pm
maybe this is a stupid question but i was under the impression that muvar could only be used on an html ONLY page. i have not bothered to use it anywhere simply because i dont have any sites that are just html.
an email would be very cool, aaron, if you get a second to tell me how wrong i am!
Comment by joel — December 24, 2007 at 10:01 am