Are You Guessing - Or Testing?
by Robert F.
Abbott
Having trouble deciding which slogan to use on your Web page? Or what
to call your newsletter? Or what to charge?
Well, you're on the Web now, and that means you can do quick, easy, and
inexpensive testing to get answers. I’ve done it a number of times and it’
s been a useful experience each time. Here are three examples: Change and
Measure, A Banner Example, and Online Questionnaire.
Change and Measure
A key objective for one of my websites is to get visitors to subscribe to
my newsletter, Abbott’s Communication Letter. When I started working to
improve the ratio, the conversion rate was about 9%; that is, about one in
11 visitors subscribed.
In a bid to improve that ratio, I added a strong ‘Because’ section at
the top of the index page, explicitly listing the benefits of subscribing.
Again, I measured responses, getting several hundred of them before
drawing any conclusions. And what did I find? Adding the ‘Because’ section
improved the ratio, from 9% to 15%; now one in 6.6 visitors were
subscribing.
To do this kind of testing, vary just one detail at a time, measure the
results (with between 100 and 400 responses), and if the change worked,
make it permanent and start over to test another change.
A Banner Example
Here’s another challenge that seemed natural for testing: Which of two
slogans should I use on the front cover of my book, A Manager’s Guide
to Newsletters: Communicating for Results?
Solution: I created banner ads for each slogan, and tested them with
the help of Mike Scanlin of Four Corners Effective Banners. Each banner
got 3,000 impressions and when the test was done I had my answers.
To test with banners, start with a control banner and change just one
thing on a second banner. Then review the click-through statistics to find
out which works best. Of course, you can then take the best one and start
testing it.
Online Questionnaire
Here’s another type of testing: an online pricing questionnaire. I wanted
to sell more copies of the electronic version of my book and wondered what
the optimal price might be.
At the same time, Internet entrepreneur Ken Evoy developed a way to
test prices using the power of the Internet. His online surveys use the
Internet to 'gather, analyze and deliver the data' at
http://myps.sitesell.com/acl.html .
In this test, visitors to my website were given information about the
book (but not the price) and asked to complete a quick, six-question
survey. In exchange, they got a free report and a big discount if they
bought the book.
When I had received enough responses to the survey, I was able to see
clearly the optimum price and the effects that different prices would
likely have on sales and profits.
How to Test
If you try to test a lot of concepts at once it gets complicated. Believe
me, I’ve tried. So change just one detail at a time, measure the results,
compare with the original, keep or reject the change, and try another.
How many responses should you get? Try for 400 responses if you want to
assess the opinions of more than 10,000 people, even millions of people.
Does 400 seem too few to measure what millions of people might do? Well,
assuming your test is reasonably well designed and you’re getting a fairly
representative sample of the population, 400 provides a commonly accepted
level of accuracy. For example, political pollsters can accurately
forecast the voting intentions of literally millions of people after
talking to just 400 of them.
Can you get by with fewer than 400 responses? Sure. You won’t have as
much confidence in the results you get, and that’s okay if the stakes
aren’t high or if you’re surveying a smaller population.
The important thing is that you test, not guess, because testing is the
foundation of a successful initiative. And, using the power of the
Internet makes it quicker, easier, and cheaper than ever before to test.
That’s a deal you don't want to pass up.
Robert F. Abbott, the author of A Manager's Guide to
Newsletters: Communicating for Results, writes and publishes a free online
newsletter with communication ideas for leaders and managers
www.abbottletter.com.
|