A few years ago, we developed a test campaign for a company offering a catalog of outdoor furniture. The test included some publications in the garden market, others in the home market. Two of those in the home market had very similar demographics and editorial content. We’ll call them Magazine A and Magazine B.
Normally, we wouldn’t have included two such similar publications in an initial test, but the client was in a hurry to learn as much as possible as fast as possible.
Every single publication tested was profitable except one, Magazine A. What was surprising was that the very similar Magazine B outperformed every other publication tested by a wide, wide margin. In the long run, the home market was infinitely more profitable than the garden market.
Every time a client or prospect wants to base all future decisions on a single “test,” of any medium, I repeat this story. What if we had only tested Magazine A? Many companies I know would have concluded that the home market didn’t work, or worse, that magazine advertising just wasn’t profitable.
Testing just one publication – or list, or card deck, or insert – isn’t a test. It’s a crap shoot.
For one, the problem isn’t always the media. Your message may not be clear. The offer may not be compelling. The price may be too high. It may not be the best season. (Remember too, the best season for generating leads and for generating sales may be different.) And sometimes results just vary for reasons you’ll never figure out.
More often than not, when clients or prospects tell me “We tried it once, it doesn’t work,” digging deeper tells a different story. Often the copy, design and/or offer are so unclear or uncompelling that poor results are no surprise. Other times, we learn that “tried it once” means just that – a test so small it wasn’t a test, but a crap shoot.
So before you give up on any “test,” take another look. Did you really give it a fair shake? What elements could you change that might make a difference? Do a little brainstorming, and you may discover that you threw out the proverbial baby with the bathwater.
