To Test or to Target? Where to Start for Best ROI?

The previous post had concrete recommendations for proving the ROI of behavioral targeting. Several smart reader comments brought together a pretty clear picture.

However, when I was meeting with a number of experienced online bankers in Europe recently, the question that I received was more difficult to answer than just proving the ROI of targeting.

Namely, the question was whether one can expect greater ROI from testing or targeting? Whichever promises greater ROI, shouldn’t that be where you may want to start?

Thinking about it in terms of anecdotal results

Testing tends to be much more black and white in outcomes when you first begin to optimize a website. After all, no perfect website falls from the sky. At first you can get a big impact from making changes to messaging, calls-to-acton, layout, etc.

There are case studies, e.g. from Unica web analytics customers, where website conversions were improved by as much as 1900% over time with the help of continuous testing, measurement, and improvement.

Now, I have seen many great case studies of behavioral targeting online. But I haven’t seen one where a better targeted message (via email or website or ad network) drove a 1900% increase in conversions.

Should you count on a 1900% improvement from your testing efforts?

Most web marketers when they hear these great testing case studies can be very disappointed when their own testing efforts only yield a percentage point or two in improvment. That happens a lot too.

And depending on the size of your web business, a 1 percent improvement from testing can still be a highly profitable outcome.

But, it goes to show that a mere comparison of anecdotal results that other people achieved from testing vs. targeting is not going to be enough to help you decide whether you should start with one or the other.

A better way to think about it

Testing is a key competency for targeting. As the previous post suggested, you would probably have a hard time proving the value of targeting if your organization didn’t know how to do (hold-out) testing.

Likewise, reporting is a key competency for testing. If you didn’t know how to measure and report on key metrics, how could you prove which test candidate truly drove better results?

Growth path

That is why it does make sense to follow a maturity model where the organization grows from reporting to site optimization (with testing) and then targeting similar to the diagram below from one of Unica’s whitepapers.

Just don’t get stuck in the stage of reporting and testing but make sure you continue to climb up the maturity curve when the business value that you are generating in Stage 2 begins to level off.

Behavioral analysis and targeting (for interactive marketing) has helped companies who do it well push up business value beyond the plateau that they reached from testing alone (see previous post).