Recently, I received a tough question from a well known web analyst:

“I was trying to think through how to measure the impact of Affiliate advertising. Do you know of a way to measure if there is incremental lift due to advertising with an affiliate?”

Huh? Don’t you just check the reports in the affiliate marketing network, say LinkShare or Commission Junction? Or isn’t that a simple referral report in your web analytics?

“We have a pesky problem of having lots of customers that would have found their way to our site without the additional advertising.”

Oh boy!

Wholly, cow … that is a tough question! What is the credit that the affiliates truly deserve, i.e. the portion of sales through Affiliates where customers would - not - have purchased the product directly from the manufacturer’s site anyway?

In the offline world an equivalent question exists: What is the credit that my resellers or distribution network deserve, i.e. the portion of their sales where customers would - not - have come to the manufacturer’s own stores or call-center anyway?

So how could either question be researched? In these economic times you want to be sure not to be rewarding marketing channels for business that you would have received anyway.

Is a controlled experiment with Affiliates / Distributors possible?

If it was we could create a control group and compare lift in control group vs. test group. Alas, one can't just turn the relationships with affiliates or distributors on and off to create an experiment. Nor can the reach of online affiliates be restricted geographically, i.e. the search engines reveal them all regardless of where they are located in the country or world.

We have to look to either uncontrolled testing and/or panels, to find a solution

Comscore (for online panels) and Nielsen (for online or offline panels) suggest that the following type of analysis would be possible. Namely, one could split their panel population in two buckets:

1. Used major affiliate's sites (or distributors’ stores)
2. Didn't use major affiliate's web site. (or distributors’ stores)

Then we could calculate the percentage of each bucket that ends us purchasing the manufacturer’s product. How much more likely is the group that used affiliate sites (or distributors’ stores) to purchase our product? If 20% of group 1 buy our product and only 5% of group 2 do, shouldn’t we credit the difference to the true lift from affiliates / distributors?

Watch for bias!

There is bias in these groups that we need to correct for. The people who visited an affiliate's web site may have done so because they were already interested in making a purchase of our product. They did a search for affiliates providing the product (e.g. on comparison shopping engines) precisely to get a better deal.

In contrast, group 2 contains people who may or may not be in the market for the manufacturer’s product category.

So in order to correct for this we would have to do more. We'd need to first filter the panel to the subset that makes any purchase in the related product category at all, regardless of whether that is this manufacturer’s product or one of its competitors. Then among this group we’d apply the analysis stated above.

That should get us a little closer to the truth, I believe.

Crazy, how much thought and effort it takes though!

Multichannel metrics (& web analytics) are both easy and hard. This one is an example of how they are rather hard and how they require much more than just (web) analytics software.

P.S.: Some manufacturers will have it easier with this type of analysis, namely those where everybody in the population needs their product. Say, shampoo or groceries or tax help.

P.P.S: In the offline world there is the benefit of geographic testing. So the offline marketer could compare purchase behavior in regions where there are not any stores by the manufacturer. Or they can also look at behavior based on the driving distance of individuals to the closest store.