The Reports of CRM Failure Are Highly Exaggerated: An Interview With Gartner's Ed Thompson

"CRM has a high failure rate." That's a given with almost everyone you talk to in the industry. When you press people about that assumption, they quote Gartner. The research firm famously assessed CRM failures of 50 percent to 70 percent, depending on whom you talk to. But try to get your hands on an actual report, and you draw a blank. So we rang up Gartner to set the record straight.

Bob Thompson, founder of CRMGuru.com, interviewed Ed Thompson, co-author of Gartner's Eight Building Blocks for CRM Success and the lead for the customer experience management and SAP-CRM for the market research firm, Oct. 18, 2004. Ed Thompson is based in Gartner's U.K. office. The following transcript was edited for clarity and length.

Gartner's approach to CRM
What is CRM?
The origin of the quote
The truth behind failure
The state of CRM today
Actual success rate
The problem? Metrics

Gartner's approach to CRM

Bob Thompson
Tell us a little bit about how Gartner covers CRM analysis. I've seen different names quoted with Gartner in CRM. Who does what?

Ed Thompson
There are about 15 or so analysts in the CRM team, the core team in Europe and the U.S. There are another two or three as part of that team worldwide, so there are 18 or 19 analysts worldwide in Gartner covering the topic of CRM, four of us in Europe, one in Australia, one in Singapore, one in Latin America and the rest in the U.S. And, all the ones you've quoted are U.S.-based analysts, which is not too surprising; that's the biggest part of the team.

There are probably another 10 to 15 other analysts across Gartner who have touched on topics related to CRM, but they work in particular industry sectors, like, healthcare, government financial services and insurance. They're writing about CRM, as well, from their perspective.

Bob Thompson
Do you have a specific responsibility within that team?

Ed Thompson
I've got two or three things I cover. One is what Gartner calls the Eight Building Blocks of CRM, which is the model that Gartner put together in 2001 to sum up all the elements of a CRM strategy. I was one of the co-authors of that. And, I have another topic, customer experience management, which I lead on at the moment. I also lead on coverage of SAP CRM worldwide.

Bob Thompson
All right, quick question: What is your official definition of CRM?

What is CRM?

Ed Thompson
About 1998 Gartner sat down and wrote a big definition, which starts with the word, "CRM," and defines it specifically as a business strategy, and we've stuck with that definition now for about—well, ever since '98. But, since 2002, our message has been pretty straightforward, which is: Ignore our definition of CRM, and in fact, ignore everybody else's definition of CRM and come up with your own definition. It sounds a bit of a cop-out, but what we've learned from this is that if you sat five or six people down in the same organization and asked them what the definition of CRM was, they'd all come up with different ones.

In fact, something we're doing right this moment is questioning not the definition of CRM but whether the term "CRM," itself, should still be used. And, we're wondering whether it's finally coming to an end as a useful acronym. Gartner had TERM up till about '99. And, since then, we've stuck with the industry phrase of CRM.

Bob Thompson
TERM, meaning—

Ed Thompson
Technology-Enabled Relationship Management. We ditched that in 1999. That clearly wasn't working for the industry. What we generally believe internally at the moment, is that CRM as an acronym is mostly associated with technology, no matter what we try to do to claim it's another strategy. And, there's no way we're going to change the world view of that.

Bob Thompson
Right, but you still fundamentally position it—Gartner positions it—as a business strategy.

Ed Thompson
Yes, we do.

Bob Thompson
But, you seem to be questioning whether your clients, in fact, do that. Is that correct?

Ed Thompson
We think they don't. We think they associate it as a bunch of technologies which support something.

Bob Thompson
So, it's kind of muddied out there in the real world.

Ed Thompson
Exactly. So, the problem is, basically, that whatever we do, CRM is going to be associated with the technology. That's partly because of the marketing clout of the technology vendors. We fundamentally believe that there is such a thing as a customer-centric strategy of some sort, which organizations have pursued. They just don't call it CRM.

Bob Thompson
I don't know if you've done any formal research, but what's your opinion? If you ask 100 business people at random for their definition of CRM, and—regardless of what they said—how many of them, would you feel, would give you an answer that would basically say they thought it was a business strategy vs. those who would say they thought it was some form of technology programming?

Ed Thompson
I think most. If you asked the question loosely, to business people, rather than IT people, then they would probably answer both, to hedge their bets, in the majority of cases. But, I think if you forced them to pick one of the two, you'd probably get three-quarters saying something to do with technology, rather than a business strategy.

Bob Thompson
On this kind of a sandy base, how do you build a definition of success, when there seems to be such confusion about what CRM is, to begin with?

Ed Thompson
Well, I think the people that are confused are analysts, journalists, vendors and, perhaps, consultants. I don't think the organizations that are involved are that confused about what they're trying to do. I've called it the "flag-of-convenience" problem, in that they have a name for the program, the project, the initiative, the "whatever they call it," internally. And, the term "CRM" lost its shine in about 2001, so they renamed it in many cases, or they shut it down or repackaged it or refocused it, whatever.

But, the reason I call it a flag-of-convenience problem is that, to talk to the outside world—and by that, I mean journalists, consultants, vendors—the only term that's common is "CRM."

Bob Thompson
So, you think it has kind of a customer angle to it—marketing, sales or service?

Ed Thompson
Yeah.

Bob Thompson
Anything that kind of smells like that, the CRM label gets—

Ed Thompson
Exactly, so what organizations do is, they put up the flag—the flag of convenience, like in a ship—and put up the CRM flag and invite people to talk to them. And, as soon as you get there, to the flag, they pull it down and say, "Right. We don't call it 'CRM' around here; we call it this. Now, I've described the kind of problem we're trying to solve, can you help?" So, I think that's the way it's working right now. It's used by internal organizations but merely as a mechanism, in most cases, for talking to the outside world.

About one in six times, you'll find someone whose company has been majorly successful with CRM. And they have named their initiative CRM, and they're very proud of it. And they have people with job titles with "CRM" in them. And, they find that very perplexing because they've been extremely successful. So, they're very proud of the term "CRM." So, what you get is sort of one in six who are very proud of it and five out of six who skulk away from the term "CRM."

Bob Thompson
And why are they skulking?

Ed Thompson
Because it has lost its shine, because they viewed it —or the press viewed it or the analysts viewed it—as not meeting expectations, being a failure of some sort back in 2001. And, there were a number of reports within their organizations about problems around what was CRM, and the projects didn't succeed in delivering. That means that they don't like the term "CRM," anymore.

Bob Thompson
We did a little bit of a search. It seemed like 2000, 2001—maybe 2002—is when most of the reports—or reports about reports—came out. I've heard or seen a reference to a 50 percent to 80 percent failure rate many, many times. It's bloody difficult to figure out where that came from, but I have to say that the source seems to lead back to Gartner, Meta, maybe a couple of other firms. You guys are quoted fairly often as being the source of some of these failure rate statistics.

Ed Thompson
Exactly right.

The origin of the quote

Bob Thompson
So, I wonder if you could comment on how you came up with these numbers—or did you? And during what time frame?

Ed Thompson
Yeah, we did a study back in the back end of 2001 in the U.S. and Europe—not a massive study. But I think there were about 500-plus organizations. We were asking questions about e-commerce and on CRM. We were asking, Did it meet expectations? That was the question. And, the answer we picked up fairly consistently, was that a lot of organizations failed to meet expectations. And, I think the figure we originally quoted was 55 percent failed to meet expectations.

At about the same time, though, there was a study done by one of the big CRM consultancies. I think they kept it very close to their chest because it didn't reveal the results they wanted it to. But that was quoted by the press around the same time, and that had a figure somewhere around the same, around 55 percent or 60 percent. But they used the word, "failure," in their research summary, and what we found when we were quoted was that people took our statement: "failed to meet expectations," and they chopped the "meet expectations" off it and just said, "failure." So, we were quoted by the press as saying 55 percent of projects are failing.

At the same time, we put out a Gartner Strategic Planning Assumption—this was late 2001, early 2002—saying the "failure to meet expectations" was going to get worse. At the time, we said from every indication we're seeing, we think this "failure to meet expectations" rate will rise to 65 percent. Instead of seeing just over half of the projects failing to meet expectations, we think we'll move to two-thirds failing to meet expectations.

That was based on the fact that we'd seen a lot of organizations—big organizations—buying large quantities of software without any real justification for it in 2000 and into the first half of 2001. We could see it all kind of come crashing down from late 2001 onwards. And so we said that we think it's going to get worse. But, we never went back and did the same study to see if the "failure to meet expectations" rate got worse.

Although we've never actually gone back and measured it, we're pretty confident it did get worse, from all the other studies we've seen people doing, anecdotal comments and the qualitative research we've been doing.

Bob Thompson
Going back to these 500-plus organizations, were these selected at random; or were these big clients or your clients?

Ed Thompson
They were Gartner clients. They were large organizations. They tend to have larger projects, and everything we've seen since shows that the smaller projects do much better. The mid-size organizations haven't had as much of a problem.

Bob Thompson
So, you would at least speculate that if you looked at a broader sample with a mix of projects that this "failed to meet expectations" number would be a little bit lower.

Ed Thompson
Yeah, I think so. It's interesting, something we picked up last year in a totally different study we were doing with reference checks was from answers to two questions: whether the CRM initiative had been a success and whether they thought they were mature in terms of their approach to CRM. Mid-size organizations not only thought they were more mature on average, but they also said they had a higher success rate.

Bob Thompson
How high?

Ed Thompson
I can't remember exactly how we asked the question, so I'll have to go and have to dig it up and find out.

Bob Thompson
Isn't that part of the trick here, though, going back to your phrase, "did not meet expectations"? Was it literally as simple as that? Yes or no? You either met expectations or you didn't. Black and white.

Ed Thompson
No, it was a scale of questions. The top end of it was "absolute success," with improved ROI-type number. The next one down was something like, "met expectations and somewhat of a success." It didn't necessarily mean you had quantified it with a full-blown ROI, but it had met your expectations of being successful. The next one down was, "it failed to meet expectations but had been somewhat of a success." And then, the next one down was, "failed to meet expectations, was somewhat unsuccessful." And then down into "absolute failure."

So, with that kind of scale, a big chunk of them were in the middle—in this "failed to meet expectations but had been somewhat of a success." In other words, what they were talking about was payback. If you look at the verbatim quotes in the same study, what you're getting is, "We aimed to get a payback in 24 months, and we didn't, but we have increased sales by XYZ."

Bob Thompson
Was it only the top box that would count as a "met expectations"?

Ed Thompson
No. The top two boxes counted as different forms of success—the first an absolute success, and the second was less positive, something like, "met expectations and somewhat of a success." The majority were ticking the middle box which was:: "failed to meet expectations but was somewhat successful."

Bob Thompson
So only the top box counted?

Ed Thompson
Only the top box counted as absolute success.

Bob Thompson
Let me make sure I'm understanding you. Forty-five percent checked the top box? And 55 percent, the other four?

Ed Thompson
No. Forty-five percent checked the top two boxes and 55 percent, the other three. The bottom category was absolute failure, and that was only something like 5 percent.

The next lowest box was "failed to meet expectations but was somewhat unsuccessful," which is kind of a bit vague, I admit. But, that's where we had only 10 percent. The next box was "somewhat successful but failed to meet expectations," which was around 40 percent. Then the next box up was "met expectations and somewhat a success," which was also around 40 percent. And, we ended up with, as you would suspect, 5 percent in the "absolute success."

Bob Thompson
Would the top two boxes count? I'm trying to figure out: If 55 percent failed to meet expectations, then 45 percent met expectations in some way, right? At any rate, do you agree with the way that the media has tended to characterize this, that 55 percent of CRM projects have failed? Basically, that's the failure rate?

The truth behind failure

Ed Thompson
No, I don't, because it's not absolute failure. And if you're looking at absolute failure, you're looking at 5 percent, maybe. Perhaps if you include the "failed to meet expectations and somewhat unsuccessful" category that might add another 10 percent. In an IBM study from earlier this year, you pick up absolute failure rates of, again, 5 percent, 10 percent—where by absolute failure, I mean: did not deliver financial return; ended up being scrapped; complete disaster. And, that's pretty rare.

Likewise, at the other end, which is absolute success. That's where you tend to get most of the exciting case studies in the world of CRM, where they set out some aggressive targets and delivered on them, and everything is viewed as a major success. People have made their careers on the back of it. Now, that's a bit better than it was back in 2001 and 2002. Right now, we would put that at 15 percent, 10 percent to 15 percent in that "absolute success" category.

It was more like 5 percent when we did that back in 2001 and 2002. So, it's improved, but then, we're more cunning about how we ask these questions. Today we put in a box that says, "Too early to tell." As soon as you put that in, that's where people will actually tick that box.

Bob Thompson
That's a good bail-out question.

Ed Thompson
We were only allowing respondents to tick boxes saying "somewhat successful" and "somewhat unsuccessful." But we found that if you give them the option to say, "We don't know yet," they'll actually tick that box. And, the reason for that is, they often will say: "We need two to four years to tell whether this thing's actually paid off and been a success, and we're only 18 months into this. So we don't really want to tick any box that makes a judgment about success or failure yet." When we did this back in 2001, we didn't give them an option to tick a box saying, "We don't know yet; we haven't had enough time to work on this."

I suspect if you went back and asked them to tick a box saying, "Too early to tell," a lot of people would jump into that box.

Bob Thompson
You mentioned that you said you thought, through anecdotal evidence, that failure rates and projects that "failed to meet expectation" had increased for a time because of all of the large deployments and unused software. What do you think has happened the last couple of years? You haven't done any formal research, I take it?

Ed Thompson
We've been doing little surveys on and off, not big ones like that 500-plus thing. We've done a few like that.

Bob Thompson
So what has this research told you?

The state of CRM today

Ed Thompson
The first thing is objectives change over time. If you look at the objectives circa 2001-2002, when we were doing that study, they were about improving loyalty, improving retention and revenue growth. During 2002 and 2003—in particular, the second half of 2001, 2002, pretty much all of 2003—if you look at the objectives, what came to the top was reducing cost of sales, reducing cost of service. Retention is still there, but loyalty dropped right off, right down the list of objectives. Customer satisfaction dropped right down the list of objectives.

As economic conditions got worse, people became very short-term in their objectives. So, it meant that there was a lot of focus on ROI and payback periods and TCO. And, what it meant was, the projects that were getting authorized were smaller, with very tightly bound financial requirements, in many cases. And, interestingly, very few organizations go back and measure afterwards, but they had to put a tighter business case together to get them through, which meant you're looking at payback periods of some 12 months—in many cases, six to 12 months being much more common—and projects which would only take three months, four months, to implement, projects which could only be justified, really, in terms of hard ROI, based on cost savings.

Now, it's changed, we think, since about this time last year—about the end of 2003. Most of this year, we've seen things warming up a bit, and what we mean by that is a focus on longer payback periods. We're seeing organizations being willing to trust project managers with longer payback periods. Not massively longer, but we're looking at 12 to 18 months, not six to 12. They've got a bit more freedom. And if you look at the objectives, No. 1 would be improving customer satisfaction. There's a lot of interest in this customer experience topic, as a result of that.

One of the top three will be increasing revenue or acquiring new customers. Customer retention's still up there in the top four or five. All the cost-reduction, reducing cost of sales, service, operations cost, etc., cost of marketing—all of that stuff is dropped right down the list and doesn't even make it in the top 10 now, in terms of objectives.

Bob Thompson
Interesting. So these objectives change over time as the business climate changes?

Ed Thompson
Exactly. It correlates directly to GDP growth, actually—gross domestic product growth—because, for example, in Germany right now, where the economy's still pretty bad, they're still being justified in cost-saving terms.

Bob Thompson
So, it's returned more to the strategic kind of objectives: what CRM was about and more of the hype change.

Ed Thompson
In the first place, yes. I was speaking at the Siebel User Week. I showed the objectives from 2000 and from the last year or so, how they've slowly been changing over the last year and particularly surveys from November 2003 and April 2004. We have been asking a question about objectives for each of our Gartner CRM Summits, and we're doing them twice a year and sometimes three times a year in different geographies. So, I was able to ask this question fairly consistently. I've been able to see that swing back.

We didn't measure it frequently enough going into the downturn to be able to spot it, but we could see slap-bang in the middle of the 2002 timeframe what the objectives were then. We could see how they'd now switched around and come back the other way. And, it's interesting that if you look at retention, loyalty and customer satisfaction, they all are in the top four or five objectives now.

The next bunch, all about revenue acquisition cross-sell, up-sell, increasing profit per customer—those type of metrics—they're all secondary. And now, all the cost saving ones have all dropped down. What I was pointing out to the audience was two things: One, this change in objective means that you can't sort of, say, set the objectives for CRM and then go for it for five years.

You've got to be aware that the board's switching its mind around, and you look at the big IBM CEO survey they did last November-December timeframe that was pointing out that 80 percent of chief executives had switched their focus more to revenue growth. But, I was throwing up a hand and saying, "Beware," because although the chief exec's changed his mind, maybe the operations director or finance director hasn't changed his mind, yet. So the money isn't flowing through, in terms of IT funding, to the IT organization to back CRM technologies, yet. And the only technologies that are getting approved are still tending to be cost-savings-based ones.

And, what it really boils down to is trust. An example I gave was the sales director of one of the mobile phone manufacturers made a comment that he could create any business case you want with a spreadsheet and two nights of hard work. He said, "I've made more business cases than you've had hot dinners, so I can prove anything you like with numbers. But, what it really boils down to is: Do I trust the sales team manager, who's come to me and asked for some money for technology?"

Bob Thompson
That's right.

Ed Thompson
If I trust they can grow revenue by 3 percent, I'll give them the funding. Times are tough, and I need to put in technology to save money; I want a hard number. But, when things are on the up, it really comes about trust and taking risks. And, he said at the moment that it has not quite opened up in his company, but it's starting to open up. They're starting to be willing to gamble, shall we say, with investment funds where there is risk associated. It's not about investing if you're just saving money.

Bob Thompson
Right, so things are continuing to shift around, but if I could try to pin you down here for just a second, Ed.

Ed Thompson
Sure.

Bob Thompson
Based on your anecdotal evidence and research and all of the experience that you've had in this industry in the last, let's say, year to 18 months, what do you think the success rate of CRM is?

Actual success rate

Ed Thompson
I'm going to answer that by breaking it down a bit. I think the success rate is extremely high in call centers. I think it's still fairly low in field sales projects. I think it's extremely high in sales like e-commerce.

Bob Thompson
But, when you say it's extremely, high, you're talking about, what? Eighty percent? Ninety percent?

Ed Thompson
I'm talking about 65 percent, 70 percent having a success of some sort. In other words, again, it's this issue of: Did it meet what they originally hoped for?

If we can put a number on it and say, "We made $5 million off this." I think a lot of times, people don't. One of the pieces of advice that we're giving people is to go back and measure what they've done, because they might be pleasantly surprised by their success rates and being able to get additional funding for more projects.

In other words, what we think is the CRM market is depressed in some ways by the fact that people haven't bothered to go back and measure. And, they've actually been quite successful, but the perception is they haven't been successful. They had high hopes back in 2000 and2001, they then did something, it worked, it was OK. But then, they've kind of put it on the back burner, and they haven't gone back to revisit. If they had actually gone and checked it out, they would find it had been quite successful—very successful—possibly over three or four years.

Bob Thompson
Field sales is still fairly low? Why is that?

Ed Thompson
Field sales is the most tricky, and I think the real key factor here, of course, is why. I think the key factor is politics and power, and in the case of customer service or call center areas, you have a fairly powerless organization to stop anything happening to them. You have the ability to force through the change that needs to be changed and needs to happen and the behavior patterns and the cultural change—that sort of thing. You can also see it going horribly wrong in some cases in call centers, but what we're generally seeing is success.

If you're looking at sales organizations, they're extremely powerful, politically. I think the best example I can give you is this. About three years ago, I watched an annual sales kick-off meeting for an organization. They were going to send their best sales guys off to Hawaii. The global sales director was to congratulate the top three sales guys on the stage. The audience were allowed to ask questions, and some put their hands up and said: "They achieved target, but they didn't use the sales application. What are you going to do about it?" And, of course, what's the sales director going to do?

Bob Thompson
Nothing.

Ed Thompson
Absolutely nothing.

Bob Thompson
He's going to enter the data for them.

Ed Thompson
Yeah, exactly. He'll say, "Don't worry about it." And, of course, everyone, then, immediately stops using the application; they want to go to Hawaii next year.

Bob Thompson
What about marketing applications?

Ed Thompson
We'll have to break this down a little bit. In the campaign management area, I think, extremely successful—more and more successful because of the analytics going behind it. They're moving away from mass mailing, mass blasts, and to the more personalized approach. That's starting to yield substantial results, but that's being iterative. And the key driver there is people moving from three campaigns to 30 to 300 campaigns, which are much more targeted. And the response rates are going up and up and up. And, the lack of irritation on customers, as well, is going down and down and down—for the companies who are getting it right.

So, that's beginning to yield substantial results. Areas like MRM—Marketing Resource Management—where you're talking about the automation of the marketing function in terms of budgeting, planning, digital asset management, sign-off, working with creative agencies and creation of collateral are still in very early days. It's really hard to tell whether that's been successful at all, because it's been very political.

Bob Thompson
Right. How about analytics?

Ed Thompson
Analytics, in some areas, if you're talking about things like churn. I think the issue with analytics is it's trial and error, so one of the problems you get is if you're trying to measure absolute success on the use of analytics in one area, it's usually a mistake. Often what you're getting is people using the applications to try things out—experiment—so, they might get three failures and one success, and the success may pay for 20 failures. So you end up with a problem, where people try to measure over too short a period of time.

Often they're trying to measure it over 12 months: "Have we got a return from doing this?" And, in fact, people have made three mistakes, learned from it and they've just incorporated that in the fourth attempt to reduce churn. And bingo! It's started to have big results.

I think the issue there is the payback period on some of these technologies is three, four, five years. So, it's still pretty early. Same with loyalty management systems.

Bob Thompson
Right. So, it's going to vary by application area.

Ed Thompson
I think you'd have to say that.

Bob Thompson
If you put it all together, do you feel that the trend has been up in the last couple of years, that people are kind of getting their arms around it?

Ed Thompson
Yeah, they are. One of the things we did in 2002—we repeated this three or four times—was ask, "What do you find difficult about CRM?" We used these Eight Building Blocks of CRM. Gartner’s eight blocks of CRM are:

1. Vision
2. Strategy
3. The customer experience
4. The organizational collaboration and transformation issues
5. The process changes you have to make
6. The customer data
7. The technology
8. The metrics associated with it

We've used this since 2001 when evaluating case studies, when we're doing our CRM Excellence awards, when we're trying to evaluate whether people get a return from the changes they make and to evaluate where they find things difficult.

What you find in asking organizations what they find difficult is that people find the easiest thing is to create the vision and to get some buy in, so there's this myth that "we never get any buy in from the top of the organization." They usually do, but then executives get bored and then move on to something else. Often they can put together a vision and articulate what they're trying to achieve. It's actually one of the easiest things to do. Funny enough, consistently, we find that the actual technology implementation is the second easiest thing.

The problem? Metrics

So, the question is: What do they find difficult? No. 1 is metrics. They consistently find: "What do we measure to tell whether this is a success?" is very difficult to quantify. So, if you say we're trying to improve customer retention or reduce churn, what's your metric for churn? What's your metric for retention? Is it retention of the household? The product? The customer? The customer's family? The customer's products? You can measure it in many, many different ways, and often, people haven't delved down deep enough to work out what it is they're actually trying to achieve, which key metric—which lever—they're trying to pull.

If you get into discussions about that in most organizations, all hell breaks loose. And this, I think, is why it can't produce this question of: Are they failures or are they success? The answer is they just don't know, because they don't know what they're trying to achieve because they never were granular enough about what the objective was. So, I think when you say you're trying to improve customer satisfaction, well, OK, which metric of customer satisfaction are we measuring? That's the No. 1 problem.

The No. 2 problem that comes up consistently is processes, people having no process definition. When you say, "We're going to improve the order-to-cash process," or "We're trying to improve the campaign-to-compensation process" or whatever end-to-end process people are working on, the problem is nobody owns it. It cuts across different departments, and therefore, it breaks down at the point where the marketing department passes the leads to the sales guys or the sales guys are passing on a customer to the service department—or whatever. When those breaks happen, the customer's left hanging in the breeze, and there's no owner to it. There's no understanding of the end-to-end process. It's not fully automated. It's all in people's heads.

But, if you look at organizations now, you see a much bigger focus on improving customer processes, and you see a much bigger focus on looking at the process from the customer's point of view, which is a big, big change over the last four years since 2000. At that time, we didn't see a whole heap of organizations focusing on the process, from the customer's standpoint.

The third area is people. Look at QCI, who are part of the Ogilvy Group, who do a benchmark for a customer for CRM—very interesting, they have benchmarks on 600, 700 companies now—and compare them on their scale. It takes about six or seven weeks to interview an organization and pool the data together and then consolidate it. They score 1 to 100. They said the average score of the organizations they survey is about 32 or 33 out of 100.

So the most stunning thing I find from what they learnt is that the biggest return you get is in changing the way employees behave by incenting them or restructuring them or giving them more power. That has the biggest financial return—not technology or anything else you do. They also said that consistently, they find organizations where one department is a leading best-in-class example and then right next to it is another department who are the worst example in their industry in the same organization.

Bob Thompson
What determines one vs. the other?

Ed Thompson
Well, what they're doing is looking for best practice in, let's say, the marketing function and how they create collateral. Right next door to that might be another part of the marketing function, where they are doing direct marketing campaigns. One might be very absolutely slick and extremely professional and cleaning up the data and working very accurately and targeting customers and getting fantastic returns and making customers happy and selling more products. Meanwhile, the collateral guys are taking eight weeks to create a photograph of the smiling face of a pet dog in someone's house. So, what you've got is one side stretching away into the distance, while the other side's holding them back and screwing up for the company as a whole.

So the issue, in most cases, is looking for best practice process by process and copying your competitors in some areas just to get yourself up to be consistent across the organization. Just by doing that, you can double your score within QCi. The best scores they've seen are something like 70 or 80 out of 100. The average is only about 33, They use a different model to our eight building blocks, but we have compared our research with their benchmarks and we get pretty consistent results. They often find the strategy/metrics area is a big problem; the understanding of what the value proposition is to the customer is a big problem. The organizational change issues and the process issues that that we find are very similar in both research approaches.

Bob Thompson
We've run a survey on CRMGuru the last three years, and strategy, metrics and organization and people issues are usually in the top three or four.

Ed Thompson
It's exactly the same as ours. We asked a slightly different question. We said, "What are you interested in at our CRM summits?" And, that's moved around a bit. We saw last year—April 2003 until April 2004—customer experience shot right to the top: How do you improve? What is the customer experience? How do you improve it? How do you measure it? "How do you design the customer experience?" is the question that comes through, as opposed to just letting it happen for the customer.

There's a lot of discussion around that, which often leads into discussions about branding and the brand promise or expectation-setting. And feedback systems have become a popular topic within the broader subject of customer experience—not just annual customer satisfaction surveys. That's been very hot, but that's dropped down now. The No. 1 area that came up this last time we did it, in July for the October CRM Summit, was "business decisions for CRM executives." Decision-making—or decision-taking—and governance were up there in terms of popularity. Metrics are still in the top four or five. One is about difficulty, and the other one we're asking is the business value. It's interesting to see which ones bounce to the top, but there are four or five topics that tend to move around the top 10 all the time.

Bob Thompson
Well, you're one of the most knowledgeable people I've every spoken to about CRM, having studied it so many years, and I really appreciate you taking the time. Thank you.

Edmund (Ed) Thompson is a vice president and research director in Gartner's Customer Relationship Management practice, responsible for managing and coordinating the CRM strategy research agenda in Europe and research into customer experience management worldwide. He has a bachelor's degree in economics from Portsmouth Polytechnic and a diploma in marketing.