Managing a business is like flying an airplane. You need a good pilot and equipment to take off and keep it flying to the right destination. During flight, the pilot has a dashboard of instruments to indicate speed, height, direction, rate of ascent/descent, fuel, engine status and so on.

In business, the equivalent of that cockpit is the Balanced Scorecard, developed in the late 1990s by Robert Kaplan (Harvard Business School) and David Norton. The idea was to provide a balanced measurement of company performance using a combination of financial, customer, process and employee perspectives.

Measuring customer satisfaction and loyalty is a bit more complicated than counting revenue, however. In fact, some experts I interviewed for this article said that one of the problems with loyalty research is that it's too much like "quantum physics." Actually, I think quantum physics is easier.

Relationship Health Indicators

Loyalty gurus generally agree that customer loyalty is more than just repeat-buying behavior. Michael Lowenstein of Harris Interactive, for example, says that loyalty is a complex combination of a "state of mind" and a "set of behaviors." OK, so how are we going to measure that?

In 2006, Fred Reichheld's book, The Ultimate Question (Harvard Business School Press, March 2006), proposed a simple method to measure loyalty, called the Net Promoter® Score. (Note: Net Promoter is a registered trademark of Satmetrix, Bain and Reichheld). Based on responses on a zero- to 10-point scale, customers are grouped into promoters (9 or 10), passives (7 or 8 ) and detractors (0 to 6). Subtract the percentage of detractors from promoters, and, voilà, you've got a Net Promoter Score (NPS).

With that, Reichheld asserts, you can get rid of those long surveys and expensive loyalty researchers. Just focus on improving your NPS, and your company will grow. Prominent brands like Amex, GE and Intuit have embraced the method, which has created a bandwagon effect the likes of which we haven't seen since TQM, BPR and CRM.

The trouble is, I've found no solid evidence that NPS is the single best customer feedback metric for all companies. Or, that it's even a valid metric in all situations. But I suppose a book entitled, One Pretty Good Question That Might Help You Grow, wouldn't sell too many copies.

Simple, or Simplistic?

A growing cast of NPS critics says Reichheld has gone too far, dumbing down customer loyalty into an overly simplistic measurement. Jim Barnes of BMAI Strategy reviewed 15 loyalty models used by major corporations and found three groups of factors: functional, experiential and emotional. "Reducing loyalty to a single number is laughable," he said.

Recent independent research suggests that, at best, NPS is not the "best" metric and can't be applied in all industries. At worst, it's just plain wrong and will lead managers astray. While Reichheld points out that his critics are protecting their business interests, it's also fair to say that the hype surrounding NPS has been good for sales of Reichheld's books and keynote speeches, and probably helps Satmetrix sell more customer feedback solutions, too.

Timothy Keiningham of IPSOS Loyalty seems to be leading the charge to put Reichheld's claims up for public inspection. In particular, Keiningham and his academic colleagues attempted to replicate Reichheld's comparison of NPS and ACSI (American Customer Satisfaction Index), where he claimed that there was no correlation between ACSI and growth. The new study found that the ACSI and NPS offer similar predictive capability using Reichheld's own data. Keiningham says this finding suggests bias in Reichheld's research, attempting to discredit ACSI, a competing loyalty metric.

Laura Brooks of Satmetrix, who conducted the original research with Reichheld in the 2000-2001 time frame, flatly denies any bias. Brooks claims that Satmetrix "has tangible evidence in enterprises we've worked with." Maybe so, but I'd suggest this is like a software vendor saying it has a few successful customers, and then proclaiming all other customers will receive the same benefits. That's called marketing, not research.

Larry Freed, president and CEO of ForeSee Results, which supports the ACSI, found in an online retail study of 8,400 consumers that NPS doesn't cause growth but, rather, is "spuriously" correlated to growth, while the real growth "driver" was good old-fashioned customer satisfaction. He also says, as other researchers have told me, that NPS has statistical shortcomings. Why go to the trouble of asking people for a zero-to-10 rating, then "throw away" information by categorizing into three buckets? A simple mean (average) score would be more reliable (less statistical error). But perhaps not as memorable as "net promoter."

Respected academics, who shouldn't have any ax to grind in this debate, find little support for NPS in independent research. In the paper, The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Business Performance, Neil Morgan, a professor at Indiana University, and Lop Leotte do Rego, a professor at the University of Iowa, found that "metrics based on recommendation intentions (net promoters) and behaviors (average number of recommendations) have little or no predictive value." The most effective measurement? Average satisfaction scores.

For his part, Reichheld won't respond to his critics directly. And he declined to answer my questions by phone or email. (For the record, CustomerThink has no vested interest in any specific loyalty methodology.) His stock response is that "the value of NPS will not be determined in academic debating circles but in the real world of customers and employees." To be fair, in July 2006 he noted in a blog that NPS "does not provide proof of a causal connection between NPS and growth" and that some of the original collected data was "imperfect." You think?

So it's a standoff. The NPS camp (Bain, Reichheld and Satmetrix) has a simple metric with proprietary data they say proves it works. Academics and loyalty researchers often make their data public, at least for peer review, but keep their loyalty models locked up in their own black boxes.

Managing Loyalty in the Real World

Yes, NPS can work for individual companies. Intuit, one of the most customer-centric companies around, is a Net Promoter advocate. To understand why customers are promoters versus detractors, Intuit uses a text analytics solution from Clarabridge to mine write-in comments from TurboTax users. Now you've got the best of both worlds: a very short survey with diagnostic capability. (See my blog, Use Text Analytics To Listen to Customers on Their Terms.)

FileNet, a content management vendor acquired by IBM in 2006, has more than five years of experience with customer loyalty programs, under the direction of Suhail Khan, head of Worldwide Customer Loyalty. FileNet used to calculate a "loyalty index" but switched to NPS a couple of years ago and found it to be "easier to digest" for the organization and a "great tool to drive change." What's more, Khan says FileNet can link NPS improvements to revenue growth. Technology provider: Satmetrix.

But some companies, initially drawn to the simplicity of NPS, are changing it to suit their unique business needs. In short, they don't use the "just one question" methodology that Reichheld recommends.

Model Metrics, a salesforce.com consulting partner based in Chicago, found inspiration in the NPS concept but adapted the methodology. The firm's founder, Adam Kaplan, and principal consultant, Kevin Turner, say they also ask the customer to rate the quality of service and likelihood of future business. Three questions along with comments keep the survey short and response rates high, while still providing enough information to take action. (See Turner's CustomerThink article, Does the Ultimate Question Work for Loyalty? My Experience Says Yes.)

Thermo Fisher Scientific spent several years refining its loyalty management approach. Tricia Rakiey, Global Customer Allegiance leader, says that the company was initially drawn to the NPS concept but, like Model Metrics, wanted more questions to add diagnostic abilities. Surveys can run 20 to 30 questions long, with critical questions covering overall customer satisfaction, ease of doing business, likelihood to repurchase and likelihood to recommend. While the jury is still out in terms of statistical proof, Rakiey says that the company's composite Customer Allegiance Score has helped to rally people to work on customer issues. Technology provider: CustomerSat.

And, of course, many companies don't rely on NPS at all. A recent CustomerThink member survey found that just 18 percent have implemented NPS. Jill Griffin of The Griffin Group says that about one-third of her clients have expressed some interest in NPS. Still, the most popular implemented metrics for CustomerThink members are overall customer satisfaction (69 percent), likelihood to recommend (56 percent) and likelihood to buy again (43 percent).

Egg, a European provider of digital banking and financial services, doesn't use NPS, and hasn't found the underlying likelihood-to-recommend question to be very useful, either. According to John Jennick, head of Customer Experience and Action, Egg has used a "test and learn" approach to find the best questions.

While willingness to recommend is part of the mix, it's not the "one question" that drives Egg's growth. In fact, Egg found that asking if the interaction with Egg "made me feel my time was well spent" provided results that helped managers make better decisions on corrective action needed. Technology provider: Confirmit.

NPS Bubble Will Burst

Let's tip our hat to Reichheld for making customer loyalty a board room discussion. NPS popularity has also been an overdue wake-up call to loyalty researchers that they should be more practical. Especially for smaller businesses that can't afford their services.

But I'm dismayed at the finger-pointing I've seen in the loyalty research industry. It's unprofessional, reminding me of software executive flame wars. I don't know whom to trust. Certainly not Reichheld, anymore.

Furthermore, I'm very disappointed that Reichheld has not provided any third-party supported research that NPS truly is "the one number you need to grow." Cases studies and stories aren't proof. The fact that GE likes NPS doesn't mean it will work for everyone, or anyone, else. Remember Six Sigma?

Also keep in mind that revenue growth isn't the only important business objective. Improving gross margins, net operating cash flow, shareholder return and market share were also ranked high in our recent CustomerThink survey. Moreover, Jeff Marr of Walker Information points out that in some business models, the key to growth may hinge on prospects, not necessarily existing customers.

I predict that NPS hype will peak in the next year, then decline as more business leaders realize it's not the quick fix they thought it was. Then it's back to the book store to find the next fad diet.

Recommendations

In the meantime, what can you do? For starters, have an open mind. Unless you want to bet your career on one magic number, invest some time to understand both sides of the debate.

1. Remember that any loyalty metric doesn't cause growth; it's just an indicator. Make sure it's one of the key "dials" on your business management dashboard, in balance with other key business measurements. Odds are, revenue growth isn't the only thing that matters.
2. Keep surveys as short as possible but not too short. Depending on the situation, long surveys can work quite well. But for transactional surveys, a few questions (five to 10) is the limit. Make sure they include the right questions—those that link to improved business performance and provide adequate diagnostic capability on underlying loyalty drivers.
3. If you decide to rely solely on NPS, validate it with common sense and some history. Not so sure? Then hedge your bets by also asking questions about overall satisfaction and likelihood to repurchase. Randy Brandt of Maritz Research says the right questions will "depend on market conditions, rules of competition and buying patterns."
4. Brand your metric and make it your own. Develop an internal marketing campaign to explain what the metric means (no statistical arguments, please) and how it helps the organization succeed.
5. Reward managers and employees for improving customer loyalty, but don't overdo it. Management judgment has not gone out of style. If you make the rewards or penalties too strong, you'll stimulate unwanted "gaming" behavior. Remember, counsels Jeanne Bliss of CustomerBLISS, the point is to drive change within the organization to "get people to work better together" serving customers.
6. Plan to evolve and refine your measurements and rewards. You'll need to make adjustments over several years to keep it working just right.

Which brings us back to the key question: Is there one "ultimate question" that will effectively measure customer loyalty for all businesses? The short answer: No.

But the "ultimate answer" is that you must figure out the right metrics for your business. What's at stake is your customers' loyalty—and your future success. Do you have anything better to do with your time?