The Madness of Metrics: Be Careful What You Measure

A friend told me a story about when he was a customer service call center manager. He was concerned that the focus on dealing with an ever-increasing number of customer calls might be affecting the center's ability to keep up with customer correspondence.

He decided to measure the number of unanswered letters at the end of the month. He was surprised and frankly delighted to find that the number was zero. However, he was a little worried about the quality, as he knew customer service was still receiving a large number of letters and he had seen stacks of them on desks during his frequent walkabouts.

He, therefore, decided to leave the measure in place, just as precaution. But every month, there it was, zero letters waiting to be dealt with. Several months went by. One day, he happened to be in the mail room at the beginning of the month and saw a large box on the floor addressed to customer service. He asked the mail room manager what the box was for. The reply? "Oh, that's the customer letters box. It's goes into Collections just before the end of the month and comes back to Customer Service about a week later."

My friend didn't know whether to be angry that his people had hoodwinked him by sending the letters out of customer service when they were counted or to admire the creativity, imagination and sheer cheek required to pull it off.

Origins
"What you measure is what you get." I am sure those words echoed round my friend's head. But this is, by no means, an isolated incident. Every day we drive our organization using thousands of metrics, many of them contradictory and most against the interests of the customer and those who serve them. How did this happen?

It was 1776 when Adam Smith published The Wealth of Nations, in which he defined the division of labor. Job specialization as a means to improve productivity was the big idea that fuelled the Industrial Revolution. Business strategy expert Charles M. Savage, Ph.D., writes:

Smith broke up the pin-making process into 18 steps. This fragmentation of work was magnified into separate departments, each with its well-protected silo. It was reinforced by the writings of Frederick Winslow Taylor [known as the father of scientific management] and Henri Fayol [of the classical school of management theory]. Taylor's scientific management should really be called "stupefying management" because it assumed the worker was not smart enough to know what to do. Fayol locked people into managerial boxes with his chain-of-command model, a true chain around people's creativity.

So Smith & Co. condemned customers to more than 200 years of fractured experiences and workers to the endless frustration trying to make internal processes work better for customers. And the way they reinforced this artificial and un-natural structure? You got it: metrics. "If it moves, measure it; if it doesn't, paint it!" And of course, the metrics all measure the productivity, cost or efficiency of the fractured component of the process, pitching team against team. Savage says, "The factory model too often leads to structured distrust." You bet it does.

And we are still doing it. Still making the basic mistakes of, not the last century, but the one before that! The irony in all this is that it's not only customers and employees who don't want it to be this way, but also executives don't really want it to be this way. And I don't believe shareholders want it, either. So why do we continue doing it? Could it be we know what not to do, but we don't know what to replace it with?

We all know the rhetoric: Be more customer-focused. We've all seen "Customer First" programs. But why don't they work? It's because we say one thing and measure another. We just don't walk the talk. We say, "focus on the needs of customer," but we measure call handling time. We say, "improve the customer experience," but we measure the productivity of each functional component of a process, not the end-to-end cycle time. We even use internal service-level agreements (SLAs) as a defense mechanism to protect us from blame (another trait of command-and-control organizations). At the end of the day, workers do what they are paid to do, which is usually underpinned by the wrong metrics.

I could fill pages and pages with devastatingly stupid examples of metrics that create the wrong outcomes, as I am sure all the readers can. One particular example serves as a warning to us all.

Creativity
A home electronics manufacturer made product components in separate factories and delivered these to a warehouse for collation and onward shipping to the customer. The warehousing manager was measured on warehousing costs, turnaround time and lost goods. Like my friend's customer service team, he came up with an idea to improve them all. He proposed abolishing the warehouse and shipping from each component location direct to the customer. The board approved this because of the large savings.

So customers received, not one package with complete products, but two or sometimes three, depending on what extras they ordered—and never at the same time. So what happened? Customers by the thousands swamped the call center because only part of their order had arrived, and returns increased by more than 1,000 percent in a week. Did the company really save money? It depends on how you do the costing, because, like the metrics they create, financial folk can be quite selective with their version of the truth.

The solution? Align the metrics with the declared vision of the business. Only then will it force the right behaviors to happen across the organization. If you are truly focused on customer satisfaction, then EVERY operational measure should be either expressed in these terms or clearly connected to this as an outcome.

Processes should be designed around customer events, and the key metric is the end-to-end cycle time of that process, as well as the number of improvements to the process made by customers or customer-facing people. The leadership team should then review only these measures, not reinforce the Smith-Taylor-Fayol model of two centuries ago by focusing on revenue, profit and cost. But only if they really believe that satisfied customers leads to long-term loyalty and value.

As a kid, I was always told, "Be careful what you wish for, as it may come true.