Methods on Reporting Customer Satisfaction

Mir
Member

Posted 24-May-2005 07:18 AM
Hi...

Had a quick question about different methods to report customer satisfaction and which is more preferred. Using responses from a survey with a numerial scale for overall satisfaction, which method is ideal and for what reasons? Frequency of scores above a certain score (e.g. % scores above 6 on a 10 point scale) or simple average score?

Are there are any publications that I can check out that give details on which method should be used?

Any insight is appreciated...thanks!

Haroon

Graham Hill
Guru
Member

Posted 25-May-2005 05:45 AM
Mir

Customer satisfaction is a difficult thing to measure.

Many companies make the mistake of asking customers satisfaction-related questions about the things that are interesting for the company to know about, rather than the things that actually drive customer satisfaction from the customer's perspective. The implication is that you need to understand what actually drives customer satisfaction first, before, you can start to ask customers whether they are satisfied about the company's delivery of these things. In addition, it probably makes sense to ask customers about their committment to continuing to do business with the company too.

The relationship between customer satisfaction, retention, loyalty and profitability are complex. Most people who work in this field suggest that customer satisfaction scores of less than 9/10 put those customers at risk of defection; whether that is driven by service failures or by competitor activity.

Look at the American Customer Satisfaction Index website at http://www.theacsi.org/overview.htm for more information on customer satisfaction. Or look at 'Satisfaction: A Behavioral Perspective on the Consumer' by Richard L. Oliver for a detailed discussion of all aspects of customer satisfaction.

Customer committment is starting to be put forward as a better predictor of customer retention, loyalty and profitability than customer satisfaction. The South African market researchers Jan Hofmeier & Butch Rice have done the best work in this area.

Look at their conversion model website at http://www.conversionmodel.com/ for more information on committment.

Another area that is making a reappearance in association with customer satisfaction is the role of emotions (strictly speaking feelings, as emotions are pysiological responses to environmental stimuli!). For example, research in Australia has shown that even customers who had a customer satisfaction score of 7/7 (equals 100%) used negative words like frustration and anger 25% of the time when asked to descibe how they felt about their experiences with a company.

Look at 'Emotional Value: Taking Customer Service to a New Level' by Janelle Barlow and Dianna Maul for more information about this research and the role of emotions in customer satisfaction.

So there you have it. Customer Satisfaction is complex. You need to measure it in two steps. You need to incorporate commitment into it. You should think about incorporating emotions too. And there are lots of good sources of information to get you started.

Graham Hill
Independent CRM Consultant

Mir
Member

Posted 25-May-2005 06:58 AM
Hi Graham...

Thanks for the response...much appreciated...my problem however, is looking at how to analyze the data itself...we've been using simple average so far at our organization, however, I've been asked to look into different ways of how to express the results for an overall satisfaction question that we have on our surveys. One possible method is to use frequency (% responses above a certain result) or stay with the current method.

Is there any philosohical reason that would justify the use of one method over the other? Average vs. proportion percentages.

Also in our organization's case (pension plan administrator), customer retention or profitability isn't an issue (we're not-for-profit and customers join the Plan when they work for an employer that provides the Plan), however, we're still interested in measuring satisfaction since it gives an idea where we need to improve our services and communications and good satisfaction can imply that the organization is running a quality operation.

David Mangen
Member

Posted 26-May-2005 09:28 AM
Mir,

I believe that the answer to your question really depends on the distribution of responses to your satisfaction questions. As a general rule, however, I suspect that most of the time the preference for averages vs. percentage Top X Box scores comes down to whether the recipient of the information is comfortable thinking in linear, continuous terms (likes averages) versus thinks in categorical terms (prefers percentages).

However, returning to the issue of the distribution, and presenting this in extreme terms, let's assume that you are asking satisfaction on a seven point scale. If your distribution is:

Score Count
1 30
2 0
3 0
4 0
5 0
6 0
7 70

(Edited Note: My attempt to create a table is not working very well. The 2nd number in the string belows under the "Count" header.)

You would end up with a Top Box score of 70%, with an average of 5.2. Now, reporting the "Bottom 2 Box" score (something which is often done) would give you some idea about the distribution as well—clearly this is a situation where satisfaction is highly polarized.

Contrast that to the following distribution:

Score Count
1 7
2 0
3 0
4 0
5 43
6 50
7 0

The average is about the same—5.22, but Top Box would be 0.

I would argue that this latter, admitedly extreme, example would reflect a far easier "turnaround" situation than the former in that the total pattern is generally more favorable—less customer delight, but substantially lower resentment.

I would encourage you to do two things. First, develop a couple of metrics—overall average, Top Box, Bottom 2 Box, etc. More importantly, however, get out of thinking about just one metric. The key is understanding the total distribution.

David J. Mangen

Mir
Member

Posted 26-May-2005 09:41 AM
Hi David...

Thanks for the response...I actually do have multiple metrics (average and 3 boxes—mid, low, and high), the problem is that one has to be selected to be used in a corporate scorecard, and I've been asked to see what the industry viewpoint is on reporting averages vs. high box. Based on the existing data, the results are mainly clustered around the higher range of the scale.

Basically what I'm trying to find out is there a preferred method—it would seem that there isn't since the nature of the data dictates which of the methods is suitable. Which method would be suitable for a not-for-profit organization?

Susan Abbott
Member
Picture of Susan Abbott

Posted 26-May-2005 11:02 AM
Hi Mir -
Actually there are some preferred methods of analysis for this type of data. Most analysts prefer the 10 point scale for this, so you are already off to a good start.

Loyalty studies show that even those who rate satisfaction an 8 may not be loyal. This is because "satisfaction" relates to expectations, which may not be all that high, and are always changing. "Service quality" is something that people evaluate against some internal standard.

However, there is evidence that people who rate you a 9 or 10 (especially 10) are worth studying as a group. The approach I would suggest is to classify your data into groups: the 9/10 group, perhaps a 5—8 group, and so on. Then compare AVERAGES on other data between those groups.

Other data that would be useful would be things like: type of service used, frequency of use, facility location, numbers of services used, etc. Anything behavioral, not demographic is usually more informative.

You will likely find some interesting patterns worth investigating. You are looking for one or two other measures that seem closely correlated with satisfaction. (There are statistical methods for this, but you don't have to go that route if you don't have the budget. Seeing a strong pattern is sufficient)

Another school of thought is to subtract the "dissatisfied" (e.g. 1- 3) from the "satisfied" (e.g. 8—10), and focus on trends in that number over time.

You will find Reicheld's articles in Harvard Business Review (e.g. The One Number) useful in that respect.

Once you have your data, you might find it useful to run some group discussions with people who can help you understand specific elements of it. (e.g. focus group with users).

Good luck!

Customer Experience Strategist
** Qualitative Research ** Change Management Support
blog: www.arc.typepad.com/customercrossroads

Susan Abbott
Member
Picture of Susan Abbott

Posted 06-Jun-2005 06:05 AM
Hi Mir -
thinking further on your situation as a pension administrator...
I'm even less sure that satisfaction means anything, since the individual respondent's frame of reference is probably non-existent, if they are pension plan members. That is, they have no one to compare you to. For a discussion of the impact of frame of reference on perception, an excellent discussion can be found in the book "How Customers Think" by Gerald Zaltman, published by HBS Press.

If they are HR people who are in charge of your hiring you to manage their pension plan, and are in a position to posess some knowledge of the alternative choices, your satisfaction data probably mean more.

Since you have already collected the data, I would still suggest just looking at "excellent" scores, or "excellent" minus "poor" metric. And focusing on increasing those.

Cheers, and good luck.

Customer Experience Strategist
** Qualitative Research ** Change Management Support
blog: www.arc.typepad.com/customercrossroads