What’s your take on the great NPS debate? are we using it correctly? How robust is NPS to be used as measure of success and long term growth? Are there guidelines to when it should be used?
Too many companies these days are hooked on measures of customer trust ratings. Studies upon studies of the relationship between customer trust and business revenues tells us that it’s right to do so. The impact of bad customer trust ratings go beyond a company’s boundaries, easily impacting market share, profits and opportunities for true growth. But is it the end all and be all, the ultimate measure of business success?
Whenever a customer feels misled, mistreated, ignored, or coerced, chances are good that you will have created a disgruntled customer base. You would be lucky enough if the worst that happen is to churn the most loyal of your customers to your competitors. But as customer experience studies have shown, disgruntled customers find ways to get even – they drive up service cost by frequently reporting problems, they gripe to anyone who would care to listen, and even affect your frontline service team with their complaints and demands. Disgruntled customers ‘detracts’ from business growth, and the more of these customers you have, the more they inhibit and strangle your company’s opportunities. On the other end of the spectrum, when customers are delighted, they willingly come back for more. Not only that, they become advocates for what you are trying to sell. Advocacy ensures customers’ enthusiastic cooperation and your business gets free marketing from customer promotion, which in turn fuels growth.
The idea that customer trust and loyalty are the key to profitability and sustained growth makes perfect sense. Hence, companies feel it is imperative to be able to track how many of your customers fall into these two critical groups, and NPS seems like the Ultimate Question. But are we using it correctly? How robust is NPS to be used as measure of success and long term growth? Are there guidelines to when it should be used?
Net Promoter Score is based on the aggregation of data from a ten-point rating-scale question : How likely are you to recommend your provider to others? Customers who rate 9 or 10 are aggregated to form the ‘Promoters’ segment. Those that rate a 1,2,3,4,5 or 6 are categorized as ‘Detractors’. NPS is the difference between the percentage of Promoters and Detractors. It’s simplicity is its beauty. However, it is important to know the pitfalls of NPS before deciding to use it.
Firstly, NPS is a business-level metric, not a customer-level metric. Relative NPS should mean that the principal business is comparatively better or worse than other businesses in the category based on the differences in NPS levels. This is by virtue of NPS being an aggregate-level metric. Unfortunately, aggregate-level metrics don’t work if the intention is to understand share of wallet. This analysis must be done on customer-level metric. As a cardinal rule in statistics, it must be first proven that there is a strong relationship between the variable you are tracking and the outcome variable before you can aggregate the data. Without going into too much detail as to why, the “average” you come up with by aggregating the data cancel out the extremes. Hence, using an aggregated metric to represent the individuals within the group will be an ecological fallacy – you mistakenly think you understand individuals within the group simply with just one number that represents them.
The most glaring concern with using NPS is the compounded error. From a marketing science and statistics point of view, it is enough to discourage decisions made only on NPS. Professor Claes Fornell – the world’s leading authority on customer satisfaction measurement and customer asset management has stated it perfectly:
The problem has to do with how the numbers are assigned: A perfectly good scale is ruined to the point that it generates very little useful information. A competent measurement methodology looks to minimize error. But here, the opposite is done. Instead of getting precision, random noise is produced. From a single scale, we have not only converted something continuous to something binary, but we have done it three times (percent of customers likely to recommend, percent of customers not likely to do so and the difference between them). Each time, we have created a new estimate. All estimates contain error. Going from a continuous scale to a binary one introduces even more error.
If that’s not enough, taking the difference between the two estimates with error leads to exponentially greater error. In the end, we have produced a large amount of random noise, but very little information. When it comes to looking at changes over time, we further compound the problem. For each time period comparison, there are now six estimates and the final calculation is the percentage difference of customers that are likely to recommend. I have seen published reports sold for several thousands of dollars in which almost all the reported change is due to random noise. For managers, it’s bad enough to chase the numbers they can’t effect, but to chase randomly moving targets can do a great deal of harm to individual and company performance.
The bottom line : You can very well use NPS but first establish that there is a strong relationship between NPS and whatever variable you want to explain by variations in NPS, such as share of wallet.
As a cardinal rule : It must be first proven that there is a strong relationship between the variable you are tracking, whether it be satisfaction, purchase intention or purchase value and the individual’s NPS scale score before you can aggregate the data to report causality