Is NPS truly ‘the growth marketer’s secret weapon’ or more ‘snake oil and fake science’?

“Net Promoter Score was the raging score that everyone zeroed in on and obsessed over. I was in a senior role at an international company, and we had our NPS on our wall, updated every Friday.

“We’d spend the week after the update searching to find out was going on with the score because it had dropped by 0.5 points. Then Friday would come again, the update would happen and the score had jumped up 1.5 points.

“So, we'd ditch the hunt for the 0.5 drop from last week and head off looking for the reason for the net gain of 1. And this carried on for months.”

Running around to Yakety Sax music aside, the popularity of NPS has continuously grown so much that today even the global Growth Marketing Conference series has called it “the growth marketer’s secret weapon”.

But is that true?

The history of Net Promoter Score

In December 2003, Bain & Company consultant Fred Reichheld wrote a Harvard Business Review article that introduced NPS to the world as “the one number you need to grow”. 16 years later, Bain reports that more than 200 major companies have publicly stated that they use the score. But it has also become one of the most controversial marketing metrics.

For NPS, companies ask customers – typically over automated emails but also through methods such as personal phone calls – to rate on a scale of 0-10 the likelihood that they would recommend the business to others.

Those who answer nine or ten are ‘promoters’. Seven or eight are ‘passives’. All others below are ‘detractors’. NPS is calculated by subtracting the percentage of detractors from that of promoters, resulting in a score between -100 and 100. Passives are ignored.

“The percentage of customers who were enthusiastic enough to refer a friend or colleague – perhaps the strongest sign of customer loyalty – correlated directly with differences in growth rates among competitors,” Reichheld wrote.

“The only path to profitable growth may lie in a company’s ability to get its loyal customers to become, in effect, its marketing department.”

The potential uses of NPS

Brett Zucker, chief marketing officer of US digital typesetting and typeface design company Monotype, likes NPS because it holds companies to high standards. He also believes everyone in an organization – even those who process invoices from outsiders – influences the score.

“The detractor range is three times the size of the promoter range, so to get scores consistently in the promoter range, a company must really commit itself at every step of engagement – not just product experience – to delighting customers,” he said. “The philosophy that everyone in an organisation has an effect on NPS is a must.”

Tina Dobie, chief customer officer at the Wordpress hosting platform WP Engine, thinks NPS should be used to measure long-term relationships and short-term transactions. The company also has a senior team member follow-up with detractors to see what might help.

“NPS has no value unless you do something with it,” she said. “But if you do, it’s a gift. Hearing the voice of your customer and having the customer feel like he or she has been heard is vital.”

Many companies use NPS with other metrics. WP Engine includes customer effort score (CES). Merkle, a global performance marketing agency, combines it with customer satisfaction score (CSAT).

“We do support the use of NPS because it is relatively quick to deploy and easy to administer,” Richard Lees, the firm’s EMEA executive vice president of solutions, said. “Its simplicity means response rates are usually high, making results representative across different segments or customer groups.”

“Nevertheless, we wouldn’t recommend the use of NPS in isolation. If a client finds that it has a high degree of detractors, trying to understand why and find a solution is a core part of closing the loop on the NPS process. This is a step many businesses fail to complete.”

Understanding the bigger picture

Christina Stahlkopf, associate director of research at international customer agency C Space, used a theme park as a hypothetical example of how companies can understand the context of a NPS score.

“If you ask customers directly about their experience, they will say they waited in line too long,” she said. “What they won’t say is how that wait made them feel – like their time was being wasted, or they were being ripped off.”

“Maybe a source of frustration about the line isn’t how long people have to wait. Maybe the issue is that park goers can’t tell how long they have left in the queue, that is there is no management of expectations – which leaves customers feeling ignored and disrespected. So, placing signs along the way updating riders on the wait time is all that is needed.”

For those who advocate for NPS, the importance of acting upon gathered scores cannot be minimised.

“It’s only by closing the loop – going back to the customer who has given the negative feedback and rectifying their issue – that it can actually act as a positive driver to improve the score,” Lindsay Willott, chief executive of the business rating platform Customer Thermometer, said.

“In fact, it is often detrimental to the customer experience if a customer takes the time to complete an NPS survey and the company who has sent it doesn’t bother to respond.”

NPS also has fierce critics

But opinion is divided. Michael Scott has used NPS for more than 12 years and is currently the co-founder and chief executive of the British CX platform The One Question. He now thinks the metric is “fundamentally flawed” and has replaced it with his company’s trademarked Customer Happiness Score.

Scott gave the example of two people visiting a car garage and having the exact same experience. One scores the business a nine while the other gives it a six.

“By their very nature, opinions are subjective – and so are interpretations of the NPS scale,” he said. “Two people could have the same experience and could give entirely different scores.”

Moreover, NPS accentuates the extremes by disregarding the ‘passives’ in the middle. If a restaurant receives all 8s, then the NPS would be zero because the equation takes into account only scores of 9-10 (positive) and 0-6 (negative). Even though eight out of ten is not a bad result.

“In this respect, NPS is an ‘all or nothing’ system,” Scott added. “It provides very little insight into where you can make improvements and can be confusing and even detrimental to staff morale.”

Bruce Clark, an associate professor of marketing at Northeastern University in Boston, echoed similar thoughts.

“If the promoters and detractors represent 30% of the customer base, 70% of your base is passive,” he said. “I suspect this is not unusual. Most customers don’t particularly care about your product one way or another. Understanding how to move those passives, however, is probably the biggest marketing opportunity for many firms.”

“The wording of the question, ‘How likely is it that you would recommend?’ is neither an indicator that the respondent intends to recommend nor that he or she does [actively] recommend,” Clark added.

For more critical thoughts on NPS, the professor also referred people to this Medium essay by Jared Spool, founder of the User Interface Engineering research organisation in Massachusetts.

Colin Cather, creative director at the British agency Bottle PR, believes that NPS is a type of ‘north star’ or ‘one number to rule them all’ metric that is ‘kryptonite for smart strategies.’

“NPS is sometimes used as a proxy brand-strength metric,” he said. “And yet it tells you little or nothing about awareness, penetration, salience or distinctiveness. It might say something about ‘advocacy’ – but this is a poor signal about how to grow.”

Byron Sharp, the director of the Ehrenberg-Bass Institute for Marketing Science at the University of South Australia, was characteristically more blunt back in 2008 and called Reichheld’s analysis ‘sloppy’ in a Journal of Marketing Research article that year.

"This is snake oil, fake science,” he wrote. “The lesson for market researchers and insight directors is just how easy it is to make compelling slogans from incorrect findings."

Indeed, NPS can be an example of businesses preferring to gauge what is cheap and easy to measure rather than what is actually important.

Bain & Company, where Reichheld is now a fellow and founder of the firm’s Loyalty practice, did not respond to requests for comment for this column.

NPS can be inaccurate or manipulated

In a speech at an event in England earlier this year, I discussed how much of the internet is fake. The same might be true for NPS scores through either direct manipulation or unintended mistakes.

First, Peter Abilla, vice president of growth marketing at the blockchain company ThunderCore, noted on LinkedIn that companies rarely analyse promoter, passive and detractor scores independently, leading to overgeneralisations based on a single net number.

Second, the design of the question itself can greatly affect the score. Just read this story from Mark Kirwin, director of learning and development at Blueprint Management Consultants, on his website.

A few months ago, I spoke at the ZeeMelt conference in Mumbai on the biggest myths in marketing today. Weeks later, I received this email from my Four Seasons hotel there.

When I saw this NPS question, I instinctively went to click ‘zero’ out of quick instinct because ‘10’ is usually on the right in English-language forms. Luckily for the hotel because I loved the accommodations, I stopped myself before it was too late.

And that is just one example of how UX can affect results. After all, political pollsters have always known that the exact text and design of questions can greatly influence the responses.

NPS is increasingly annoying

The biggest problem with NPS, however, is that the principle behind the metric has infected almost every aspect of our lives. Everyone and everything now seemingly wants a quick rating on something.

In part, you can thank – or blame – Dubai. In 2017, the country installed ‘happiness meters’ at government offices and other locations to measure and improve service offerings.

Similar to NPS, people quickly record either a positive, neutral or negative interaction. The idea spread. Personally, I have now seen similar machines at nearly every airport I have visited in the past two years.

And sometimes it leads to unintentionally hilarious results, as John Sills, a consultant at the British CX agency The Foundation, noted on Twitter.

Think I’ll give this touchscreen customer experience survey a miss... pic.twitter.com/ZH1rOZXsPs

— John J Sills (@johnjsills) July 12, 2019

The NPS middle ground

As with most things in marketing and communications, there is the thesis, antithesis and synthesis. NPS is likely neither as wonderful as proponents claim nor as useless as detractors state.

“Gartner likes NPS as a measure and uses it in some instances, but it needs to be deployed in the right situations – namely relationship surveys with senior leaders that have visibility beyond a single transaction or interaction,” Pete Slease, vice president of the analyst firm’s customer service and support practice, said.

“Asking NPS after a single transaction is akin to asking a cruise line guest to recommend after seeing her stateroom. It’s just not enough of an overall experience for the respondent to provide useful information about the likelihood to recommend.”

Instead, Slease, like WP Engine, recommends the additional use of CES to evaluate the easiness of an interaction for a prospect or customer. He does not suggest using CSAT because he said it correlates poorly with most desired financial outcomes.

Maxie Schmidt-Subramanian, a principal analyst at Forrester Research, said NPS can be useful.

“I am an advocate for having the right customer experience measurement framework that ties customers’ perceptions of their experiences to an organisation's financial and operational metrics,” she said. “Only then can companies make the business case for CX and identify operational levers to improve CX.

“NPS can have a place in it. Unfortunately, the hype around NPS led firms to embrace it without making sure it’s a good fit for their situation and without complementing it with other metrics and processes.”

Before companies decide on specific metrics, Schmidt-Subramanian said they should first ask themselves whether NPS is a leading indicator of the consumer behaviours that matter to the organisation, whether NPS motivates employees to improve, and whether NPS accurately measures what a firm wants to understand.

Remember the big picture

Smart Design design director Omar Bakhshi is a former OgilvyOne creative director who had set up the UX practice there. He believes the focus on NPS is driven partly by short-termism that focuses on quick profits and share price increases while ignoring the bigger picture.

“The real issue is too many brands have taken their eye off the ball when it comes to their brand purpose,” Bakhshi said, citing examples such as Nike’s need to inspire athletes and Lego’s desire to invent the future of play. “Marketers should be responsible for manifesting the brand purpose out to customers.”

“But if the brand purpose is right, strong and actually provides meaning to customers today, everything else will fall into place: communications, marketing, design, product ecosystems, employee engagement – and probably NPS scores.”

The Promotion Fix is an exclusive biweekly column for The Drum contributed by global keynote marketing speaker Samuel Scott, a former newspaper editor and director of marketing in the high-tech industry. Follow him on Twitter. Scott is based out of Tel Aviv, Israel.