Flawed Feedback
How organisations undermine their own sense making
“If I could, I would add an eleventh commandment to the first ten: Thou shalt not distort, delay, or withhold information.” — Donella Meadows1
Although organisational culture is inherently complex, it’s tangibly experienced by people as “the way we do things round here”.2
The three most important things people in an organisation “do round here” are:
Make sense of the organisation and the ever-changing context in which it operates;
Make decisions about what to do — and what not to do; and
Take actions in the world.
Sustained success ultimately depends on:
how and how well the organisation does sense making
how and how well it does decision making
how and how well it does action taking, and
how and how well these three essential activities are joined up.
For an organisation to thrive in our increasingly uncertain and unpredictable world, sense making, decision making, and action taking must become tightly coupled, rapidly and repeatedly iterated, deeply embedded, and widely distributed throughout the organisation.
That, in a nutshell, describes a future-fit culture.3
Unfortunately, organisations often attempt sense making in ways that don’t make much sense.
Take for example the note I was given by a checkout operator at our local convenience store: “Hope you had a fantastic instore experience today. Can you rate our service please at <QR code/URL > Today you were served by Jamie. I only get recognised when it’s a 10”.
How unbiased is my feedback likely to be when prompted by: “I only get recognised when it’s a 10”?
If I thought Jamie’s performance was only a 6, would I bother going online to score him? Does this prompting mean Jamie gets more 10’s than reflects the actual average instore customer experience..?
This gaming of the system is one of many deficiencies of feedback scoring systems in general, and the deeply flawed but widely used Net Promoter Score (NPS) in particular. 4
Then there’s the question of whether Jamie's performance on the checkout actually drives whether, as a customer, I have “a fantastic instore experience”.
What if Jamie performed brilliantly on the checkout, but my overall experience was dominated by the fact the store had run out of my favourite bread..?
What if I was in a hurry, got stuck behind a shopper who couldn’t decide which cheese to buy, and the store layout made it impossible for me to bypass them, finish my shop, and get on with my day..?
The only way the convenience store’s feedback process could evaluate “a fantastic experience” is if by “fantastic” they mean the original definition: “weird; insane; make-believe.” 5
But it’s not only feedback systems based on NPS that are Not Proper Sensemaking.
The world is awash with similar make-believe feedback systems masquerading as, masking, and manipulating organisational sense making.
Take the ubiquitous happy sheets used to evaluate learning and development events.
From 2010 — 2015 I was retained as a specialist advisor on innovation leadership and learning with one of the world's leading providers of executive education.6
They were regularly scored #1 worldwide in custom executive education, but still relied on scores on happy sheets to assess their course presenters.
What happy sheet scores actually measure, and therefore encourage presenters to deliver, is infotainment — information delivered in entertaining ways that participants appreciate as light relief from their normal day-to-day work.
Unfortunately, infotainment rarely triggers lasting learning in participants, nor worthwhile change in their organisations…
My most memorable experience of this came from an early foray into delivering skills development training for technical managers some 40 years ago. 7
I was apprenticed to an energetic Scotsman who, at the end of the final session of a four-day course, handed out the obligatory happy sheets and proceeded to explain the feedback scoring system in a deliberately fierce and slightly scary Glaswegian accent:
“On the feedback form you have to enter a letter from A to D.
So, if you think the course was amazing, “amazing” begins with an A — so you enter A on the form.
If you think the course was average, “average” begins with an A — so you enter A.
If you think the course was awful, “awful” begins with an A — so you enter A.”
He was an adequate and entertaining instructor.
But his chutzpah, and his deliberately cultivated half-joking, half-menacing explanation of the scoring system meant he invariably scored straight A’s. 8
A big part of the problem is that feedback metrics like these seek to serve two purposes: on one hand they purport to measure service effectiveness; on the other hand they’re used to reward or penalise those delivering the service. 9
The latter purpose creates a powerful incentive to employ tricks, like those used by the instructor above, to manipulate customers into providing flawed feedback that undermines the former purpose.
The old adage: “what gets measured gets managed” is very often: “what gets measured gets manipulated”.
If you want further evidence that the feedback systems organisations employ rarely provide senior executives with good sense making about the actual experience of customers — and the employees who directly serve them — you need look no further than the hugely successful TV franchise Undercover Boss. 10
First launched on the UK’s Channel 4 in 2009, the series places senior executives undercover in the body of their own organisations, where they learn first-hand how out of touch with reality are the perceptions they’ve formed via their flawed formal feedback “dashboards”.
Undercover Boss has been franchised in over twenty countries — suggesting how common and widespread the problem is of deficient sense making, leading to poor decision making and ineffective action taking in organisations around the world.
This reliance on flawed formal feedback systems is a symptom of a more serious organisational malaise — senior executives stuck in the dangerously outdated legacy mindset that their role is making decisions.
In future-fit cultures of innovation, agility, and adaptiveness, senior executives know that their role is not making decisions, but creating conditions in which good decisions get made and implemented continuously, dynamically, throughout the organisation. 11
Do you want to be a more effective catalyst for creating future-fit organisational cultures of innovation, agility, and adaptiveness?
Why not sign up for updates on the upcoming Foundations for Future-Fit Culture course?
Early registrations will receive a special discount when the course is available. Find out more and register your interest by clicking the button below…
Donella Meadows Thinking in Systems — A Primer on Goodreads. The book was originally circulated as a draft in 1993, and versions of this draft circulated informally within the systems dynamics community for years. After the death of Meadows in 2001, the book was restructured by her colleagues at the Sustainability Institute, edited by Diana Wright, and finally published in 2008.
Find out more about organisational culture, how it emerges, and how it can be influenced in this previous article.
Find out how to access the leverage for creating a future-fit culture in this previous article.
The many fallacies of Net Promoter Score (NPS) and why it’s Not Proper Sense making are neatly laid out in this article by User Experience (UX) Designer Jared Spool.
If you’re curious about who it was, check out my LinkedIn profile.
Cambridge Consultants, the open innovation lab I worked for at the time, was asked by an external training provider if one of our people would teach on their course for technical managers, and I was encouraged to explore the opportunity.
He explained his secret score elicitation strategy to me after a few bevvies at Ronnie Scott’s jazz club in London — part of an all-night bender, which he regarded as a necessary rite of passage for his apprentice infotainers…
Presenters got a relatively low base rate of pay with a significant bonus geared to happy sheet scores.
Find out more about this vital shift in the senior executive role in this previous article.