Measure Up: How to Dodge Cognitive Biases Using Calibration

“What gets measured gets managed” - Peter Drucker

How do you Measure Up?

“Boxing,” Sugar Ray Leonard wrote, “is the ultimate challenge. There’s nothing that compares to testing yourself like the way you do in the ring.”

Aside from the fact that he’s one of the most successful athletes in the history of the sport, it’s hard to argue with Sugar Ray. Before the fighters even step in the ring they’ve been pushed to their limits mentally and physically.

At the highest level, pre-fight training camps are usually 8-10 weeks long and daily activities include gruelling long distance runs to build endurance, intensive resistance workouts to develop power and agility and many hours of practicing boxing skills – hitting pads and bags, shadowboxing and sparring.

Everything is planned with military precision and every detail is recorded and scrutinised, whether it’s the fighter’s latest running time, the video from his latest sparring session or his scale weight as he seeks to make the limit for the fight.

With every day that passes, the fighter and his team know how far they are from hitting peak performance for fight night.

This intense process of deliberate practice is something that’s replicated by high performers across disciplines, whether it’s music, sports or academics.

However, without the ability to evaluate our own performance we’ll struggle to master anything, even with thousands of hours of practice. A lack of self-awareness means we’ll fail to change things up when something’s not working or give up just as we’re about to make a breakthrough.

Metacognition and Modes of Thinking

To be accurate self-observers, we need to develop our levels of metacognition, the awareness of our own thoughts and mental processes. With a more accurate view of reality, we can make better decisions and reflect on what we need to improve.

Unfortunately for us, our minds can often sabotage this process and lead us down blind alleys, so we need to be on our guard at all times.

In Thinking, Fast and Slow, cognitive scientists Daniel Kahneman and Amos Tversky define the two analytic systems that govern our thinking.

System 1, the automatic or fast system is unconscious and intuitive and leads to immediate, instinctive behaviour, like a fighter dodging a right hook in the ring.

System 2, the controlled or slow system, relies on conscious analysis and reasoning to consider choices and make decisions. An example of this would be the boxer mentally rehearsing his own strategy between rounds.

The problem we often have is that the rapid intuitive insights of System 1 are often wrong. When it comes to learning, this means that we suffer from illusions of knowing, which cloud our judgment of our own abilities and can mean we end up sticking with ineffective learning strategies that aren’t working.

Incompetent and Unaware

The research of cognitive scientists David Dunning and Justin Kruger shows that people often overestimate their own performance and see no need to try and improve because of this.

In a 1999 study, participants were given a simple logic test and asked to rate their own performance afterwards. The least competent students tended to wildly overrate themselves – students who scored in the 12th percentile on average estimated that they were in the 68th percentile – displaying a clear tendency for inflated self-assessment from illusions of knowing.

Despite our difficulties in judging our own performance Dunning and Kruger proved that metacognition could be taught to help people improve their self-assessment in a follow up study.

The experiment was repeated as before with similar results, but afterwards half of the students were given ten minutes of training in logic and the other half did an unrelated task. All the students were then asked to re estimate how they’d done on the test and those who received the training became more accurate estimators of their performance.

Calibration, Sparring and Testing

So how can we avoid the traps of System 1 thinking and improve our metacognition to form an accurate assessment of our own performance?

One of the best tools available to us is what cognitive scientists call calibration, the act of aligning our assessment of our own performance with objective feedback. This can be done through  testing ourselves regularly by practicing our skill of choice in context.

Returning to the boxing ring, sparring provides the most direct feedback possible – if your skills aren’t up to scratch, you get knocked out.

For academic skills the consequences of inaccurate self assessment might not see you lying face down on the canvas, but not testing yourself before an exam is like showing up to a fight without sparring any opponents beforehand.

Not a good idea in my book.

The Takeaway

Being able to monitor our own performance is a critical element of effective learning but we’re often misled by cognitive biases, which mean that we’re less likely to recognise our own incompetence, and may continue to use ineffective techniques when they’re not serving us well.

Try This

1) Find a Mentor

As it’s difficult to assess your own performance level, having a mentor who can give you objective feedback is extremely useful and can help you to find a way through the sticking points and plateaus. Ideally the mentor will have achieved your desired learning goal and will have helped several others achieve it as well.

2) Test Yourself Regularly

All learners should be encouraged to test themselves through practice tests and quizzes for academics and practice in context for skills and sports. Treat practice as the real thing as much as possible  so that you can realistically assess your level and improve over time.

3) Learn from Your Peers

Join a community of learners in your chosen field and observe the top performers. What are they doing that others aren’t?

Also assess your competence through interactions with your peers – at a Spanish language exchange you could see how well you’re able to hold a conversation with the same native speaker as the weeks progress.

Each week, rate your fluency with a score from 1 to 10 or even ask the other person to rate you.  Guard against excessive self-comparison but remember that using it sparingly is useful in the right context.