“When all the experts and forecasts agree, something else is going to happen”- Bob Farrell
“Experts say…”. How many times does one hear or read that rote phrase when someone wishes to make a point that they expect to be uncontested, or taken at face value- the so-called appeal to authority. Sometimes the “experts” exist, and sometimes they are even recognized “experts”, yet far too often the designation is partial and self-serving. Being seen as a certain type of “expert” can be quite lucrative these days!
Use of the phrase is responsible for much lazy and ill-thought-through “judgement” in this world.
Of course, we are not in any sense denigrating specific, evidence-based expertise. Our world could not function without it, nor could the (re)insurance industry.
Unfortunately, however, “experts” are often no more expert in their judgement than a coin toss, particularly in the realms of finance and economics, in terms of their predictive abilities, and prone to making grandiloquent statements which sound useful, but mean nothing and have no information value.
As the initial quote implies, expert opinion can be in agreement, and be completely wrong, or misguided. After all, predictions of, say, the next recession are notoriously unreliable; and groupthink is always dangerous; while a premium is placed on being certain, which leads to binary outcomes, and much subsequent explaining of why the expert’s statement was somehow misinterpreted or misunderstood.
As Philip Tetlock, founder of the Good Judgement Project, and a UPenn professor, has repeatedly demonstrated, there is both science and art in becoming a true expert in terms of decision-making and forecasting.
In his book for the general reader, Superforecasting, Professor Tetlock laid out what he sees as the characteristics of a skilled forecaster. It makes interesting reading, and we make no apologies for referring to it as a useful checklist for each of us.
- Cautious: Nothing is certain.
- Humble:Reality is infinitely complex.
- Nondeterministic:Whatever happens is not meant to be and does not have to happen.
Abilities & thinking styles:
- Actively open-minded: Beliefs are hypotheses to be tested, not treasures to be protected.
- Intelligent and knowledgeable, with a “Need for Cognition”: Intellectually curious, enjoy puzzles and mental challenges.
- Reflective:Introspective and self-critical.
- Numerate: Comfortable with numbers.
Methods of forecasting:
- Pragmatic: Not wedded to any idea or agenda.
- Analytical:Capable of stepping back from the tip-of-your-nose perspective and considering other views.
- Dragonfly-eyed:Value diverse views and synthesize them into their own.
- Probabilistic:Judge using many grades of maybe.
- Thoughtful updaters:When facts change, they change their minds.
- Good intuitive psychologists: Aware of the value of checking thinking for cognitive and emotional biases.
- Growth mindset: Believe it’s possible to get better.
- Grit:Determined to keep at it however long it takes.
If there is one clear conclusion to be drawn from the above summary it is that being dogmatic and certain is a negative characteristic for anyone who wishes to have a reasonable chance of excelling at the art of prediction.
From our point of view, we would certainly subscribe to this approach. We do not claim to be “experts”, even though we do believe that we can evidence (through our track record over the past 8+ years of risk selection, underwriting and risk management) that we have some expertise in what we do (and so should be evaluated based upon that)- while being equally willing to acknowledge that we always need to re-appraise, refine and hone that expertise, and recognize that being “absolutely certain” is a dangerous mindset.
The Awbury Team