Models: Fit for Purpose…?

No, we are not talking about the catwalks of London or Paris!

We have mentioned before our skepticism about the unthinking reliance upon algorithms and financial models; and how such behaviour led at least in part to the recent Financial Crisis.

Now we are returning to the topic, because recently we read a paper written by Paul Pfleider of Stanford University, entitled “Chameleons: The Misuse of Theoretical Models in Finance and Economics”- the sub-title warmed our poor analysts’ hearts!

In very broad terms, the basis of Mr. Pfleider’s thesis is that many models on which those in economics and finance rely (models, which he terms “chameleons” and to which we at Awbury would apply the term “chimaera”) are based upon assumptions which may have little connection with the real world, or with the purpose for which they are supposedly intended: they have not gone through relevant and valid “filters”; in essence a process that incorporates experience from the real world- in many cases plain old-fashioned common sense and experience.

It is a cynical truism that if one “cherry picks” the data one can produce a desired result, which may appear plausible; but is, in fact, biased and misleading. In the scientific world, there are brewing and historic scandals about peer-reviewed papers that were, upon closer examination, or the failure to replicate outcomes, demonstrated to be negligently-produced (to be charitable), or simply fraudulent- “cold fusion” anyone?

Conversely, it is also possible to “cherry pick” assumptions in building a model; which can lead to equally, if not more insidious outcomes, depending upon use and prevalence. This is not to say that assumptions are problematic in themselves- they are essential, and no financial or economic model can reflect fully the complexities of the situations to which they are applied. One simply has to understand and accept that as a fact; and exercise the appropriate level of skepticism and judgement.

Of course, there are some outcomes which appear counter-intuitive, or downright odd. M. Pfleider uses the analogy of quantum mechanics. The crucial point here is that quantum mechanics has been demonstrated empirically to be a reality, even if it is one that many of us probably grapple with in terms of conceptualization and visualization.

At Awbury, we of course use models as part of our underwriting and analytical processes; but either we build them ourselves, or put those provided by external parties through skeptical review, based upon our long experience of financial and related models.

So, beware of models that seem too perfect, or which its creator is unwilling or unable to explain, or tries to treat you as an idiot because you are “not smart enough” to understand it. Usually, such condescension is a sign of weakness and the creator’s own uncertainties.

-The Awbury Team

Standard

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.