Risk is Not the Science of Calculated Odds; It is an Art of Skepticism

We can measure and compute risks more accurately than any period in history. But is this a good thing for risk management? I’m not sure, and I see two separate problems.
One issue is that more information can confuse rather than inform. Just ask yourself if the risk assessments of nutrition researchers have helped. After reading contradictory reports, are you more or less confused about the impact of alcohol, eggs, or fish? It’s just hard to sort out quality information.
But more risk information brings about a second, even more dangerous issue. It’s the issue that we might forget modelling limitations. Precise predictions will make us confuse the infinite complexity of life with the finite representation of a model that’s based on a game of chance. This problem has been dubbed “the ludic fallacy” by risk practitioner Nassim Taleb (ludus means “game” in Latin).
If you really want to think about risk, get off of your computer and start arguing with a friend. You’ll start thinking about the issues that really matter.
I will now discuss three areas where it’s tempting to succumb to the ludic fallacy.
Investing
Academic finance has proposed many risk measures. Investment managers are well versed in “the Greeks”—symbols for the statistical values describing a stock’s performance.
The data are meant to inform us about how to choose investments. Nonetheless, they should not substitute for an individual investor’s consideration of risk!
I learned this during an investment seminar ten years ago. The presenter listed ten different risks for individual investors.
How many dealt with “the Greeks?” Just one—the risk of losing part of the investment.
The other nine risks were: lack of cash, inflation, deflation, taxes, societal changes, early death, unexpected disability, estate taxes, and psychological risk.
These risks are tremendously more important than the sole measurable risk of losing investment value. But we fail to consider them! Instead, we focus on the science of analyzing “the Greeks.”

Medical
There’s a trend in hospitals to use evidence-based models. These are statistical models that assess risk using decision trees and Bayesian probability.
Let me be clear. I’m all for these models—as more data is available, they will only get better. They will help correct for individual doctor biases and lead to better treatments. But I hope they will not lead doctors to overlook a true assessment of risk.
A simple example: suppose a patient in a ward is on blood pressure medication but still measures abnormally high. A doctor is faced with many possible treatments—raising a dose, introducing a new drug, or recommending a lifestyle change. An evidence-based model would be great for measuring the relative benefits and might even compute a solid course of action. But I hope the doctor will take a moment to assess the other risks before proceeding.
There are all sorts of reasons the patient could not be responding. Perhaps the patient didn’t get the right medicine to begin with. Orders can be transcribed incorrectly, nurses make mistakes, pharmacies can mess up, and patients can lie about taking medicine.
I have heard numerous stories about doctors worrying about a medicine mistake when the real issue was a practical mistake.
Medical models cannot encapsulate the practical risks of patient management–that’s largely a clinical issue. It is necessary for doctors to think about the models along with practical considerations of risk.

Buying a House
Many of my friends want to know whether it’s better to buy or rent. I often refer them to fancy calculators, like the New York Times calculator.
These models are useful, but they don’t do justice to the risks of home ownership. There are so many factors to consider—mobility, inflation, location, taxes, and maintenance—to name a few.
Almost every homeowner I know has told me some horror story about the unexpected risks of buying a house.
One person lost a career opportunity in another state since his house would not sell.
Another person encountered unexpectedly high condo association fees and it’s draining cash flow.
A third person has become paranoid about keeping the house clean, since it is his property. It’s to the point where I don’t think he enjoys living there (this is a psychological risk).
Buying a house is much more than an investment with tax considerations. Of course, the practical risks are much harder to quantify, and consequently don’t get as much attention. Finance calculators to extremely complex issues are probably not helping us make the right choice.

Conclusion
I certainly look at models to help me inform risk. If nothing else, it’s entertaining to play around with the numbers.
But foremost, I try to keep the models in perspective. They are just one of the many steps I will take to assess risk. The risks not captured in the models are often the most important ones.
I close with a skeptical passage that suggests when our Western culture started to get it all wrong:
Probability is a liberal art; it is a child of skepticism, not a tool for people with calculators on their belts to satisfy their desire to produce fancy calculations and certainties. Before Western thinking drowned in its “scientific” mentality, what is arrogantly called the Enlightenment, people prompted their brain to think—not compute.
—Nassim Taleb, The Black Swan





7 Responses to “Risk is Not the Science of Calculated Odds; It is an Art of Skepticism”
This risk issue is a huge headache for several organizations. I used to manage a “ropes Course” and the insurance to cover the risk was incredibly high. I went to Canada for a facilitator training on risk-assessment and engageded in a discussion of US insurance rates vs. Canadian rates. In Canada the rates of insurance were much lower because of the universal health care. Several of the Canadian Ropes Cours Builders insurance companies would not allow them to build in the US.
Risk is a matter of uncertainty.
By michael cardus on Apr 3, 2008
I think Presh is trying to point out that most models don’t..wait, do any models account for all 10 factors in risk? And without doing so, without doing so comprehensively (is that even possible) its foolish to be very certain about their predictions / recommendations.
By RohoMech on Apr 3, 2008
Michael Cardus: That’s a very interesting story. Pricing risk is a tough job and often depends on who ends up paying.
RohoMech: Yes, one should carry skepticism to model predictions.
By Presh Talwalkar on Apr 3, 2008
Modeling risk and recognizing risk are two different things. Decisons and their scope should be made on both the modeling aspect and the recognition of risk. You can’t let the modeling of risk issues interfere with your decision making process. Sometimes decisions can’t wait or the worse risk may be in not taking any decision at all. A good risk model should be able to quantify the recognition of a risk to a certain extent but you should not become a slave to it and let it drive your decision. We tend to like our own risk models and we forget that it is to quantify it to a certain degree of probability but not to a complete certainity.
Nice summary by Roho Mech!!
By Mahesh on Apr 4, 2008
Mahesh: Excellently stated.
By Presh Talwalkar on Apr 5, 2008
Taleb is an interesting writer, but prone to overblown statements.
Model risk is well known, especially by poker players.
Taleb is complaining about a particular model of financial risk in which risk is reduced to volatility.
To better understand the tremendous strides we have taken in understanding and modeling risk, I would recommend both Berstein’s book on Risk and Hackings’ book on the Emergence of Probability.
By michael webster on Apr 15, 2008
Michael Webster: Your observation about Taleb great…it made me laugh.
I appreciate the book rec’s to give another perspective–you are very well read
By Presh Talwalkar on Apr 15, 2008