Executive Summary
As a growing body of research shows, our brains are not quite the logical, rational decision-making machines we think they are – or at least, wish they could be. Instead, our brains take shortcuts; we substitute easier questions for difficult ones, often without realizing it, and respond accordingly with our words and our actions. This can be especially problematic in the world of financial planning, where we often ask clients to make difficult decisions with limited information.
As a result, questions like “what is an acceptable probability of success/failure for your retirement plan?” often get switched for other questions, like “how intensely bad would you feel if your retirement plan failed?” While the questions are still similar, there is an important difference: if you have not clearly defined both the meaning of success and the meaning of failure, your clients may misjudge the intensity of the consequences, leading to an irrational and inappropriate decision about how much or how little “risk” to take.
The inspiration for today's blog post is a chapter from Daniel Kahneman's "Thinking, Fast and Slow" where he explored how our brains often figure out how to solve difficult questions by – sometimes unwittingly – substituting an easier question, and answering that instead.
Substitution Questions
For instance, if someone asks us “How popular will the president be six months from now” most of us don’t actually go through the detailed analysis to project how changing trends will impact the president’s popularity down the road. Instead, our brains will tend to take a shortcut, like looking at how popular the president is now, and simply projecting that into the future (perhaps with some very small adjustments). Similarly, if someone asks you how happy you are with your overall life right now, you’re more likely to simply reflect what your current mood is right now – it’s an easier, more readily available question your brain knows how to answer. If your mood is good, you'll probably say life is pretty good; if your mood is bad, you'll answer accordingly, too.
The research shows that for some types of questions, the easier question we substitute may simply be one that reflects the current situation or environment, such as our mood for our overall life or the current popularity of the president for an estimate in the future. With some kinds of abstract questions, however, we have to go a step further.
Relating to Intensity
It turns out that one thing our brains are especially good at is thinking in terms of intensity, and relating things on different scales to a similar one based on intensity. For example, Kahneman’s book gives the example of a situation: “Julie read fluently when she was four years old.” If a researcher were then to ask you a relative question, such as “How tall is a man who is as tall as Julie was precocious” it turns out that most people give a fairly consistent response – in this context, we relate it to some height that is as remarkable as a four-year-old who can read is also remarkable.
In fact, within a consistent cultural environment – where we all tend to relate to the context similarly – we could also answer questions like “What level of income in your profession matches Julie’s reading achievement?” or “Which crime is as severe as Julie is precocious?” or “Which graduating GPA in an Ivy League college matches Julie’s reading?” The point, simply put – we’re quite good at relating comparative intensities on different scales and translating between the two.
Combining Substitution and Intensity Questions
In some situations, our brains will attempt to answer difficult questions by using a combination of substitution, for an easier question, along with intensity to answer that question. For instance, Kahneman's book notes that when we are asked a question like “How much would you contribute to save an endangered species?” what most people really answer is the question “How much emotion do I feel when I think of dying dolphins?” We grade the answer on our emotional intensity scale, relate it to an associated dollar amount, and answer the question accordingly.
In fact, in one experiment (which has since been confirmed repeatedly) first conducted not long after the Exxon Valdez spill, researchers asked various groups of participants about their willingness to pay for nets to cover oil ponds in which migratory birds often drowned. The participants were asked to state how much they would pay to save either 2,000, 20,000, or 200,000 birds. Logically, saving 200,000 birds should be worth radically more than saving just 2,000 birds, but the researchers found in fact that the average contributions of the three groups
were $80, $78, and $88 respectively. The number of birds made remarkably little different at all. In this case, it was because the participants first substituted the question “how much would you pay to save XXX birds” for the easier question “how intensely do you feel about the image of a helpless bird drowning because its feathers are soaked in thick oil” and then related the intensity of their thoughts about a single bird into a dollar amount. While our brains are good at relating intensities, the substitution effect meant that the participants had virtually entirely disregarded the part of the question about the number of birds (without even realizing it) and the magnitude of the project and its success or failure.
Substitution and Intensity Questions in Financial Planning
Reading through this discussion on how our brains evaluate difficult questions was striking to me, because we often ask similarly challenging questions of clients as a part of the financial planning process. And given the research on how our brains think, it’s almost certain our clients engage in a similar process – which is somewhat concerning, as the consequences can lead to very distorted conclusions, such as the study participants who were only willing to pay 10% more to save 100 times as many birds.
For instance, imagine the situation where the client is asked to decide what probability of success is acceptable for his/her retirement plan. As the conversation often goes, the client is asked which plan is preferable: one that has an 85% probability of success, a 95% probability, or if the client would like to save more/spend less/retire later so that the plan can have a 99% probability.
In practice, our brains have little framework to really evaluate such probabilities; in the end, we don’t really know how to evaluate a retirement that has a 90% probability of succeeding. Instead, the research suggests that we probably substitute an easier question, such as “how intensely bad would you feel about running out of money in retirement?” Given our ease of converting intensity questions on different scales, the brain can easily answer this question, evaluating the intensity of negative feelings about the potential adverse outcome, and then converting them to a 1% - 100% scale. Clients who have intensely bad feelings about a potential retirement “failure” will give higher required probabilities of success, while clients who are less emotionally distressed at the thought will answer lower probabilities. Thus, notwithstanding the original question, clients who suggest a preferred probability of success are probably not actually indicating how much risk (of failure) they wish to expose themselves to, but instead are indicating how distressing in their minds that failure would be.
Framing the Consequences In Monte Carlo Analysis
The reason that this substitution effect matters – where clients answer the question “how much risk would you like to take” with the easier “how intensely bad would you feel if the adverse risky event happened” is that as planners, we often do a poor job of effectively defining exactly what the risky outcome would be.
For instance, think again about the scenario where a client is asked what probability of success would be preferable for a retirement plan: 85%, 95%, or 99%. In asking this question, we generally leave it up to the client to imagine what failure would look like. Without any other information, the logical conclusion – and in fact, the one sometimes implied by the planner – is that failure means a total loss of assets. Lifestyle and enjoyment ends. The family home is sold. From now on dinner is dog food, and you can never afford to see the grandchildren again.
Yet when we look at the realities of a retirement plan and how the financial planning process is executed, this is really a gross mis-statement. As discussed previously on this blog, the reality is that the probability of “failure” would be more actually characterized as a probability of adjustment instead. It represents the odds that the client would be heading down an adverse path that, through monitoring, might require a mid-course correction to get back on track. As research by Guyton in the Journal of Financial Planning has shown, “mere” spending cuts of 10% in difficult times can be effective to get clients back on track, and in fact they generally can make up the spending cuts and more in the future when the good returns come back.
So imagine a world where two clients are asked a similar but different question:
Client A: “What probability of success would be preferable for your retirement plan, 85%, 95%, or 99%?”
Client B: “What probability of success would be preferable for your retirement plan, 85%, 95%, or 99%, where a ‘failure’ means you would have to engage in a 1-5 year spending where your spending is cut by 10% until the market recovers?”
In reality, both scenarios describe the same situation, at least how it would likely play out with a planner engaging in an ongoing monitoring process with a typical client (who could intervene with a mid-course correction if the client was heading towards a danger zone). Except the reality is that because the scenarios have a very different implied outcome – client B faces “just” a potential 10% spending cut for a few years, while client A is left to his/her own imagination about how catastrophic the failure must be (given no other information) – the clients may convey very different comfort levels and risk preferences, even though it’s actually the same planning scenario, because they're expressing different intensities around what they think are different outcomes!
Which means the bottom line is that in situations where clients are invited to make a decision about how much risk to take, it is crucial to define what a risky event or an adverse outcome really means. Is it a retirement plan that requires some mid-course corrections with moderate spending cuts, or total destitution? Is it a portfolio that could experience a 20% pullback, or a 100% total loss? If the consequences aren’t defined clearly, the client at best will simply infer whatever consequence he/she thinks would be the result, judge the intensity accordingly, and make a decision about risk taking. At worst, the client infers the wrong risky outcome, leading to an entirely inappropriate conclusion about risk taking. Because like it or not, the research – as discussed in Kahneman’s book – clearly shows that this is how our brains operate.
So what do you think? Do you always clearly define the consequences of a potential risk decision? Do you frame for clients that probabilities of success are about success versus total failure? Or success versus moderate mid-course corrections? Do you think it would change their decisions about which retirement plan they choose?
I’ve never felt comfortable with Monte Carlo simulations and do not use them. I think it is a huge mistake to tell a client they have an 85% probability of success. It projects an image that there is more control than reality.
My opinion is that planners should focus more on helping clients make reasonable adjustments when things change and focus less on false assurances.
Thanks for a great comparison between using Monte Carlo analysis with that section of Kahneman’s book. I have had concerns with how advisors and clients use and interpret MC, and your comments add yet another issue I had not seen.
When using Monte Carlo, we need to make sure clients understand that all we simulate is how a fixed spending scenario survives over a certain period under a large number of varied portfolio returns. The percentage of success is a pass/fail for that test, and doesn’t show what percentage of the trials would have permitted more spending, nor what portion could have passed with moderate spending reductions (as you cite from Guyton’s article). Change any of the spending assumptions, and the MC pass rate changes.
I hope I have educated my clients that MC “results” are helpful educational aids to indicate if our current direction is acceptable, or if a course correction (saving rate, timing and size of spending) should change. Sadly, many people seem to take the mental shortcut of thinking the result is a chance of success, just like your chances of winning the lottery.
Regarding keeping track of “what percentage of the trials would have permitted more spending, nor what portion could have passed with moderate spending reductions” – good things to track, by the way – if you keep track of that information for each scenario, and have a flexible enough spreadsheet, these things and many more can indeed be tracked, arranged into distributions, and used to put the results in better context.
You are spot on about the necessity of proper framing. Monte carlo is not a financial plan. Monte carlo is merely a tool…an often misused and misunderstood tool. And I would argue that it is most effectively used as a research tool rather than something to base decisions on.
Modern financial modeling is an incredibly powerful example of how our industry takes shortcuts instead of answering the real question (at least when we do not provide the proper context and framing). This problem is extraordinarily pervasive in the investment realm, as James Montier recently pointed out in his keynote presentation at the CFA Institute Annual Meeting: http://bit.ly/Iw7KS2, and it is beginning to creep into planning as well.
As Montier explains, “mathematics is ordinarily considered as producing precise, dependable results. But in the stock market, the more elaborate and obtuse the mathematics, the more uncertain and speculative the conclusions we draw therefrom. Whenever calculus is brought in, or higher algebra, you can take it as a warning signal that the operator is trying to substitute theory for experience…If you were to give a bunch of monkeys a CAPM pricing model or a VAR risk model, you will end up creating a financial crisis,” he said. “In fact, I am pretty sure that is just what we have done.”
Human beings feel more confident when we eliminate variables and uncertainty, which is something we attempt to do through this type of analysis. However, unless we give this type of analysis its proper respect and truly recognize its limitations, we are going to do every bit as much harm as good.
Michael,
All very good points. I very much agree with you. I am in no way trying to mischaractarize our past bear markets as black swans. You mention that the bear markets were both probable and expected, and I completely agree.
I also agree that the term black swan has been abused. I think it was Taleb who coined the term and he meant it to be related to fat tails, which of course could be both bad and good, and which both occur often in the financial markets. The exceptionally good results you mention should also be construed as black swans.
My point was that from a client’s perspective I feel it tends to send a wrong message.
I’m still not sure I’d agree it sends a “wrong” message – that’s a very judgment-laden term – but certainly I’d agree, as was part of the point of the post here, that most clients do NOT have the tools and framework to evaluate what it means. We think in binary terms – the event happens, or it doesn’t. We really don’t have the equipment to think probabilistically, which means at best we have a high burden as advisors to translate it into something more relevant for making a decision.
– Michael
I guess “wrong” is not the right word. I guess my feeling is the tool is often misconstrued even with the advisor’s best intentions and efforts.
Anyway, thanks for the post. Your blog is a pleasure.
Mike,
Indeed, no question results are often misconstrued. I question whether all advisors even fully understand the implications of the Monte Carlo tools they are using, much less translate them effectively for clients.
That doesn’t necessarily make the tool bad. It is a huge concern regarding effective advice, though. :/
– Michael
I agree. When I first entered the business I was at a large brokerage firm that utilized them and the inputs ended up being widely optimistic.
It was kind of Monte Carlo simulation as a sales technique!
I do feel it can be a very good “behind the scenes” tool if used well and can help advisors in recommendations to clients.
And that, Michael, ought to be our value proposition. We spend too much time attempting to get things exactly right, more precise, etc., believing our results yield better advice and a higher degree of confidence. However, greater precision in our work often results in less accurate and less realistic results.
We absolutely have the equipment to think probabilistically, but that requires us to think. Tools can too easily become a crutch that eliminates the need to think. If we focus instead on guiding people toward decisions that yield the highest probability of success given an unquantifiable amount of uncertainty, our clients will be far better off.
Good Luck getting a retiree to actually cut spending an any year by 15%. This is why the statistical tools are very important in helping us understand what our clients can successfully spend each month and year starting from the beginning of retirement
Trey,
I think it depends on the client and the nature of their spending.
We have clients where a 5% spending cut would be traumatic. We have others where a 20% cut would be easy – they’d just trim or downscale the number of lavish cruises and vacations they take for a year or two.
I do think we could use a better framework for discussing with clients the flexibility of their spending, though. It’s not a conversation we’re typically trained to do.
– Michael
This is precisely why I have incorporated 1.)RiskalyzePro and 2.) ‘trading’ (w/ stop losses) into my practice.
I, like Khaneman, believe the most important # an advisor needs to know is the client’s “Pain #”. Knowing the point where a client’s emotional state will overrun their rational self is the goal.