Executive Summary
Monte Carlo analysis has become an increasingly popular arrow in the financial planner's quiver, as an improvement over the oversimplified traditional straight-line projection. Unfortunately, though, use of Monte Carlo analysis has begun to focus excessively on a singular probability of success, that itself can be almost as misleading as straight-line projections when not viewed in proper context. However, this is not a flaw of the Monte Carlo approach itself, but instead of the tools being used by financial planners. Instead, what's ultimately needed is software that shows not just the probability of success, but also the magnitude and consequences of failure, and a sensitivity analysis that helps clients understand the impact of the trade-off decisions they have available. What can ultimately result is a "next generation" of Monte Carlo analysis, that provides a more useful, relevant, and actionable framework to help clients make effective financial planning decisions.
The inspiration for today's blog post is a recent discussion I had with another financial planner, who was very critical of using Monte Carlo analysis, suggesting that it provides results which aren't meaningful to clients because it doesn't help them to understand trade-offs and make decisions.
"But that's not a problem with Monte Carlo analysis," I replied, "that's a problem with bad Monte Carlo software!"
Probability Of Success AND Magnitude Of Failure
It is certainly true that simply presenting a singular "probability of success" for a client's retirement plan is not a very effective tool for making a financial planning decision. Although as a methodology, analyzing a financial planning scenario on a Monte Carlo basis is at least better than simply using a straight line projection, because it accounts for the possibility of a range of outcomes and the impact of volatility and return sequencing on the results, trying to draw a conclusion from a single data point is problematic.
For instance, as I've written in the past, when viewed solely on the basis of the probability of failure, the plan with the lowest risk of failure might not even be the best choice. This is because just looking at the probability of success or failure fails to take into account the magnitude of the failures that do occur. Would you rather have scenario A, an 85% probability of success, or scenario B, a 90% probability? Clearly, the latter. But what if the 15% failure scenario A only required a 5% spending cut to get back on track, and the latter 10% failure scenario B would require a 50% spending cut, including selling your house, to get back on track? Would you still prefer B with a 10% chance of losing your house to A with a 15% chance of just losing a few nights out on the town? Suddenly, the opposite choice emerges; the scenario with the less intense failure scenario is preferable. In fact, it appears that in most situations, when clients select what probability of success they want, they're in fact expressing how intensely concerned they are about the failure scenario.
Monte Carlo Sensitivity Analysis
While better quantifying not just the probabilities of success but also the magnitude of the failures is an improvement to the Monte Carlo process, it still falls short of becoming an effective decision-making tool for clients, because ultimately know the probability of success and the magnitude of failure is still only relevant when it's compared to other scenarios with different results. In other words, it doesn't help to know that scenario A has an 85% probability of success and a 15% probability of a 5% spending cut until you also know the details of scenario B and can compare the two.
While that may be helpful in a subset of situations where a client has two specific "what-if" scenarios to compare, in the more generalized case, clients often don't realize or understand what trade-offs they have available, and the impact of those decisions. For instance, what's the benefit of saving 5% more per year? Is that better or worse than cutting spending 5% in retirement? How does that compare to retiring 1 year later? Or adding 10% in equity exposure to increase growth?
In the ideal world, Monte Carlo analysis software should provide a "sensitivity analysis" that shows the impact of various adjustments to the plan and the associated probability of success. Such an evaluation can also help the client understood how sensitive the plan is to its underlying assumptions - for instance, what happens if long-term returns are lower or inflation is higher in the future? Results could be presented in a table, as shown below:
|
Factor to Change |
Impact on Probability of Success |
|
+/- 5% in retirement spending |
9%
|
|
+/- 5% in pre-retirement savings |
3%
|
|
+/- 1 year in expected retirement date |
6%
|
|
+/- 1 year in expected mortality age |
2%
|
|
+/- 10% in equity exposure |
5%
|
|
+/- 1% in long-term expected bond return |
2%
|
|
+/- 1% in long-term expected stock return |
3%
|
|
+/- 1% in long-term expected inflation rate |
6% |
(Editor's Note: For simplicity, it is assumed for the purposes of Figure 8 that positive and negative changes to the factors have a single positive or negative impact on the probability of success; in point of fact, though, it is possible that a positive change could have a positive impact of greater magnitude than a comparable negative change, or vice versa.)
Bringing It All Together
For the typical client, who likely has little familiarity with Monte Carlo analysis, retirement planning in general, or how various retirement-related decisions can impact each other, a sensitivity analysis can become a launching point for further discussion about various trade-offs - developed specifically for the client's own individual circumstances. For some clients, adding equities might be more appealing than retiring later; for other clients, it might be preferable to save more now if they can clearly see how it impacts their ability to spend more in the future. It can be very helpful to know, per the chart above, that charts to retirement spending have 3x the impact of changes in pre-retirement saving, for a particular client scenario.
Ultimately, a combination of several factors, and/or changes to a greater degree than the sensitivity analysis shows, will still require a more detailed "what-if" scenario comparison. Going deeper into specific scenarios also helps to show the impact on not just the probability of success, but the magnitudes of failure as well. Overall, by providing a starting point for understanding what changes do and do not have the greatest impact on the plan, clients can gain a better perspective on the available trade-offs and decide what's most important to them.
But the bottom line is that while Monte Carlo analysis results are sometimes hard to connect with or act upon for clients, that may be a consequence of poor outputs, not a problem with the underlying approach itself! What's ultimately needed is a "next generation" of Monte Carlo analysis tools that provide a more useful, relevant, and actionable framework to help clients make effective financial planning decisions.
So what do you think? Would a sensitivity analysis help you to explain the consequences of trade-off decisions for clients? Do you think it would be useful for clients trying to make decisions? Does your Monte Carlo software offer this feature? Do you wish it did? Would your clients think differently about probabilities of success if we also more clearly showed the magnitude and consequences of the failures?
