Executive Summary
Despite the recent Nobel Prize given to its originator, the Shiller cyclically-adjusted P/E (or "CAPE") ratio continues to be controversial, driven in no small part by its current "reading" that markets are overvalued, yet they continue to climb higher and higher... a formula that has played out in the recent past as well, in both the mid-2000s and especially the late 1990s. Yet the poor recent predictive performance of Shiller CAPE shouldn't entirely be a surprise; it has never had a particularly high correlation to year-over-year market returns.
Nonetheless, the reality is that while Shiller CAPE has little predictive value in the short term, its correlation to market returns is far stronger over longer time periods; Shiller CAPE shows its strongest correlation to nominal returns over an 8-year time horizon, and is actually most predictive of real returns over an *18* year time horizon... supporting Benjamin Graham's old adage that the markets may be a voting machine in the short run, but they are ultimately a weighing machine in the long run as valuation eventually takes hold. On the other hand, over very long time horizons (e.g., 30 years) Shiller CAPE once again begins to lose its value as other longer-term structural market factors take hold.
The fact that Shiller CAPE is a strong predictor of market performance in the long run (but not the "ultra" long run, nor the short run) suggests that the valuation measure does have use, but only if applied in the correct contexts. For instance, while all this suggests that Shiller CAPE may be a poor market-timing investment indicator, clients who are retiring and exposed to "sequence-of-returns" risk over the first half of their retirement may benefit greatly by adjusting their initial spending levels in light of market valuation at the start of retirement. Similarly, those considering the benefits of delaying Social Security - or choosing to annuitize or claim pension payments over an equivalent lump sum - would do well to evaluate their decision in light of whether there is a market-valuation-based headwind or tailwind underway. Thus, even if Shiller CAPE is a poor market-timing indicator, that doesn't mean it's useless at all when it comes to retirement planning!
Short-Term Predictive Value of CAPE
Shiller CAPE - a "Cyclically-Adjusted" P/E ratio based on the current price of the market and a 10-year inflation-adjusted average of trailing earnings - has both gained in popularity and notoriety in recent years, and especially since its originator Professor Robert Shiller was a (joint) winner for last year's Nobel Prize in Economics. Shiller's Nobel Prize implicitly and explicitly acknowledged his valuation work and some of its predictive value; yet at the same time, after nearly two decades of elevated earnings and P/E ratios, punctuated by two significant bear markets (2000-2002 and 2008-2009) but also three astonishing bull markets (the late 1990s, 2003-2007, and 2009-present), many are questioning whether Shiller CAPE is really all that predictive or not at this point.
A look at the relationship between the Shiller P/E ratio - or its inverse, the Cyclically-Adjusted E/P (or CAEP) ratio - and 1-year stock market returns suggests that the critics are right about the value of CAPE, at least in the short run. There is almost no evident relationship between the E/P ratio of the market and its return over the subsequent year. The correlation of the charts below (Shiller CAEP in red, returns in blue) is a meager 0.23.
On the other hand, few advocates for long-term value investing would make the case that valuation is an especially effective predictor of short-term market performance. Deep value investors are long known for buying and holding unloved stocks for an extended period of time. In fact, it was the father of value investing himself, Benjamin Graham, who is noted for saying that the markets are like a voting machine in the short-run, and only in the long run are they a weighing machine that actually takes into account the true economic substance of a company. Which suggests that perhaps Shiller CAPE, too, deserves to be evaluated based on its performance in the long run, not the short run.
Long-Term Predictive Value of Shiller CAPE And CAEP
As Graham's statement would suggest, the predictive value of Shiller CAEP does in fact improve over longer time periods. For instance, the chart below graphs the Shiller CAEP (red line) and the subsequent (nominal) annualized return of the markets over the next 10 years (blue line). Although there are still some significant gaps, over this time horizon the correlation more than doubles to 0.53 (compared to the 1-yr return correlation). (Note: Given the ratio is E/P instead of P/E, a high E/P indicates the market is favorably valued and should align with higher returns, and vice versa.)
On the other hand, while the predictive value of CAEP improves over longer time periods, the reality is that there is such thing as being "too long term" to use Shiller CAEP to predict returns. Accordingly, the chart below graphs Shiller CAEP vs 30-year returns, where the predictive value is actually even worse than on a 1-year basis (a correlation of only 0.19).
Of course, over time horizons this long, inflation is also a significant factor that can distort returns; in fact, Shiller acknowledged the impact of inflation in calculating the CAPE ratio in the first place (as the trailing 10-year earnings are adjusted for inflation). When the Shiller CAEP is measured against 30-year real returns, the predictive value improves notably - the correlation rises to 0.50. However, significant deviations are present, even at extremes; note that, as the Philosophical Economics blog also recently pointed out, the 30-year real return is almost exactly the same in 1929 and 1981 (indicated with red arrows), despite the fact that the Shiller CAEP was at opposite extremes during these time periods (marked with purple arrows)!
To explore the variability of the predictive value of Shiller CAEP, the chart below graphs the correlation between CAEP and nominal or real returns over varying time horizons. As the chart reveals, CAEP is most predictive of nominal returns over an intermediate term time horizon (peaking out over an 8-year time horizon). Over a similar (or shorter) time horizon, CAEP is actually less predictive of real returns, although over longer time horizons CAEP actually becomes more predictive, ultimately peaking out with the maximal correlation and predictive value over an 18-year time horizon!
Relevance Of Shiller CAPE For Financial Planning Decisions
Given the "goldilocks" predictive value of Shiller CAPE ratios - it doesn't work over time periods that are too short, nor those that are too long, but just the ones right in the middle - the results suggest it's especially important to make the predictive time horizon of market valuation to planning decisions that have relevance over that time period.
Accordingly, the results suggest that using Shiller P/E valuations as a "market timing" indicator will fare quite poorly; there is very little relationship at all between market valuation and short-term market results, as evidenced by both the historical data, and the experience of any investor over the past two decades and the astonishing breadth of bull and bear markets that have occurred over the time period. On the other hand, while P/E ratios appear to have far more predictive value over longer time periods - especially 10-20 year real returns - the sad reality is that most investors simply do not give their advisors time horizons that long before judging them for results. In other words, from a practical perspective it may not matter if market valuation is right "in the long run" if you're at risk of being fired for short-term relative underperformance in the meantime (e.g., for getting conservative in overvalued markets and then being forced to wait years before the markets "prove you right" as would have occurred for the conservative investor in the mid-2000s or especially the late 1990s).
Nonetheless, this doesn't mean that Shiller CAPE has no relevance for financial planning decisions. For instance, prior research from the May 2008 issue of "The Kitces Report" has shown that Shiller CAPE ratios have an astonishingly strong -0.74 correlation to safe withdrawal rates and can help predict a reasonable starting point for retirement spending; because the long-term sustainability of retirement spending is most sensitive to an unfavorable sequence of returns in the first half of retirement, Shiller CAPE's predictive value aligns quite well and helps to provide valuable insight about whether the prospective retiree faces an important headwind or tailwind in the early years of retirement. Notably, the results indicate Shiller CAPE is more correlated to safe withdrawal rates than it is to market returns themselves!
Information about long-term returns, and whether the investor likely faces a valuation-based headwind or tailwind, can also have implications for the discount/growth rates used to evaluate other strategies as well. For instance, the relative value of delaying Social Security is impacted by the assumption regarding real returns for the available portfolio (as there's less value to delaying if you could earn more with your money by taking benefits earlier, and conversely there's more value to delaying if the opportunity cost of your investment portfolio was limited anyway). Similarly, this would imply that the implicit market-hedging value of delaying Social Security is more relevant when Shiller CAPE is high (and there is a greater risk of substandard returns) than when it is low. More generally, higher or lower expected returns - as adjusted for market valuation - would have implications for the relative value of any lump-sum-versus-annuity decisions (whether considering the outright purchase of an immediate annuity, or evaluating a pension-versus-lump-sum scenario).
The bottom line, though, is simply this: while the data does suggest that market valuation tools like Shiller CAPE are a poor predictor of short-term market performance, and may be very limited as a market timing tool to improve performance, the longer-term predictive value of Shiller CAPE and its CAEP inverse suggest that it is still relevant for planning decisions where the focal point truly is on long-term returns, from setting an appropriate safe withdrawal rate (or possibly even an optimal asset allocation glidepath) to evaluating the opportunity cost of funds for lifetime income strategies like delaying Social Security, purchasing an annuity, or considering a pension lump sum!
In point of fact, perhaps this distinction between valuation's long-run-but-not-short-run predictive value was the very reason the Nobel Prize committee gave its award jointly to Shiller for his work on how stock returns can be predicted in the long run, while simultaneously giving it to Eugene Fama for his efficient markets research showing that stock returns cannot be effectively predicted in the short run!
What do you think? Do you find Shiller CAPE ratios to be relevant in your practice? Do you use it as an investment tool? Do you use it in other contexts like determining sustainable retirement spending or the value of delaying Social Security?
Wonderful food for thought. I’ve added this to the CAPE Research Catalog at http://www.pe10ratio.com.
Kay
Thanks Kay, I hope it’s helpful food for thought! 🙂
– Michael
As an extension of the annuity decisions thought process, you could add in decisions regarding variable annuites with living benefits. Could be more attractive to pay more for these guarantees with high CAPE’s.
Omar,
Actually, it’s kind of the opposite. Variable annuity income guarantee costs are so high that with a low return environment, the “floor with upside” turns out to be “floor as a best case scenario”. And as a floor with no upside, the variable annuity income guarantees are not as compelling as some of the alternatives out there. See http://www.kitces.com/blog/do-todays-guaranteed-living-benefit-annuity-riders-really-offer-enough-to-be-worthwhile/
– Michael
That is not always the case, though it may be true today. Back in the mid 2000’s, costs were lower and benefits were higher with CAPE’s still being high. Other factors, like interest rates, and “once bitten twice shy” of insurers have had a huge impact on VA riders. Additionally, at one point, some VA companies offered 5% plus CPI. Point taken, though, there are many factors involved here!
Omar,
I never said this was always the case. The discussion was about what retirees could/should do given the reality they face today. Retirees can’t go backwards and buy today the annuity guarantees that existed 10+ years ago.
As I’ve written previously, though, many of the prior contract guarantees were so compelling that advisors should be VERY cautious about surrendering existing contracts with living benefit riders. See http://www.kitces.com/blog/strategies-for-existing-variable-annuities-with-glwb-or-gmib-riders/
Respectfully,
– Michael
Michael, I am concerned about the US market being a little too hot and was studying these rates earlier today while I am also reading / scanning Asset Allocation by Roger Gibson (c 2013), Intelligent Asset Allocator B Bernstein (c 2001) and Reducing The Risk of Black Swans L Swedroe and K Grogan (c 2014) for the past several weeks. (I ordered BB’s latest book Rational Expectations today too.) Above you comment about our clients sometimes not having incredible staying power and I agree. I support the client’s desires for their Investment Policy Decisions! However, because of the market heat I have been thinking about my thoughts / spin on potentially reducing current Stock Allocations. The Shiller CAPE primary focus is on US Stocks – but if they dive we have seen evidence the global market will likely slide as well. I have started to look for slight new wisdom about tactical asset allocation. I am not a fan of market timing. At times it is hard enough for some to keep our allocations on target; think 2008 for example. In addition there is not enough time for some of us to recover from another lost decade. To sum up – Yes, current pricing should play a (material) role in asset allocations, additions, rebalancing, sustainabllity and withdrawals. We should be cognizant of the price of tomatoes (B Bernstein). Thanks for providing these high quality products for our consideration and reflection!
Great thoughs Michael.
It’s always a challenge to pick what news sources to read regularly, and
getting harder with so many sources of info happy to push updates our way. Investment
News is particularly prolific though often pretty good quality. I’m happy to
have added your blog to my list of items I always try to read. I’m sure you’ve
given a recommended reading list, can you give me an idea of where to find it.
Jeff
Thanks Jeff! I hope it’s helpful!
– Michael
Thanks for writing the fine article, Michael. It is important and brave stuff.
I don’t agree when you suggest in the headline that P/E10 is not a good tool for long-term market timing. I certainly agree that short-term timing doesn’t work. But there is now 33 years of peer-reviewed research showing that long-term timing ALWAYS works and it is important for people to know that.
I understand why you don’t want to say openly that long-term timing always works. It upsets Buy-and-Holders when they hear that. But that is the case and it is a very, very big deal. The research that I did with Wade Pfau shows that investors can reduce the risk of stock investing by nearly 70 percent by giving up on Buy-and-Hold strategies and engaging in long-term timing instead. That’s a very big deal indeed.
Rob
I just feel that when we start to think the long term stock market can be “predicted,” that is when we set ourselves up for failure.
The very worst thing you can do when working on someone’s retirement predictions (IMHO) is to use averages, ten years such as the CAPE, or average market returns for that matter.
Also the fact that this article states this method is not good for longer term predictions such as 30 to 40 years makes it doubly dangerous. Once you go into retirement, there are usually no “do-overs 10 years down the road – you have to know you are good for 30-40 years and you can’t do this with the CAPE.
Dave,
If you feel that average market returns are inappropriate, and adjustments like CAPE to average market returns are also inappropriate, what exactly are you suggesting people do? Enter retirement utterly and completely blind with NO assumption of any form of returns at all?? Just randomly guess at how much money they need, since there’s no associated return assumption at all?
That aside, the article never stated that CAPE has NOTHING to say about 30-40 year returns. Just that it’s LESS predictive than it is at shorter time periods. CAPE is still predicting 25% of the variability in 30-year returns by the data presented here, which is still WAY more information than a blind guess.
– Michael
Thanks Michael. I have a related question about an interesting article you wrote in the Journal of Financial Planning, titled “Improving Risk-Adjusted Returns Using Market-Valuation-Based Tactical Asset Allocation Strategies”, I’m not sure of publication date. ( http://www.onefpa.org/journal/Pages/Improving%20Risk-Adjusted%20Returns%20Using%20Market-Valuation-Based%20Tactical%20Asset%20Allocation%20Strategies.aspx )
If I recall, you looked at a very moderate tactical approach where one would hold their personal “normal” proportion of equity to fixed-income for the vast majority of the time when Shiller PE’s are between their historical 10th percentile and 90th percentile, i.e. the middle 80%. If Shiller PE went above 90th percentile, one would reduce equity by 10%, and if it went below 10th percentile, increase equity by 10%.
Do you believe that the results and exec summary from that article hold?
I won’t try to summarize the results (readers can refer to the article), but here was the exec summary for reference:
1. Studies examining the value of active management strategies tend to analyze performance within asset classes against narrowly defined benchmarks; there is little research analyzing tactical asset allocation strategies that change allocations among asset classes, rather than within them.
2. The distribution of expected returns and volatility are statistically significantly different at valuation extremes than they are from the general distribution of returns. As a result, the efficient frontier itself can shift because of varying capital market assumptions across different valuation environments, which in turn implies that asset allocations should change as market valuations change.
3. A basic market-valuation-based tactical asset allocation strategy that underweights equities (relative to bonds) in overvalued environments, and overweights equities in favorably valued environments, can lead to higher returns and improved risk-adjusted returns.
4. The results for improvements in return and risk-adjusted returns hold up on an ex ante analysis and a historical analysis.
5. The improved results—comparable to the value of rebalancing—are sustained even when accounting for reasonable tax assumptions, in large part because of the relatively low turnover necessary to achieve improvements through basic tactical asset allocation strategies.
Many thanks.
Thank you for a great exposition of the Shiller PE and its value. Since it is calculated with current price in the numerator with ten years of adjusted earnings in the denominator, it is not surprising that it does not have immediate predictive value.
It is also easy to see that it would not really help with baskets of growth stocks including such names as NFLX, AMZN since it doesn’t treat rapidly changing earnings situations. I would suspect this would apply to emerging market indices as well.
I wonder if a shorter term earnings figure could yield better short term results.
In any case, if we believe Shiller PE predictability over 8 years, could it possibly be used to guide investment in index funds in the shorter term if their prices are higher or lower than the expected trajectory in the long run?
We can’t rely on a number like this as the media does while checking our brains at the door!
Great article!! Are you aware of a study that did the same analysis (looking at the correlation between Shiller P/Es and short/medium/long-term returns), but using developed market stocks and emerging market stocks?
Alvin,
You might check out Meb Faber’s work on this. See http://papers.ssrn.com/sol3/papers.cfm?abstract_id=962461 and some of his prior SSRN articles as well. Also more info at http://www.mebfaber.com.
– Michael
Thanks so much Michael! I’m really glad the financial planning field has you.
Michael,
Curious if you’ve reviewed Jeremy Siegal’s latest paper in the FAJ on the CAPE. He suggests that today’s seeminlgy elevated CAPE is due in large part to GAAP changes, and adjusting for this results in a CAPE that is today very modestly above historic levels (rather than seemingly WAY above hist levels). Any flaw in his logic that you can detect?
Many thanks for the gift of your writing!
Best,
Frank
Fascinating subject! I was interested in calculating the combined effects of P/E ratios, interests rates, and inflation rates on subsequent market returns, since interest rates and inflation affect business growth, and accordingly, how much investors are willing to pay for stocks. I experimented with different variables backtested to 1972, and I found that when adding together the CAPE, the Federal Funds Rate, and the CPI (which together I’ll call the “indicator total”):
-At the beginning of each year, if the indicator total was 39 or lower, then for the following 10 year period, a portfolio of 60% stocks/40% bonds outperformed a 40% stocks/60% bonds portfolio 91% of the time.
-If the indicator total was 40 or higher, over the next 10 years a 40/60 portfolio performed better than (or in one case, the same as) a 60/40 portfolio 100% of the time.
For the calculations, I used the U.S. total stock market index and U.S. intermediate-term treasury bonds. For performance, I used the compound annual growth rate.
Any thoughts on the validity of this approach?
One can see the results of market timing using the Shiller PE ratio compared with a static asset allocation in portfoliovisualizer.com.
The main premise of the market timing model is the following:
PE10 >= 22: 40% stocks/60% bonds
14<= PE10 < 22: 60% stocks/40% bonds
PE10 < 14: 80% stocks/20% bonds
The PE ratio and asset allocation is determined at the beginning of the calendar year and held for the entire calendar year. The portfolio visualizer data goes back to 1972 for some asset classes and back to 1985 for securities with ticker symbols.
As the author has noted it may be a good measure for retirement planning as one is more concerned with sequence of return risk than capturing a few bps of return. The model seems to flash the alert early, which helps to reduce drawdowns and calendar years with negative returns.