Executive Summary
Benchmarking is a standard tool for investors and investment professionals to evaluate the results of an investment manager. In a world of investing within asset classes and style boxes, the benchmarking process is relatively straightforward – any particular investment offering can be easily matched to an appropriate benchmark. In a world of unconstrained, “go-anywhere” style managers, though, the benchmarking process is less certain. Common methods to determine an appropriate benchmark – such as an ex-post regression of what the fund was invested in – can obscure the actions of the manager, for better or for worse. Is the only solution to simply select an arbitrary benchmark and proceed accordingly? Can we eschew a benchmark altogether?
I was interested to see a recent article highlighted in Advisor Perspectives, by Andrew Pyne of PIMCO, called, “Equity Investing: From Style Box to Global Unconstrained.” I was hoping to see if PIMCO had any new solutions to the problem of benchmarking unconstrained portfolios, but alas I found that Pyne was covering territory that has been familiar for some time. For Pyne, and for PIMCO, the question is, “should managers be style-constrained within the equity universe or should they essentially be given the flexibility to wander outside of a particular Morningstar-defined style box within the equity universe?” In concluding that managers should be given this freedom, Pyne reviews the study on “Active Share” by Cremers and Petajisto at Yale University, which found that during the past several decades portfolio managers have become less “active” and more like “closet indexers.” I devoted an entire chapter to this study in my 2009 book, “Buy and Hold is Dead (AGAIN), the Case for Active Management in Dangerous Markets.” I concluded then, as now, that the study is only interesting in the context of comparing the performance of style-constrained money managers to a single index, and has little to say about managers of portfolios that are completely unconstrained at the asset class level.
What Does It Mean To Be An Unconstrained Manager?
In PIMCO's definition of “unconstrained”, managers would be free to own LG (large growth), LV (large value), MG (mid growth), MV (mid value), SG (small growth), SV (small value), and even international equities in their pursuit of higher absolute returns. To be clear, though, PIMCO is not advocating that managers should be free to own any asset class including non-equity asset classes. Even PIMCO's “unconstrained” funds generally still must stay within their overall asset class; to do otherwise would create even more chaos than allowing so much freedom within one asset class. For example, PIMCO publishes the constraints for their Unconstrained Bond Fund. According to PIMCO, the purpose of the constraints is to make certain that the fund retains the characteristics of a bond fund while giving the manager flexibility to own any kind of fixed-income security that might add to returns.
Clearly, PIMCO hasn’t decided to put themselves out of business by creating a fund that doesn’t at least fit into a fixed-income style. After all, how would investors know what to do with such a fund? If it isn’t a fixed-income fund, then what is it, and how would you use it in the context of a diversified portfolio? Fortunately, at least the fact that the fund is still a “fixed-income fund” allows for some way to benchmark PIMCO managers to determine if they are earning positive alpha.
But when the portfolio is truly unconstrained and has no clear benchmark, how do you do the math? I know of what I speak when asking this question, because our portfolio construction at Pinnacle Advisory Group is not constrained at ANY level, including the asset class level. Instead, we are constrained by volatility, not by asset class targets, to allow our analysts freedom to select ANY asset class that they believe offers good value (and avoid any asset class that does not), in a similar manner to the recent "Same Returns, Less Risk" Wall Street Journal article. While the flexibility is very positive for allowing our analysts to do what they do best, in the process we have created a relatively new kind of investment animal that makes life difficult for everyone who wants to analyze our investment returns, regardless of whether they are a retail investor or an investment professional.
Choosing A Benchmark For Unconstrained Portfolios
Pinnacle’s not-fully-satisfactory solution to the problem of needing to benchmark our five unconstrained portfolio strategies is to throw up our hands and choose the simplest two asset class blended benchmarks we can think of, which are comprised of the S&P 500 Index and the Barclay’s Aggregate Bond Index. In a recent presentation, Carl Noble, CFA, Pinnacle’s international analyst, gave us some other examples of fund companies wallowing in this unfamiliar territory. The choice of benchmarks seems arbitrary. Notably, when utilizing benchmarks constructed of traditional asset classes like stocks and bonds, any allocation in alternative investments like hedge funds, managed futures, and private equity, or other alternative asset classes with presumably low correlations to stocks and bonds such as gold, real estate, and commodities, would constitute what we call “benchmark risk.” Benchmark risk is deciding to own asset classes not contained in your benchmark that could cause you to have relative underperformance due to low correlation to the asset classes in your benchmark. Noble’s examples included:
GMO Asset Allocation Fund: Benchmark is GMO’s Global Balanced Index of 65% MSCI All-Country World Index and 35% Barclays Aggregate Bond Index. (Note: Alternative investments were recently 16% of their total portfolio.)
Blackrock Global Allocation Fund: 36% S&P 500, 24% FTSE World ex-US, 24% BOA ML 5-YR US Treasury Bond, and 16% Citigroup Non-US Dollar World Government Bond Index
Ivy Asset Strategy: At Ivy, they apparently don’t show any custom blended benchmark for the portfolio, but instead present portfolio returns and then separately present S&P 500, BarCap US Aggregate, BarCap US T-Bill 1-3 month, and Lipper Global Flexible Portfolio returns.
I can certainly understand in the case of Ivy, why you would want to show your performance without presenting a blended benchmark. Unfortunately, though, even if you don’t offer a blended benchmark, the industry will be sure to give you one - how else can a prudent potential investor determine if the fund's management is adding value for its cost?
Finding The "Right" Performance Benchmark
Some seem to think that this difficult decision about choosing a benchmark rises to the level of a conspiracy by money managers to mislead investors into thinking they are generating excess returns (by selecting an ‘easy’ benchmark to beat) when in fact they are not (when compared to a ‘more appropriate’ benchmark), although in a word of arbitrary benchmarks the difference is not always clear. This notion of seeing the world through the lens of money managers who misuse benchmarks in order to show excess returns was reinforced to me at the recent FPA Retreat where I spoke on the topic of tactical asset allocation. Afterwards, an animated discussion ensued with a financial planner who passionately described the process of finding the correct benchmark for money managers as (my words here) correcting a fraud being perpetuated on the investing public. When asked how he would create a benchmark for Pinnacle’s completely unconstrained portfolio construction, he answered that he would probably take what we owned ex-post and then “average it out” in order to create the proper benchmark. His particular take was once the “proper” benchmark was applied, most money managers would see their alpha disappear. His job, of course, was to help his clients find the “correct” benchmark.
In fact, Solow and Kitces made this very point in an article penned for Advisor Perspectives in August 2010 titled, “When Active Management Matters.” Our article offered several criticisms of an article by Roger Ibbottson, et. al., that appeared in the Financial Analyst Journal in March/April of 2010 titled, “The Equal Importance of Asset Allocation and Active Management.” In the paper Ibbottson used performance-based style analysis to reach the conclusion that 80% of fund manager returns were due to “market” returns, and the remaining 20% of returns were split 50-50 between investment policy and active management. In our response we pointed out that for global unconstrained managers where being unconstrained meant being free to change portfolio asset allocation, as opposed to just being flexible within a single asset allocation, the ex-post regression analysis would grossly distort the analysis.
The Problem With Ex-Post Regression Analysis
While an ex-post regression analysis helps to understand what a fund has owned in an attempt to determine how to benchmark it, such a process creates significant problems when applied to an eclectic manager who is free to actually change portfolio policy on an ongoing basis. The approach can unfairly reduce the excess positive returns earned by the manager, and also potentially hide the bad decisions of a poor manager as well.
For instance, imagine a manager who owned the S&P 500 from 1996 to 1999, the Russell 2000 from 2000 to 2002, an emerging markets index from 2003 to 2008, and gold from 2009 to 2011. By picking one of the top asset classes every year for 15 years, the manager would have generated extraordinary returns. However, if the manager was benchmarked on an ex-post basis to each asset class owned in any particular year, the manager would appear to have an alpha of 0% each and every year – because the real value the manager delivered was not generating alpha within the asset class owned relative to a benchmark, but by changing the asset class (and the ex-post benchmark along with it!) at each turn of the market cycle. And the manager would be indistinguishable from a second one who owned bonds from 1996 to 1999, large cap stocks from 2000 to 2002, bonds again from 2003 to 2008, and large cap stocks again from 2009 to 2011 – even though this manager would have accumulated an incredible trail of negative returns and produced radically less wealth for clients. In both cases, if the managers owned their indices for any particular year, their alpha would be 0% - even though, clearly, when viewed across asset classes the first manager generated incredible positive alpha, while the second generated comparably incredible negative alpha.
Choosing A Benchmark In The Real World
So here we are. Pinnacle Advisory Group, a portfolio manager that is completely unconstrained by asset class targets, can choose a completely arbitrary benchmark, or we can choose to allow the industry to pick a benchmark for us, which seems to take the form of either averaging our actual allocations ex-post and assigning benchmark indices to each allocation, or using performance-based style analysis which uses mathematical regressions to determine, ex-post, what our asset allocation “should” have been. Alternatively, we might try to benchmark to the client's financial planning goals, but in practice this communicates to the client as though we have an absolute positive return benchmark (which is unrealistic and poor expectations management), and does nothing to really measure whether Pinnacle as an investment manager is adding value relative to simply choosing to index instead. In the end, none of the options seem particularly appealing; at times, it almost seems better simply not to report a benchmark at all.
Nonetheless, it is also reasonable, as a consumer of investment management, to insist on a sensible method (such as GIPS) for comparing performance results to a benchmark in order to evaluate how a particular money manager is performing. Unfortunately, in our case, when the portfolio is unconstrained by asset class targets, it seems that we must ask our clients to fully understand the limitations of the benchmarking process, in addition to fully understanding our prior investment performance in the context of a complete market cycle, and to fully understand our process for actually finding value in dangerous and volatile financial markets.
To me there are two ways to look at benchmarking. There is the asset class question which is problematic in a “go anywhere approach” except maybe to compare to peers. There are benchmarks for hedge fund returns that can do anything.
The other way to look at it is from the consumer’s perspective. I have a choice. I can give you $1 or I can put the $1 in a simple allocation strategy. Even comparing your performance net of all fees over a couple of market cycles compared to a 60/40 Broad Market Index/Barclay’s Aggregate Index gives me information.
This issue is becoming more and more problematic as time goes on, and the “sideways” secular bear market rages on. We see so many “studies” incorrectly (in my opinion) determine that “active management” cannot beat an index. Though this may be true statistically, to a degree, at the micro, security-picking level, it fails to account whatsoever with macro, asset-allocation decisions.
In my opinion, “advisors” (as opposed to fund managers of constrained, style box funds) should benchmark one of two ways: either use a “risk free return + X” philosophy (which I think has its own problems), or more appropriately, utilize a benchmark of blended indexes that would constitute a “well-balanced” portfolio using typical buy & hold / EMT strategies, for each risk level.
So for example, maybe it means for a “balanced” portfolio, it would be an appropriate mix of large/small/mid/international/EM/bonds/cash/commoditites/RE/etc.
Robert,
While I’m a general fan of the idea of “risk free return + X” as a ‘target’ return, I find it’s extremely problematic when someone seeks to use it as a benchmark for regularly (i.e., shorter-term) performance evaluation.
It’s difficult to look at a target return of “risk-free + 5%” and not just see an absolute-return-7% benchmark. As many managers have shown, it’s hard enough to succeed at absolute return when your benchmark is 0%. An absolute return benchmark of 7% would be brutal!
I think ultimately the “right” path is your second option, and it’s what we’ve ultimately concluded internally at Pinnacle – essentially, what is the passive, strategic, buy-and-hold portfolio a comparable client would have owned for the given level of [target] risk. But unfortunately now we’re talking about counterfactuals (“what would the client have owned if I wasn’t doing what I do”), which still gets us back to the unfortunately-arbitrary world.
– Michael
Excellent post!
First Eagle (another unconstrained manager) has a wonderful document titled “Avoiding the Benchmark Trap” (http://bit.ly/LlmMLF) that discusses some of this in greater detail.
I couldn’t agree more that ex-post analysis is a completely unfair and unrealistic way to analyze an unconstrained manager. Of course it strips out the alpha, as most of the alpha generated is exactly a byproduct of changing the allocation, not picking an outperforming fund / stock / index / manager.
I think that part of the problem with benchmarking is that we (collectively) spend way too much time as “long-term investors” evaluating short-term performance against arbitrary benchmarks (and having an intentional, thoughtful benchmark does not make it any less arbitrary).
Creating the perfect benchmark seems to be a bit of an odd exercise anyway, considering that for most of our clients, the purpose of investing is to protect and grow the purchasing power of their money.
I submit that if we benchmark the last decade’s performance against the S&P 500 and we slightly outperformed it, it is not a success story…we still failed the client.
Jeremy Grantham (GMO) writes often about “career risk” and that if we simply track a benchmark closely, we are going to keep our job (and our clients). Unfortunately, many unconstrained managers are unfairly “penalized” for short-term underperformance against those arbitrary benchmarks. There is a growing legion of advisors (such as the one Mr. Solow references at FPA Retreat) who make their living (and reduce their career risk) by making these inaccurate and misleading comparisons.
Because we are aiming to preserve and grow purchasing power of our clients’ assets, I prefer to benchmark long-term performance against inflation (similar to the concept of “risk-free + X”, but instead as “inflation + X”), but agree that it is a horrible benchmark in the short-term. I don’t really have an answer to solve the question of benchmarking short-term performance, outside of asking why a long-term investor would do that in the first place?
Joe,
To answer your question, I think the answer is: “Because investors paying a fee for investment advice have a right to evaluate the performance of the advisor/manager without being expected to pay the fee for an expected period of time with no accountability.”
Of course, “what is the proper time horizon for evaluating a manager or strategy” is itself a horrendously messy question (and a coming topic for a future blog post!). But I would posit there’s some difference between “short-term” (too short to measure anything useful), “intermediate-term” (long enough to measure a manager, not long enough to evaluate the success of a goal), and “long-term” (long enough to measure success relative to the goal, or perhaps long enough to reach/achieve the goal in the first place).
We have a tendency to steer clients to the long term, while they often fixate “unreasonably” on the short term. While I grant some bad stuff happens with too much short-term thinking, i have to admit that advisors implying to clients that you can’t evaluate the effectiveness of a 30-year retirement strategy until it’s been 30 years is simply not realistic either. We can’t expect clients to go that long without being accountable for results.
I think ultimately that intermediate term time horizon is probably the “right” one. How exactly you define that distinction is still a bit muddy, though.
– Michael
Fair points…and I concur about the intermediate term, as that seems (presumably) to align the closest to a full business cycle. After all, we are investing in businesses, right?
I really don’t have a good answer to the first comment. One on hand, I agree with that, but at the same time, it results in the very benchmark trap we seek to avoid.
Perhaps the answer lies in setting appropriate expectations from the outset, which is also topic for another day.
My two possibilities would be:
1) Sharpe Ratio. This would account for performance over risk-free and adding value by reducing volatility. Obviously, the next question is, to what Sharpe Ratio do you compare it? S&P 500? the Sharpe Ratio of some blended portfolio? And we’re right back where we started.
2) My “fuzzier” idea is to let individual clients choose their benchmark based on their risk tolerance and/or goals. Maybe use Morningstar’s asset allocations or iShares allocation ETFs. This directly targets the counterfactuals, but involving the client in the decision would better align with their goals while managing expectations to expect volatility.
Overall, do you see this being a standards type of issue where the only inherent problem is being unable to compare unconstrained manager A with unconstrained manager B because they are both using different custom benchmarks. Would it work if eventually, there was a universal agreement on any arbitrary benchmark?
http://xkcd.com/927/
Aaron,
Ultimately, I think for better and for worse, the benchmark has to be in the advisor’s hands. Otherwise, you end out with some very perverse results – for instance, two clients who had the same portfolio with the same investments and the same decisions, but one had more alpha than the other because of the client’s arbitrary benchmark (which may or may not have been coherently formed in the first place).
The rhetorical question becomes: How is an advisor supposed to seek performance relative to a benchmark when every client has a different benchmark for the same portfolio?
– Michael
PS- Kudos for the excellent XKCD reference. 🙂
Michael,
Maybe the answer is for the client to actually maintain some portion of the portfolio(s)in the passive, strategic asset allocation strategy and not just have it as a hypothetical benchmark. And then some portion can be managed by one or more unconstrained managers. One strategy will be core, the other(s) will be a satellite strategy. And the client will choose the percentages, in part, after reading this blog post and the accompanying comments.
If you are running an unconstrained fund, I would suggest that the benchmark should be against the “promised” yearly real-returns (after inflation) of the fund and an accepted measure of risk/return.
If you really want to compare performance to a passive index allocation, then you could do worse than to use a global market index made up of all major asset classes rebalanced every year to their initial weight, such as the one found here:
Global Market Index (GMI)
http://www.capitalspectator.com/archives/2012/01/major_asset_cla_7.html
What do you think?
Cheers,
Bruno
A true unconstrained portfolio is one in which there no constraints upon asset allocations as long as they sum to 1. The controversial thing is that they can thus have negative allocations of some assets— which you do by borrowing or shorting. The benchmark is whatever the statistical model you’re using for the allocation says it should be.
Bigeater,
Yes, but since you get to set the design for the statistical model in the first place, the benchmark rapidly gets distorted to “whatever you want it to be that makes the model look good” and two portfolios can have the same allocation but different benchmarks because they have different underlying models. Thus, the point of the post about the difficulty in benchmarking!
– Michael
Yes, the benchmark is a product of the model. But that’s a GOOD thing because the allocator only gets to set his or her benchmark once. He says: I’m going to use Allocation X of these N assets to achieve Result Y with a 90 percent confidence factor. If he doesn’t come within a reasonable distance of his predicted results or deviates from what he said he was going to do, you take your money and go elsewhere.
The problem is that everyone is asking the wrong question. External benchmarks are meaningless because every client has different money needs. The arc of my life may not correspond to the S&P500 or the price of gold, so why peg my investment performance to something that’s out of sync with my individual need for money. Maybe I’m a low-maintenance person and I want to invest my money for my great-grandchildren’s future? Or maybe I need a weekly shot of cash for trips to the track. Either way, what benchmark would help me evaluate my returns?
The elephant in the room is ethics, transparency, and honor. Will the investment manager do what he said he was going to do with your money regardless of pressures to buy or sell assets that are advantageous to the firm but detrimental to the portfolio?
My original point though is that Mr. Solow and PIMCO are not talking about a true unconstrained portfolio. “Unconstrained” doesn’t mean the guy gets to move your money willy nilly as the mood strikes him.
If you look up the term in a CFA textbook, it means that an investor can allocate positions in negative percentages through shorting or borrowing.