| |
 May 2010 Good Statistics, Bad Results Even Normally Reliable Measures Can Let You Down
Nevertheless, there are times when even the most respected statistics can be misleading. Reliable friends like standard deviation, percentile rankings, and even the Sharpe Ratio can send misleading signals. It's not often that they go bad, but when they do, they can be just as misleading as the worst pro-forma earnings report.
What causes good statistics to suddenly go bad? Fortunately, it's nothing inherent with the statistics themselves but rather the circumstances. As an investor, analyst, or advisor, it's incumbent upon you to recognize these situations and act accordingly. In most instances, these are times when quantitative analysis alone is not sufficient. You've got to look beyond the numbers and truly understand what they're telling you.
Risk's Volatility Standard deviation is perhaps best known for its role in Modern Portfolio Theory which holds risky assets can be combined together to increase portfolio return while controlling overall risk. Standard deviation is typically used as the risk measure when creating efficient portfolios.
But underlying the theory is the assumption that standard deviations remain relatively constant over the investment time horizon. That sounds reasonable, but often it's simply not true. One could plausibly argue that it's not reasonable to expect risk to remain even relatively constant among individual securities. Given that they're continually buffeted by non-market risks, no one would really expect their overall risk -- their standard deviation -- to remain anywhere near constant. While that's probably true for individual securities, it's also true for diversified portfolios as well. Consider, for example three popular mutual funds: Growth Fund of America A, Calamos Growth A, and Fidelity Magellan. Morningstar classifies all three as Large Cap Growth funds. All are popular among both advisors and individual investors, and all three have been media darlings for outstanding short-term performance over the past decade or so. All three fish in the same pond of stocks -- large cap growth -- for their holdings. All have over eight billion dollars under management, with none being super-agile start-ups. With such similarities one might expect them not only to offer similar returns, but risk levels as well. As it turns out, however, nothing could be further from the truth. Chart 1 shows their 5-year standard deviations relative to their style benchmark, the Russell 1000 Growth Index. If their standard deviations stayed fairly constant over the measurement periods, it would hover around the 0 level -- essentially tracking that of the benchmark index. Growth Fund of America (blue line on Chart 1) actually does a fairly good job of this. But notice the fluctuations of the other standard deviations. Not only do they show quite a variation, to a great extent, they move inversely with one another. This is particularly noticeable in the periods ending in 2003, 2004, and 2005. The range of variation around the benchmark risk is especially remarkable for Calamos which moves from a high of almost 20 percent over to a low of 2 percent under in just one year. Analysts screening for funds with low levels of risk (or simply risk near that of the benchmark index) would get vastly different results with Calamos Growth and Fidelity Magellan on a year-to-year basis. Portfolios built using Modern Portfolio Theory and using these funds would have little real reason to trade as predicted. The volatility of risk from one period to another is roughly equivalent to the first derivative of risk. Every investable asset has a certain degree of risk, but the volatility of that risk is also a critical element in assessing its value and use in a diversified portfolio. When standard deviations themselves have a high standard deviation, their value to long-term investing is greatly diminished. Analysts who fail to realize this are doomed to learn from it in years like 2008.
The Cream of the Crap Instead, individual stocks and even broad areas of the market trade differently. When Consumer Staples are down, Tech stocks may be up, or vice-versa. While stocks within a given sector don't always trade in tandem, they often trade alike relative to the broad market. When the Tech sector is down, most stocks that live there are, too -- that's why the sector is off.
This isn't a difficult concept, but many analysts seem to forget it. Here's how it happens: With diversification being so revered, even the most sophisticated investors require their investments to be spread not only over individual securities, but across sectors as well. It's not uncommon for investment policy statements to require portfolios to hold stocks from all ten S&P 500 sectors regardless of current market conditions. Managers may have leeway to tilt the portfolio towards the top performing areas but may still be required to at least have some exposure in the laggards. Not only does this increase diversification, it also reduces the risks inherent to market timing. The theory sounds great, but it begins to break down when the portfolio manager attempts to find the best representatives of each sector. It's not a problem in hot sectors because just about every stock does well there, but it is an issue for the poorly performing ones. Many investment policy statements require the manager to only purchase (or hold) stocks from the top one or two quartiles (25 percent groupings) of the sector. This is an attempt to assure that holdings are at least above average if not the best performers. However, when the entire sector is out of favor, this assures virtually nothing, not even better performance than the appropriate unmanaged benchmark index. In this instance, diversification is no remedy. Again consider an example of mutual funds - broadly diversified portfolios. As you'll notice from Chart 2, the last funds in the top quartile (the 25th percentile) of Mid Cap Value Funds trailed the benchmark Russell Mid Cap Value Index for three consecutive years, 2004 - 2006. In other words, not all funds in the top 25 percent of the sector beat the unmanaged index for any of the three years running. Although you'd probably feel safe picking "winners" from the top quarter of all funds in the index, you still had a pretty good chance of ending up with a loser. Interestingly, this happens when the entire sector tends to be up, not down. Active managers find the benchmark much easier to beat when it falls -- particularly when it falls as precipitously as it did in 2008 -- than when it's up. Why? Because when stocks are falling, all they have to do is hold a little cash in the portfolio and presto, they outperform. When the sector on the whole is on the rise, it takes a much more astute manager to build a better portfolio than the index itself. As we've pointed out before (here and here), mid caps have been the top performing capitalization over the past decade. When all managers have a hard time beating the unmanaged index, even the best is an underperformer. Investment policies limiting managers to only the top percentiles of the sector may seek to assure that only the best investments make their way into the portfolio, but when it's a hot sector, they only get the best of the bad.
Not so Sharpe The Sharpe Ratio is one of the simplest and most widely utilized measures of risk-adjusted return. Calculation is simple, just think about what's being measured: Return per unit of risk. As discussed above, standard deviation is a widely accepted measure of an asset's overall risk, so it's reasonable that it serve as the divisor, the measure of risk. The numerator in the ratio is return, but not just overall return because at least a small portion of that could be obtained with little or no risk by investing in the so-called "risk free asset". Because an investor can get this return (generally measured by the yield of the 90-day Treasury Bill) with virtually no risk, this part of an assets total return should be excluded when calculating its return per unit of risk.
As a result, the Sharpe Ratio is expressed by the simple expression: (Asset Total Return - Return of Risk-Free Rate)/(Asset Standard Deviation) This ratio is not only elegant, it makes a lot of sense. Think back to those fractions you hated in third grade. Remember the effects of changes in the numerator and denominator? In the case of the Sharpe Ratio, the result increases if (1) The asset's total return is larger, (2) its standard deviation is smaller, or (3) both. The ratio declines when the opposite occurs. It all makes perfect sense -- unless the ratio is negative. Generally the Sharpe Ratio is positive, but when assets have been in decline and longer term total returns are negative (as many are now after 2008), the ratio goes negative. When it does, those third grade relationships between numerator and denominator actually work against it. To see why, consider the following example illustrated in Chart 3. First consider Funds A and B. Over the past five years, A has relatively "better" return, losing four percent as opposed to B's loss of six. Fund A also has lower standard deviation yet look at the Sharpe Ratios: -0.6% for A and -0.4% for B. In this instance, B's higher standard deviation actually worked to reduce it's negative Sharpe Ratio making it appear to provide better risk-adjusted return -- but in reality it really didn't. Some analysts adjust for this by considering the absolute value of the Sharpe Ratio when returns are negative. This is shown in the final column of Chart 3. (As you probably already know, the absolute value is the distance from zero -- either positive or negative -- of the ratio's value. As a result, the absolute value is always positive.) In this case, the absolute value of the Sharpe Ratios for Funds A and B (final column in Chart 3) give a more appropriate relative order. Fund A's value is higher than B's. But in general practice, this is not an adequate solution. To see why, consider Fund C which actually had a positive 5-year return and the same standard deviation as Fund A. Its Sharpe Ratio conveys its superior risk-adjusted return relative to A and B. However, when using the absolute value of the Sharpe Ratio, it trails both. Even when the majority of total returns are negative, there are generally a few funds that do post positive results. Because of this mixed performance, the absolute value of the Sharpe Ratio can be just as misleading as the unadjusted ratio itself. In this case, the best solution is to avoid using the Sharpe Ratio at all. With so many investment alternatives, a good quantitative screen or factor model is often a good way to narrow the universe to focus on the options with the most promising characteristics. Unfortunately, as the examples above illustrate, the results can at times be critically misleading. No matter how well thought out or constructed the quantitative process, it should not be relied upon in a vacuum. Instead, the analyst should always view the results in the context of previous market conditions in order to weigh the validity of the statistics themselves. Security evaluation processes should be as consistent as possible, but that doesn't mean they should not be adaptable when necessary. Getting no results is preferable to getting misleading ones. Search this site! Just enter you key word or words:
Get current quotes or follow your own custom portfolio,
courtesy of E-Line Financials:
|
|||||||||||||||||||||||||
FTER THE TECH BUBBLE AND ALL that followed, virtually no one follows the tout sheets also known as "analyst research". Market gurus come and go, but they, too, have proved to be remarkably fallible -- and often at the worst possible times. Many investors are now more circumspect, preferring to do their own analysis. Despite all the fancy alternatives out there, there's still a lot to be said for the good ol' dividend discount model.
