| |
 May 2007 You Be the Judge A Review of P3 and P4's Performance Metrics
It’s not the facts that are different, it’s the perceptions. Although investors like to think they always make well reasoned decisions, studies in behavioral finance suggest otherwise. Many of our choices are based on our own perceptual biases
In a way, that’s probably a fortunate thing since for every buyer, there has to be a seller. If everyone always agreed on the facts, there’d be a lot less trading.
Quants are generally perceived to be more objective than most. After all, they’re relying heavily on statistics and data, not subjective bottom up stock picking. Even so, there’s no way to escape the impact of perception. Quantitative Portfolios 3 and 4 offer a good example.
Simple Question Both models were based on market data from the decade of the 1990s so understandably, both ended up with a growth tilt. That was fine when they were initially launched in July 2000, but it quickly started to work against them when the market – and particularly growth stocks – swooned. Within a year of their launch, P3 and P4 were both well underwater. In fact, through the end of April 2007, they’re still more than 10% below their July 2000 starting point. The S&P didn’t fare much better, but it was able to scratch out a 2% gain over the same period. This would lead one to conclude that P3 and P4 have failed in their objective of tracking and exceeding the S&P 500.
Or maybe not. On a cumulative basis, (Chart 1) they’re certainly well below the index, but is that the real measure of success? Here’s where perception comes into play and here’s where you can be the judge. Your challenge is to decide of the models have lived up to their goals. This is really a simple question, but as you’ll find out, the answer doesn’t come so easily. The facts, however, are there for all to see.
Tracking First consider tracking. What does it mean to “track” the index? Generally you think of this in terms of matching or following. A traditional statistical measure is correlation. Two series (in this case the models and the index) are said to be highly correlated if they tend to move in the same direction with similar magnitude. They’re said to be negatively correlated if they move in opposite directions with similar magnitude. Finally, they’re uncorrelated if their movements show no relation to one another. Correlation can be measured statistically with values running from +1 (complete positive correlation) to -1 (complete negative correlation). A value of 0 would indicate completely uncorrelated movement. Most observed values fall between the extremes showing degrees of positive or negative correlation. Just looking at Chart 1, it’s pretty obvious P3 and P4 tend to trade like the S&P 500. The paths are similar, but with the exception of a few days at the outset, P3 and P4 are always lower than the index. Given that, you might conclude that their direction is similar but the magnitude of change is considerably different. But look again: Most of the difference in price levels occurs in early 2001 when the models fell much harder than the overall index. Since then, the distance between them has remained relatively unchanged. Did the models just get off to a bad start? The statistical correlation coefficients suggest they did. For the period July 1, 2000 through April 30, 2007, the correlation with the S&P 500 for P3 and P4 is .857 and .919, respectively. That’s certainly not perfect positive correlation, but it’s still quite high for actual market observations. This suggests that despite the difference in cumulative returns, the models have actually done a good job tracking the S&P 500. Even if you don’t agree after seeing the chart patterns and the correlation coefficients, you have to admit the answer’s probably not a clear cut as you originally thought.
Climbing the Mountain Actually, it all depends on what it means to “exceed” the index. No, this isn’t an attempt to sound like Bill Clinton, it’s a legitimate issue.
Obviously if you define “exceed the index” as only applying to the July 2000 – April 2007 period, the models have been utter failures. Neither has come close. But this is where mountain charts fail to tell the entire story. Usually they tend to make results look rosier than they really are, but the exact opposite is true for P3 and P4. A mountain chart is nothing more than a cumulative return chart. You frequently see them in mutual fund ads touting the “long-term performance” of the funds. Invariably they depict the fund moving rapidly ahead of whatever is used as the benchmark and then maintaining the lead as time passes. Here’s a little secret: If the chart didn’t look like this formidable mountain (hence the name), the fund company wouldn’t be using it in its advertisement. Mountain charts look great if the first few periods allow the fund (or stock, or index, or whatever) to establish that big lead because it will mask subsequent periods of underperformance. On the other hand and for the very same reason, an initial period of underperformance is extremely difficult to overcome. That’s precisely the problem confronting P3 and P4 and, by the way, the reason you’d never see it in a mutual fund ad for a fund with the misfortune of launching right before a 3-year bear market. The bottom line is this: A mountain chart is heavily dependent on what occurred in the initial period. Indeed, if a different starting date was selected for Chart 1, the results would look dramatically different. Consider Charts 2 and 3 that show cumulative returns for P3 and P4, respectively, using start dates of July 1 for the years 2000-2006. Clearly the start date makes quite a difference.
So maybe a cumulative mountain chart isn’t the best – or at least only way to compare the models’ returns to the index. If you think about it, that’s not the only way you evaluate stock or fund returns, either. Instead, you probably also look at periodic returns (e.g. 1, 3, 5-years), batting average, up and down capture ratio, etc. Why not do that here, too?
Exceeding Periodic returns are another issue. The 1, 3, and 5-year returns appear in the top part of Chart 4. Both P3 and P4 have done well in the 3 and 5-year periods while lagging for 1-year. From this the conclusion would seem to be that they have managed to exceed the index in the long-term although they are presently failing to do so in the short-term. That’s far from a complete failure, is it? In fact, almost no one would expect them to exceed the index in every period. It’s more realistic to expect consistent performance across periods. That’s where a measure like batting average comes in Batting average is essentially just like its baseball namesake. It divides the number of times the model exceeds its benchmark (the number of hits) by the number of measurement periods (number of at-bats). Just as a ballplayer’s batting average doesn’t distinguish between singles and homeruns, an investment’s batting average doesn’t account for magnitudes, either. All it measures is the percentage of time the benchmark was exceeded. We calculated the monthly batting average vs. the S&P 500 for P3 and P4 for the period July 2000 through April 2007. No, they didn’t beat the index in every period, but the results were still better than Ted Williams' mark, .524 and .585, respectively. While it would be nice if the marks were higher, they did "beat" the index in more than half the months. Does this indicate they successfully “exceeded” the index? You decide.
Bringing It Home Once again it’s one of those situations where you win some and you lose some. As you’d expect, the quant models tend to do better when the market is headed up (high beta) and when growth is in favor (that growth bias again). P3 only "beats" the index in 2003 and 2005 while P4 surpasses it in 2002, 2003, 2004, and 2005. P3 is on top in only 2 of 7 years, but P4 exceeds the index in more than half (4 of 7). Does this meet the definition of "exceed"? So there you have it. There, in five charts are all the facts you need to evaluate P3 and P4. Did they track and exceed the benchmark index or are they failures? The facts themselves won’t decide this. You’ll have to. Search this site! Just enter you key word or words: Get current quotes or follow your own custom portfolio,
courtesy of E-Line Financials:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ACTS CAN BE FUNNY THINGS, especially in investing. Two people can look at the same data and come away with completely different opinions. Oddly enough, the facts may support them both.
