By Anne Pycha
“Basketball’s decision-makers, it seems, are simply irrational,” wrote Malcolm Gladwell in 2006. On the eve of March Madness 2011, his words have taken on new meaning, thanks to a recent study conducted at Virginia Tech.
Each spring, 68 basketball teams participate in the NCAA Division I Men’s Basketball Tournament, also known as March Madness. Of these teams, 31 qualify automatically by winning their division championship. Remaining teams can participate only if they get hand-selected by an NCAA committee. This creates some drama and hand-wringing, especially for the “bubble” teams whose season performance might or might not warrant an invitaiton. The committee, which will announce its decisions this year on Sunday, March 13, consists of approximately ten athletic directors and sports commissioners from around the country. (That this year marks some changes; prior to 2011, only 64 teams participated in the tournament).
Virginia Tech professors Leanna House and Scotland Leman examined the decisions made by this committee from the 1993-1994 season to the present. Using Bayesian statistical methods, they found that a team’s Ratings Percentage Index (RPI) correlated strongly with an invitation to the tournament. The RPI includes three basic components: how often the team won during the season, how often their opponents won, and how often their opponents’ opponents won.
Sounds rational so far. But the RPI does not tell the entire story. House and Leman’s technical report suggests that “the committee may have a cognitive weighting scheme which does not agree perfectly with the RPI formula.” In other words, the decision-makers leave themselves some wiggle room.
So House and Leman examined an additional factor: marquee. A marquee team has a history of top performance in the tournament. The North Carolina Tar Heels offer a good example. In 1982, a player named Michael Jordan helped this team win the tournament championship. In subsequent years, the team won the championship several more times, most recently in 2009.
House and Leman’s analysis showed that, given two teams with roughly equivalent RPIs, the committee was historically more likely to invite the marquee team. In 2000, for example, the Tar Heels finished the season with 31 wins and 21 losses. On the basis of their RPI alone, they had a 32% probability of being invited to the tournament. But the team’s marquee status boosted that probability to 85%. The committee did, in fact, invite them that year.
Non-marquee teams suffer the consequences of this decision-making. In 2010, the Virginia Tech Hokies finished the season with 33 wins and 13 losses. On the basis of their RPI, the Hokies had a 31% probability of being invited. Marquee status, if they had possessed it, would have boosted that probability to 83%. The committee did not invite them.
Virginia Tech coach Seth Greenberg raised the issue of marquee status out loud last year, providing House and Leman with the inspiration for their study. In March 2010, both the Hokies (33 wins, 13 losses) and the Tar Heels (20 wins, 17 losses) failed to receive an invitation to the tournament. Just afterwards, as reported on Techsideline.com, Greenberg asked: If North Carolina had Virginia Tech’s exact same resume, would they have been in the NCAA Tournament? The answer, it appears, is yes.
Making good decisions
So, is Malcolm Gladwell right? To some extent, the answer depends upon what we mean by a “rational” decision.
Gladwell’s 2006 article on basketball’s decision-makers, entitled “Game Theory” and published in The New Yorker, focused on the problem of ranking individual players. In it, Gladwell sang the praises of comprehensive rankings. Basketball games move so quickly and involve so many moves that observers simply cannot absorb everything. As a result, they construct arbitrary rankings that rely too heavily on how many points a player scored, ignoring important details like whether the player hogs the ball.
A truly comprehensive formula can correct for this, reflecting a player’s actual performance. Gladwell quotes approvingly from a book called Wages of Wins (which also has a related blog): “One can both play and watch basketball for a thousand years. If you do not systematically track what the players do, and then uncover the statistical relationship between these actions and wins, you will never know why teams win and why they lose.”
Of course, Gladwell himself has spilled a lot of ink arguing just the opposite. His book Blink: The Power of Thinking without Thinking claimed that we often make better decisions when we have less, rather than more, information. Gladwell gives an example from the art world. In 1983, the Getty Museum in Los Angeles considered buying a marble statue from the sixth century BC. Before the purchase, they hired a geologist who concluded, after an extensive investigation of the marble, that the statue was authentic. But after the purchase, several art experts visited the museum and concluded, after a moment’s viewing, that the statue was fake. As it turns out, the art experts were right. Largely on the basis of examples like these, Gladwell pursues the idea that “there can be as much value in the blink of an eye as in months of rational analysis.”
Gladwell received many responses to the “Game Theory” article. As he recounts in Blink, a large number of his readers felt that statistics simply cannot substitute for an instinctive reaction to an athlete. Gladwell admits that they are partly right. He uses Michael Jordan as an example. At age 17, Jordan’s statistics did not put him at the top (yet). “What set Michael Jordan apart from his peers was his attitude and motivation. And those qualities can’t be measured with formal tests and statistics.” Ultimately, then, Gladwell advises that the best decisions combine both rational analysis and instinctive judgment.
Bringing instinctive judgments to statistics
Interestingly, although House and Leman remain neutral about the rationality of the NCAA committee’s decisions and do not advocate for any particular change, the general framework they use offers certain opportunities for taking Gladwell’s advice. One way to see this is by examining Bayes’s rule. In statistics, Bayes’s rule can incorporate rational analysis as well as real-world observations at the same time.
Statisticians use different methods to assign probabilities to the occurrence of events. But some of these methods use more information than others. Here is a simple example. Suppose you join a game of pick-up basketball among strangers in the park. Each team has five players. Early in the game, one player knocks the ball out of bounds. You see him do it, but since he’s a stranger and you just joined the game, you cannot remember if he is on your team or not. Given that there are two teams on the court, each with the same number of players, there is a 50% probability that the offending player is on your team.
That’s a fairly coarse calculation. But using Bayes’s rule, we can update that figure, based upon things we have observed in the real world. When you saw the player knock the ball out of bounds, you noticed that he was wearing a blue shirt. It so happens that on your team, four out of five players are wearing blue shirts. But on your opponents’ team, only one player is wearing a blue shirt. Instinctively, this suggests it’s pretty likely that the offending player is, in fact, on your team. Bayes’s rule allows us to incorporate that observation directly, updating the probability figure from 50% to 80%.
Importantly, with Bayes’s rule, it’s the actual observations we make in the real world which are used to update probabilities. You happened to notice a blue shirt, so that’s the knowledge you contribute to the statistical formula. But if you had noticed something else about the player who knocked the ball out of bounds — that his shoes were green, for example — Bayes’s rule would compute a different figure. In other words, the probability figure offers a substantial amount of wiggle room.
The Matthew effect
But the most troubling aspect of House and Leman’s results is not the general fact of wiggle room. We all know it exists. In fact, sports professionals and fans must somehow want wiggle room; if they didn’t, the NCAA could just use a computer algorithm to select all sixty-eight teams for March Madness. There would be no hand-wringing, and no such thing as a “bubble team.”
But a committee of actual human beings heightens the excitement considerably. It produces dramatic moments. And it produces funny quotes, like this one from Seth Greenberg last month on WRAL sports: “Well, you look up ‘on the bubble’ in the dictionary and you’ll see my picture also. I am Mr. Bubbalicious.”
No, the truly troubling aspect of these results concerns the “Matthew effect.” Coined by sociologist Robert Merton and discussed by Gladwell in his book Outliers: The Story of Success, this effect owes its name to a passage from the gospel of Matthew: the rich get richer and the poor get poorer.
The marquee factor uncovered by House and Leman demonstrates exactly that. The Tar Heels, by basketball standards, are rich. They possess the legacy of Michael Jordan. They have won more games than almost any other team in college basketball history. And these facts, which constitute their “marquee”, help them get richer still, increasing the probability of an NCAA invitation and providing them with another opportunity to succeed.
Outliers critiques the societal forces that give rise to the Matthew effect. “To build a better world,” writes Gladwell, “we need to replace the patchwork of lucky breaks and arbitrary advantages that today determine success…with a society that provides opportunities for all.”
If the marquee factor were eliminated, in other words, we could have a more equitable NCAA tournament. Worthwhile teams like the Virginia Tech Hokies would stand a fair chance of getting invited, and this invitation would provide them with further opportunities to succeed. Shouldn’t a guy who refers to himself as “Mr. Bubbalicious” get a fair shake?
Berri, David J., Martin B. Schmidt, & Stacey L. Brook. 2007. Wages of Wins: Taking Measure of the Many Myths of Modern Sport. Stanford Business Books.
House, Leanna & Scotland Leman. 2011. Life on the bubble: Who’s in and who’s out? Technical report, Virginia Tech.
Gladwell, Malcolm. 2006. Game theory. The New Yorker.
Gladwell, Malcolm. 2005. Blink: The Power of Thinking without Thinking. New York: Back Bay Books / Little, Brown and Company. eBook edition: April 2007.
Gladwell, Malcolm. 2008. Outliers: The Story of Success. New York: Little, Brown and Company.