March Madness meets Malcolm Gladwell

By Anne Pycha

“Basketball’s decision-makers, it seems, are simply irrational,” wrote Malcolm Gladwell in 2006. On the eve of March Madness 2011, his words have taken on new meaning, thanks to a recent study conducted at Virginia Tech.

Each spring, 68 basketball teams participate in the NCAA Division I Men’s Basketball Tournament, also known as March Madness. Of these teams, 31 qualify automatically by winning their division championship. Remaining teams can participate only if they get hand-selected by an NCAA committee. This creates some drama and hand-wringing, especially for the “bubble” teams whose season performance might or might not warrant an invitaiton. The committee, which will announce its decisions this year on Sunday, March 13, consists of approximately ten athletic directors and sports commissioners from around the country. (That this year marks some changes; prior to 2011, only 64 teams participated in the tournament).

Virginia Tech professors Leanna House and Scotland Leman examined the decisions made by this committee from the 1993-1994 season to the present. Using Bayesian statistical methods, they found that a team’s Ratings Percentage Index (RPI) correlated strongly with an invitation to the tournament. The RPI includes three basic components: how often the team won during the season, how often their opponents won, and how often their opponents’ opponents won.

Sounds rational so far. But the RPI does not tell the entire story. House and Leman’s technical report suggests that “the committee may have a cognitive weighting scheme which does not agree perfectly with the RPI formula.” In other words, the decision-makers leave themselves some wiggle room.

So House and Leman examined an additional factor: marquee. A marquee team has a history of top performance in the tournament. The North Carolina Tar Heels offer a good example. In 1982, a player named Michael Jordan helped this team win the tournament championship. In subsequent years, the team won the championship several more times, most recently in 2009.

The NCAA Committee announces its decisions on "Selection Sunday" (this year, March 13)

House and Leman’s analysis showed that, given two teams with roughly equivalent RPIs, the committee was historically more likely to invite the marquee team. In 2000, for example, the Tar Heels finished the season with 31 wins and 21 losses. On the basis of their RPI alone, they had a 32% probability of being invited to the tournament. But the team’s marquee status boosted that probability to 85%. The committee did, in fact, invite them that year.

Non-marquee teams suffer the consequences of this decision-making. In 2010, the Virginia Tech Hokies finished the season with 33 wins and 13 losses. On the basis of their RPI, the Hokies had a 31% probability of being invited. Marquee status, if they had possessed it, would have boosted that probability to 83%. The committee did not invite them.

Virginia Tech coach Seth Greenberg raised the issue of marquee status out loud last year, providing House and Leman with the inspiration for their study. In March 2010, both the Hokies (33 wins, 13 losses) and the Tar Heels (20 wins, 17 losses) failed to receive an invitation to the tournament. Just afterwards, as reported on Techsideline.com, Greenberg asked: If North Carolina had Virginia Tech’s exact same resume, would they have been in the NCAA Tournament? The answer, it appears, is yes.

Making good decisions

So, is Malcolm Gladwell right? To some extent, the answer depends upon what we mean by a “rational” decision.

Gladwell’s 2006 article on basketball’s decision-makers, entitled “Game Theory” and published in The New Yorker, focused on the problem of ranking individual players. In it, Gladwell sang the praises of comprehensive rankings. Basketball games move so quickly and involve so many moves that observers simply cannot absorb everything. As a result, they construct arbitrary rankings that rely too heavily on how many points a player scored, ignoring important details like whether the player hogs the ball.

A truly comprehensive formula can correct for this, reflecting a player’s actual performance. Gladwell quotes approvingly from a book called Wages of Wins (which also has a related blog): “One can both play and watch basketball for a thousand years. If you do not systematically track what the players do, and then uncover the statistical relationship between these actions and wins, you will never know why teams win and why they lose.”

Of course, Gladwell himself has spilled a lot of ink arguing just the opposite. His book Blink: The Power of Thinking without Thinking claimed that we often make better decisions when we have less, rather than more, information. Gladwell gives an example from the art world. In 1983, the Getty Museum in Los Angeles considered buying a marble statue from the sixth century BC. Before the purchase, they hired a geologist who concluded, after an extensive investigation of the marble, that the statue was authentic. But after the purchase, several art experts visited the museum and concluded, after a moment’s viewing, that the statue was fake. As it turns out, the art experts were right. Largely on the basis of examples like these, Gladwell pursues the idea that “there can be as much value in the blink of an eye as in months of rational analysis.”

Malcolm Gladwell (with David Remnick of The New Yorker), February 15, 2011

Gladwell received many responses to the “Game Theory” article. As he recounts in Blink, a large number of his readers felt that statistics simply cannot substitute for an instinctive reaction to an athlete. Gladwell admits that they are partly right. He uses Michael Jordan as an example. At age 17, Jordan’s statistics did not put him at the top (yet). “What set Michael Jordan apart from his peers was his attitude and motivation. And those qualities can’t be measured with formal tests and statistics.” Ultimately, then, Gladwell advises that the best decisions combine both rational analysis and instinctive judgment.

Bringing instinctive judgments to statistics

Interestingly, although House and Leman remain neutral about the rationality of the NCAA committee’s decisions and do not advocate for any particular change, the general framework they use offers certain opportunities for taking Gladwell’s advice. One way to see this is by examining Bayes’s rule. In statistics, Bayes’s rule can incorporate rational analysis as well as real-world observations at the same time.

Statisticians use different methods to assign probabilities to the occurrence of events. But some of these methods use more information than others. Here is a simple example. Suppose you join a game of pick-up basketball among strangers in the park. Each team has five players. Early in the game, one player knocks the ball out of bounds. You see him do it, but since he’s a stranger and you just joined the game, you cannot remember if he is on your team or not. Given that there are two teams on the court, each with the same number of players, there is a 50% probability that the offending player is on your team.

Pickup basketball in Philadelphia, Summer 2010

That’s a fairly coarse calculation. But using Bayes’s rule, we can update that figure, based upon things we have observed in the real world. When you saw the player knock the ball out of bounds, you noticed that he was wearing a blue shirt. It so happens that on your team, four out of five players are wearing blue shirts. But on your opponents’ team, only one player is wearing a blue shirt. Instinctively, this suggests it’s pretty likely that the offending player is, in fact, on your team. Bayes’s rule allows us to incorporate that observation directly, updating the probability figure from 50% to 80%.

Importantly, with Bayes’s rule, it’s the actual observations we make in the real world which are used to update probabilities. You happened to notice a blue shirt, so that’s the knowledge you contribute to the statistical formula. But if you had noticed something else about the player who knocked the ball out of bounds — that his shoes were green, for example — Bayes’s rule would compute a different figure. In other words, the probability figure offers a substantial amount of wiggle room.

The Matthew effect

But the most troubling aspect of House and Leman’s results is not the general fact of wiggle room. We all know it exists. In fact, sports professionals and fans must somehow want wiggle room; if they didn’t, the NCAA could just use a computer algorithm to select all sixty-eight teams for March Madness. There would be no hand-wringing, and no such thing as a “bubble team.”

But a committee of actual human beings heightens the excitement considerably. It produces dramatic moments. And it produces funny quotes, like this one from Seth Greenberg last month on WRAL sports: “Well, you look up ‘on the bubble’ in the dictionary and you’ll see my picture also. I am Mr. Bubbalicious.”

No, the truly troubling aspect of these results concerns the “Matthew effect.” Coined by sociologist Robert Merton and discussed by Gladwell in his book Outliers: The Story of Success, this effect owes its name to a passage from the gospel of Matthew: the rich get richer and the poor get poorer.

The marquee factor uncovered by House and Leman demonstrates exactly that. The Tar Heels, by basketball standards, are rich. They possess the legacy of Michael Jordan. They have won more games than almost any other team in college basketball history. And these facts, which constitute their “marquee”, help them get richer still, increasing the probability of an NCAA invitation and providing them with another opportunity to succeed.

Outliers critiques the societal forces that give rise to the Matthew effect. “To build a better world,” writes Gladwell, “we need to replace the patchwork of lucky breaks and arbitrary advantages that today determine success…with a society that provides opportunities for all.”

If the marquee factor were eliminated, in other words, we could have a more equitable NCAA tournament. Worthwhile teams like the Virginia Tech Hokies would stand a fair chance of getting invited, and this invitation would provide them with further opportunities to succeed. Shouldn’t a guy who refers to himself as “Mr. Bubbalicious” get a fair shake?

References

Berri, David J., Martin B. Schmidt, &  Stacey L. Brook. 2007. Wages of Wins: Taking Measure of the Many Myths of Modern Sport. Stanford Business Books.

House, Leanna & Scotland Leman. 2011. Life on the bubble: Who’s in and who’s out? Technical report, Virginia Tech.

Gladwell, Malcolm. 2006. Game theory. The New Yorker.

Gladwell, Malcolm. 2005. Blink: The Power of Thinking without Thinking. New York: Back Bay Books / Little, Brown and Company. eBook edition: April 2007.

Gladwell, Malcolm. 2008. Outliers: The Story of Success. New York: Little, Brown and Company.

Hokiesports.com

TarHeelBlue.com

Techsideline

Virginia Tech News

WRAL sports

 

Advertisements
This entry was posted in Statistics and tagged , , , , , , , , , , . Bookmark the permalink.

4 Responses to March Madness meets Malcolm Gladwell

  1. Donald E. Black says:

    Anne,

    I enjoyed reading your Blog. I see why your mother and dad are so proud of you.

    So who are the sixty eight teams in this years men’s tournament?

  2. Marc E says:

    Hi Anne,
    Really great write-up. I felt like I was reading a real magazine 😉
    I don’t follow college hoops too closely, but I do spend some time reading and commenting on a baseball stats blog. Compared to the average sports guy, I guess I fall into the stats-freak camp, given that I have used a calculator (and R) at my job. On that blog, though, I usually end up being the voice of the typical fan in that I generally advocate that the stats view should be tempered with some way of incorporating intuition. For example, one of the big debates is whether team-player good-character guys are worth having. Well, it’s not really a debate on this site because the stats clearly say no; the numbers purport to quantify all aspects of a players’ value and so someone like David Eckstein is perpetually ridiculed as innumerate GMs’ folly. I end up being the one arguing that these statistics only tell part of the story and that humans can be quite excellent at intuiting underlying statistical generalizations, without there necessarily being a clear way to quantify it. So maybe, Eckstein isn’t such a bad guy to have on the team after all.
    It sounds like the same thing is going on here with NCAA tourney team selection. But it seems like the intuitive component, the marquee effect, isn’t necessarily a good bias presuming you’re trying to pick the best teams.
    If, however, you’re trying to maximize TV ratings, then maybe having Duke and UNC playing every year is the way to go.

  3. annepycha says:

    Hi Marc,
    Thanks for reading and I think your comment is right on the mark. The problem seems to be that statistics for team sports aren’t all that different from statistics for an individual sport like tennis. So the role of the support guys doesn’t get factored in, even though it’s crucial. I think that’s part of the argument that the “Wages of Wins” authors were making, although I’m told that basketball stats have actually progressed quite a bit in the four years since that book was published, so I’ll have to investigate further.

    Another reader also pointed out the role of TV ratings and the cold hard cash that teams like Duke and UNC bring in. Good point!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s