NY Times Review of Super Crunchers

This Sunday's N.Y. Times had an interesting review of Ian Ayres new book, Super Crunchers: Why Thinking by Numbers is the New Way to Be Smart.   In a nutshell, this book piles on to a theme developed by Moneyball and Freakanomics -- you can profit by data mining, especially when the conventional wisdom is wrong.   

NY Times economics reporter, David Leonhardt, begins his review of Super Crunchers with an anecdote on Orley Ashenfelter's theory that the weather in Bordeaux is the best predictor of a wine's eventual price.  Apparently, Ashenfelter's theory irritated the wine establishment, which heaped scorn upon him.  In fact, Ashfelter is part of lore of the early days of the American Law & Economics Associations.  Many years ago, Ashenfelter gave the ALEA key note address over dinner and entertained the crowd with his musings on weather and wine prices, noting (to the delight of his listeners) that his theories had already produced a tidy profit.

To my mind, the principles of Moneyball-Freakanomics-Super Crunchers are hard to dismiss. Indeed, over the last several years, I cannot recall a single frontal attack on any of these ideas.  Instead, it is my observation that most people (and thus organizations and institutions) never take the time to think through their simple, stark implications--i.e., don't be complacent, look at the data, or eventually you are going to get smoked by the competition.   

Here is the key passage in Leonhardt's review:

"For all its successes ... statistical analysis continues to face tremendous skepticism and even animosity. For one thing, Ayres notes, statistics threaten the 'informational monopoly' of experts in various fields. But even to many people without a vested interest, relying on cold, hard numbers rather than human instinct seems soulless.

The problem, as Aryes points out, is that there is overwhelming evidence that human beings are notoriously overconfident in their own judgments and intuitions.  Indeed, ever since Moneyball, every where I turn I am astonished by the overconfidence of the well-placed and the well-credentialed (today's winner being Michael C. Dorf).  This creates a lot of opportunities for the rest of us.   

Comments

[John Lott]

Well, there are many critiques of the empirical work discussed in Ayres' book as well as Freakonomics, but it is just that one would never know about these problems because these authors ignore critical pieces that have been written. Presumably if these authors had a response they would at least mention the claimed problems with at least some response, but they never mention any critiques.

[Jeff Yates]

(cross posted on Voir Dire Blog)

I think that this brings up some interesting issues in moneyball theory and practice:

1. Exactly how applicable are moneyball principles to academic institution building? It seems to me that a "win" is pretty straightforward in baseball and not always so straightforward in academia. If a school plays moneyball well and gains ground (in the ratings), then isn't it likely that schools in the higher tier will simply attack the accuracy of the posited measure of "success" or "performance" or simply change the focus of top schools' mission (i.e. definition of a win). It's not as if we don't see this type of behavior quite frequently in academia.

2. Given the sluggishness of rankings and prestige ratings of schools, how viable is a moneyball strategy for making large gains? Won't the more wealthy schools (aka NY Yankees) simply "buy up" the better performers at moneyball schools? In other words, I think that this strategy rests, to some degree, on on strategic institution building (i.e. "zigging while others are zagging"), and that often turns on not only building talent (and performance) through moneyball strategies, but also in retaining those high performers once it is established that your alternative strategies paid off. This is quite simply to say that it will ultimately require $$$ outlays to make the moneyball work down the road - we don't have a year to year World Series of Academia (although maybe we should); it's a long term game.

Don't get me wrong - I'm all for moneyball strategies - but I think that these are some issues that need to be resolved to make it viable.

[Michael Dorf]

I invite readers of this post to read my post, the comments on it by others along with my responses, and my followup at http://michaeldorf.org/2007/09/dorf-on-leiter-on-althouse-on-dorf-on.html to judge for themselves whether I endorse the sort of data-blind views taken by traditional baseball scouts critiqued in Moneyball. I think that any discerning reader will see that I was making a quite different point: I was guessing that elite law schools could do a middling job at actually educating students and still not suffer serious reputational hits, because their students' talents cover up for this. (This is why the Yankees have been a good team even though they didn't follow the path of Billy Bean: their huge war chest covered up their errors.) I don't deny that eventually good innovations win out but I say that it will likely require a market leader to make the process work. Thus, the Boston Red Sox have been more successful using Bean's strategy than the A's were, because the Sox have done it with more money. So I decline your prize.