USNWR Gaming and the Failure of Self-Regulation

I recently picked up the latest issue of National Jurist, which is a publication for current and prospective law students.  The cover story this month is "The Rankings Race: How Far Will Law Schools Go?"   

The article provides plenty of examples of how law schools attempt to game their U.S. News input variables, especially with regard to employment.  Michael Sauder, a sociologist at Iowa who has written extensively on rankings, offered instances in which law schools count alumni taxi drivers as "employed" graduates.  David Bernstein (GMU Law) cited a practice in which law schools have rounded up unemployed graduates and employed them for purpose of getting over the nine month USNWR hurdle.

The former President of the Associations of American Law Schools (AALS), William Hines, is quoted that "too many otherwise sensible law schools are engaged in questionable academic practices and may in some cases be borderline dishonest."  Two years ago, a similar damning story appeared in the New York Times.  See Alex Wellen, The $8.76 Million Maneuver.   

I don't think the legal academy appreciates the implications of all this well-documented gaming.  Not only are we cutting corners writ large in order to get ahead--and hence have zero credibility to emphasize the importance of honest, integrity, and ethical behavior to our students--but we are providing overwhelming evidence that we are incapable of effective self-regulation.  Right now, gaming pays.  Further, if you are a dean, and you refuse to aggressively manage your USNWR ranking, your job is in serious jeopardy.  But calling U.S. News a collective action problem is a cop-out.  Where is the ABA, our accrediting agency, to police this behavior?   

The table below, from the Measuring Outcomes paper by Andy Morriss and I, documents changes in graduate placement since U.S. News moved to the current employed at graduation/9 months methodology.  If these numbers are to be believed--and we would advise against it--these are bullish times for newly minted lawyers.  And the rising tide has raised all tiers.

Some of these remarkable gains may be the result of competition among schools to more effectively place, or at least track, their graduates.  If that is the primary explanation, then U.S. News has actually been a force for good, at least in terms of employment.  But a more likely explanation is that any employment--law firm associate, judicial clerk, public interest lawyer, taxi driver, waiter, insurance salesman, temporary law school RA, whatever--counts for the purposes of U.S. News. 

The biggest red flag is that while total employment at 9 months for all law school graduates surged from 83.9% to 91.6% between 1997 and 2006, first-time bar passage for this same cohort dropped from 83.0% to 78.6%, presumably due to the trend toward higher bar cut scores.  See pp. 13-14.

Note that the results from a February bar will rarely be available within 9 months of graduation. So, as we note in the paper,

it is likely that much of the increased post-graduation employment is in jobs that do not require a valid license to practice law and hence are unlikely to pay a premium for three years of legal education. At a minimum, this practice has the potential to seriously mislead prospective law students attempting to evaluate the relative costs and benefits of a legal education. It is also hard to believe that this type of reporting should be considered sufficient to satisfy ABA Accreditation Standard 509, which requires a law school to provide "basic consumer information" in a "fair and accurate manner reflective of actual practice."

As a self-regulating profession, it is time to get our own house in order.

Comments

[Nancy Rapoport]

I agree with both Andy and Bill, but here's the rub: if the ABA hardly ever yanks accreditation from an already accredited school (can anyone remember the last school to lose accreditation???), then why should we assume that the ABA will be able to enforce cheating (or "puffing," which is how many schools might characterize the massaging of rankings).

As early as March 1, 2006, I posted a piece on the JURIST Forum about Enron, "managing the numbers," and how that kind of truth-shading applies to law schools and the rankings (see http://jurist.law.pitt.edu/forumy/2006/03/enron-and-managing-numbers.php). Christine Hurt made the same point even earlier (2/16/06) in a post on the Concurring Opinions blog (see http://www.concurringopinions.com/archives/2006/02/the_aba_affirma.html). And, of course, the great work that you both have done on the manipulation of the rankings makes this point yet again. (Irony of ironies, all four of us made this point long before the UH Law Center faculty meeting that was mentioned in the first paragraph of the story in The National Jurist.)

What can we do, if we can't rely on the ABA to enforce the USNWR reporting? (I'm not convinced that it's the ABA's role to enforce truthful reporting of the info given to USNWR, although it certainly is within the ABA's purview to enforce truthful reporting of the info given to the ABA itself.)

Well, we could back up those deans who refuse to play to the rankings with provosts who reward actual good work (teaching well, publishing important scholarship, serving the various communities). We could focus on what matters to us, rather than what matters to a for-profit news magazine that doesn't have to deal with the repercussions of its most-profitable issue. And we could use faculty peers to discourage all sorts of passive-aggressive, unproductive, bullying behavior. (See Jeff Harrison's wonderful MoneyLaw post on why silence in the face of bullying behavior is so damaging, http://money-law.blogspot.com/2006/11/making-nice-knowing-better-doing.html.)

We could, of course, just continue to blame the rankings, but they're just the symptom of a much larger problem: we've lost our moorings by continuing to let others define what constitutes a great legal education.

[William Henderson]

John,

I completely agree. I worry about that. The ABA needs to step in. Appeals to "just do what is right" are not enough. The situation is an utter train wreck. bh.

[John Steele]

Bill,

Good post, as always. But what about this additional problem: if actions speak louder than words, then think about what those schools are teaching their students about what conduct is expected in the legal profession.

Imagine that one of those law schools finds itself in the news a few years from now because one of its alums is indicted in an Enronesque scandal about SEC disclosures. Will the dean and profs be able to say, as I'm sure they'd love to say, "we don't have the slightest idea how alum got the idea that fudging and faking numbers is legitimate for members of the legal profession"?