By Bill Henderson & Andrew Morriss
Yesterday, Bob Morse of U.S. News published a blog post in which he signaled a change in the law school rankings methodology, specifically with regard to employment. The prevailing view on the law school administrator list-serves (which nearly a dozen people have forwarded to us) is that U.S. News will be increasing the weighting of "employed at graduation," presumably because U.S. News Editor Brian Kelly sent a letter to law school deans--reprinted in Bob's blog post--discussing the importance of employed-at-graduation as a metric.
We have zero inside information, but we are willing to bet a substantial sum that any methodology change will be in a completely different direction. Here is why. Over the last decade, fewer and fewer schools have been supplying U.S. News with employed-at-graduation data. Employment at graduation is not a statistic required or collected by the ABA; as such, its accuracy cannot be checked through cross-reference to the annual ABA-LSAC Official Guide.
But much more significantly, when a school fails to provide this data, U.S. News has--up until now--imputed the figure based on employment at 9 months. (Kudos to Ted Seto for unraveling this mystery. See Understanding the U.S. News Law School Rankings.) Crudely speaking, the magazine applied a roughly 30% discount rate on the employed at 9 months figure. Earlier this year, Paul Caron suggested that if a school's employed-at-graduation rate is more than 30% lower than its employed at 9 month rate, it is "rankings malpractice" to supply U.S. News with the data.
As readers can see from the above chart (generated by Paul Caron in his rankings malpractice post), a large proportion of law schools have figured out the payoffs. Over the last decade, the percentage of non-reporting schools has skyrocketed. With this information in mind, Bob Morse's blog post may seem less cryptic:
In an effort to make our law school employment data more reflective of the current state of legal employment, U.S. News has modified how we calculate the employment rates that are used in the new law school rankings. ...
U.S. News agrees with the efforts of Law School Transparency to improve employment information from law schools and make the data more widely available.
If the goal is (a) to utilize data that better reflect reality, and (b) provide greater transparency and access to such data, it makes no sense to increase the weight of an input (employed-at-graduation) that is either withheld by law schools or is heavily gamed. This latter point is made by U.S. News Editor Brian Kelly in his letter to the deans:
[E]mployment after graduation is relevant data that prospective students and other consumers should be entitled to. Many graduate business schools are meticulous about collecting such data, even having it audited. The entire law school sector is perceived to be less than candid because it does not pursue a similar, disciplined approach to data collection and reporting.
At U.S. News, we work to make meaningful and fair comparisons, based on industry-accepted data. ...
To eliminate some of the gaming that seems to be taking place, we have changed the way we compute employment rates for the rankings due out March 15. In addition, we will also be publishing more career data than we have in the past in an effort to help students more completely understand the current state of legal employment. We think more still needs to be done.
So Kelly is saying that employed at graduation data are important, and the magazine is tired of being gamed. Therefore, we think two methodogical changes have a good chance of being implemented:
- U.S. News is likely to heavily penalized schools that withhold the employed at graduation data. Going forward, the imputation may be far more negative than -30% off of employed at 9 months. A drop in rankings will stop in its tracks the non-response problem.
- Regarding perceptions of gaming, it is possible that U.S. News has formulated a way to quantify how many jobs at graduation map onto full-time professional jobs that require a law degree. For example, the ABA Official Guide provides lots of comparable data by practice setting. Law firms, judicial clerks, and government jobs could be weighted more heavily than business or academic jobs. Unknown may also be treated as 100% unemployed rather than the current 25% presumption of employment. Such changes would have the law schools scrambling to report better numbers in higher weighted categories rather than just finding ways to goose up the employed-at-graduation and employed-at-9 months figures. Remember that Bob Morse explicitly endorsed the Law School Transparency movement.
We would like to suggest to our colleagues in the legal academy that we are approaching an endgame. Here is the reality: prospective students are not being given an accurate picture of their future employment prospects. Why? Because we are all focused on filling next year's class with as many high credential students as possible, thereby protecting our school's place in the pecking order. Our focus is so shockingly narrow that, from the outside looking in, it appears that our intent is to deceive incoming students. Brian Kelly's letter to the deans essentially makes that point--law schools fall short on candor and ethical behavior.
The numbers that get submitted to U.S. News include many graduates who are technically employed but often significantly underemployed, often at positions that don't require law degrees. Finer grained data get reported to NALP, but they are never published on a school-by-school basis. If these data were released, prospective students may not fully process the information--that is an argument that we often hear law professors make. But that does not alter our duty to provide "basic consumer information ... published in a fair and accurate manner reflective of actual practice." ABA Accreditation Standard 509.
At some point, all our lawyerly rationalizations will come to a bad end because a governmental agency or a court is going to challenge our right to self-regulation, thus ushering in a truly disgraceful chapter in the history of American legal education.
Now is one of the very few moments in our careers as academics where we have to make hard choices and demonstrate that we warrant the trust and respect of our tenured positions. Through our governance organizations (ABA, LSAC, NALP, AALS), we need to implement a system of complete transparency on employment outcomes. If the system has real teeth, it will force us all to work very hard to ensure we are delivering value commensurate with the tuition dollars we collect.
It's the end of the road. We likely have one last chance to get it right.
Recent Comments