Vault, Inc, which is a publisher that collects industry information on various professions, recently released its listing of the Top 25 Most Underrated Law Schools. (Hat tip: Paul Caron & Volokh.) The findings are derived from a survey of 512 law firm recruiting managers, hiring partners, and corporate counsel, who were asked "to name law schools that, based on their experience as hiring managers, are underrated."
As I read the list, I wondered, what is the practical implication of a law school being underrated by the people who make hiring decisions for entry level lawyers? Presumably, it means that graduates of certain law schools tend to perform better than their school's U.S. News ranking would suggest; thus, legal employers are more likely to hire them.
If this is true, what is the source of superior performance? Here are two possibilities:
- Stronger Students. Some schools may enroll a stronger student body than their rank might suggest.
- Better Education. Some law schools may equip graduates with more or better skills than other schools of comparable rank
Based on some preliminary statistical analysis, there is fairly clear quantitative evidence for the first hypothesis. There is also some qualitative evidence for the second--enough to warrant some additional research.
These research questions are really important to the ranking debate because their answers could reveal a mechanism by which some students will discount USN. If an "underrated" school offers a better entree to coveted legal employers, why go to the higher ranked school, especially if the underrated school costs significantly less? Law schools that understand these dynamics are in a better overall competitive position.
After the jump, I elaborate on these findings, including methodology.
To create a relevant dataset, I added the Vault underrated data to a file containing the current U.S. News rankings, including academic reputation, lawyer/judge reputation, 25th percentile LSAT, and 75th LSAT. Vault has two measures of underrated, national and regional. Based on my earlier research, which showed that geographic proximity is very important in the hiring opportunities available to students at non-elite law schools, I placed more credence in the regional listings. (The national scores were also based an arbitrary 50/50 weighting of the national and regional survey results, similar to the USN overall composite rank.) I then created two variables: Vault national underrated (1 = yes, 0 = no), and Vault regional (same coding).
Stronger Students theory
If some law schools have attracted the attention of recruiters because they tend to enroll stronger students, "underrated" might show up in a disparity between overall USN rank and rank based purely on the 25th or 75th percentile LSAT statistics. I thus subtracted USN rank from LSAT rank. For example, regionally "underrated" Brooklyn Law School is ranked #58 in USN but is #36 when ranked by 75th percentile LSAT. If employers believe that raw aptitude is a good predictor of future job performance, then, from that perspective, Brooklyn is arguably "underrated" by #22 slots (36 - 58 = -22).
To test whether the LSAT rank/USN rank is associated with making the Vault regional "underrated" list, I ran a simple t-test comparing the means of "underrated" and "not underrated." (Note this was limited to the Top 100 schools because I needed an ordinal ranking rather than a tier.) The results were fairly clear.
- The mean disparity between 75th LSAT rank and USN rank was -6.75 ranking slots (negative sign denotes "underrated). The means were statistically difference at p = .015 (t stat = 2.463).
- The mean disparity between 25th LSAT rank and USN rank was -5.96 ranking slots. The p-value, however, was relatively low but above the .05 threshold (p = .127).
Arguably the 75th percentile is more relevant because many employers will not interview students at many schools who do not reach a certain grade cutoff. A better way to model this is dynamic, of course, is a logistic regression that adds some control variables or OLS in which we have the actual regional "underrated" scores. But that is for another day. The above analysis provides limited but fairly straightforward support for the Stronger Students theory.
For the record, just as I thought, the Vault national underrated variable produced wholly random and non-significant results (i.e., p > .800 for all analyses).
Better Education theory
The Better Education theory has larger potential implications for law schools because it suggests that some law schools may be adding more value than other peer institutions during three years of JD education and that legal employers are noticing. In other words, curriculum and/or quality of instruction are "moving the market."
If some law schools have gained a reputation among legal employers for better preparation for practice, it might show up in either the USN academic or lawyer/judge reputation scores. (Don't laugh: Andy Morriss and I have shown the lawyer/judge reputation score appears to be associated with better employed-at-9 months figures after controlling for LSAT, legal market size, and school OCI activity. See "Measuring Outcomes.")
Using the same methodology as above, the Rep rank / USN rank disparity had no statistically significant association with Vault underrated status: -1.97 mean disparity for Acad rep/ USN rank, p = .317; -0.59 mean disparity for Lawyer-Judge rep / USN, rank, p = .852). Of course, this may not be all that surprising since the Acad rep is clearly influenced by the prior year's overall rank. See Jeffrey Evans Stake, The Interplay between Law School Rankings, Reputations, and Resource Allocation, 81 Ind. L. J. 229 (2005).
Yet, notwithstanding these indeterminate results, many of the recruiter comments in the Vault survey suggest that law school training may be making a difference. For example, these comments on specific schools lend support to the Better Education theory:
- Baylor Law School. "Baylor has a top trial advocacy program that puts a Baylor graduate fives years ahead on the power curve." "Baylor has the top advocacy program in the country."
- Chicago-Kent and Depaul. "These schools have great training. Summer associates from these schools have been well prepared, particularly in writing, and have performed well as summer associates and associates."
- Lewis & Clark (Ore). "Lewis & Clark's specialty areas-particularly Environmental and Natural Resource Law-the varied legal clinics, externship and internship programs, and moot court selections are excellent opportunities."
- Northwestern. "Northwestern should be Top 5 -- best prepared graduates, curriculum, business focus, leadership development."
- University of NC Central (Tier 4) and Univ. of Montana (Tier 3). "With their clinical programs and emphasis on pro bono / public interest practice, these schools [NCCU and University of Montana] do an excellent job of training technically proficient citizen lawyers."
- Seattle University. "Seattle University has a very highly rated writing program."
- Southern Illinois University (Tier 3). "Southern is a newer school in a remote location, so does not have the connections other schools have, however, they have a mandatory attendance policy, strong legal writing and advocacy programs, great student-teacher ratio and probably one of the best value propositions going in terms of cost to attend. ... [I]n my experience, [these graduates ] are often better trained than grads from overrated schools you are competing with."
- University of Denver. "The school emphasizes preparing practice ready students, and they do an excellent job of that. DU students easily compete with those from top law schools across the country."
- University of Richmond. "The students that come out of UR seem to have more prepared for the practice of law than those from some other schools."
In the above excerpts, I have selected out comments that speak to quality of curriculum and training. But there are also a large number of comments that corroborate the Better Students theory. Here are just three:
- Boston College and Boston Univ. "Smarter than is ranked."
- Brooklyn and Fordham. "Both schools have entering classes with stellar credentials."
- UC Davis and UC Hastings. "Both of these public law schools have highly selective admissions policies ... "
One unknown in the above analysis is which way the causal arrow points: Are better curricula and teaching attracting better students, or are better students causing recruiters to think more highly of a school's instruction? For example, many of the schools lauded for excellent training also are among the Top 10 in disparity between 75th LSAT and USN rank: Lewis & Clark (-41); Univ. of Rich. (-29); and Brooklyn (-22).
What is interesting, however, is that some recruiters appear to be paying attention to the nuts and bolts of legal education. Who are these folks? This is a real opportunity for some law schools to build strategic relationships and outperform their U.S. News rankings. In the long run, a school's USN ranking may go up as well.