Good news -- the new NSF-funded Supreme Court Database Website is now up and running, and I can't imagine this won't make the database available to thousands more users via its easy-to-use interface. Not only can you download the most current version of the database and its companions, but you can also perform analyses right on the website. This is good stuff; check it out!
With a hat tip to the Monkey Cage, I thought folks might be interested in taking a look at this paper. I know my fellow editor Carolyn has done some research on the Spaeth data as well -- perhaps she'll weigh in on this research? (FYI, I have not yet read the Harvey/Woodruff paper.)
here. Courtesy of Profs. Lee Epstein, Andrew Martin, Jeff Segal, and Chad Westerland. JCS attempt to provide preference estimates for Supreme Court justices that are directly comparable to preference measures of Courts of Appeals judges, members of Congress, and the President. Our data extend through 2008 and correspond with the most recent version of Keith Poole's Common Space scores.
Thanks to the Empirical Legal Studies crew for letting me guest blog here. Consistent with the motto of the blog, I hope to use my time with ELS to bring some method to the madness that seems likely to swirl around Sonia Sotomayor’s confirmation hearings this summer.
For the past two years I have been compiling a dataset capturing all of the statements made by senators and Supreme Court nominees at the confirmation hearings held by the Senate Judiciary Committee.The dataset starts with the Felix Frankfurter’s hearing and ends with Justice Alito’s. I have coded by issue area all statements made by the nominees and senators. Each unit of analysis includes information about the political party of the questioning senator, the appointing president, and the committee chair. The dataset also captures instances where a court case was discussed by name, and includes variables indicating whether the statement being coded addressed constitutional interpretation, statutory interpretation, or federalism.
I hope to use this data to provide real-time commentary on the confirmation hearings this summer.Because Supreme Court vacancies occur so infrequently, our discussion of the confirmation process often lacks historical perspective. I hope this data can help rectify that, by bringing concrete information to conversations about what has – and has not – been “the norm” in this process.
I also hope that the ELS community will provide feedback about how to use and improve this dataset. This is in many ways a ‘test run’ of the data, so I look forward to hearing your comments about it. I’m also open to suggestions regarding what type of information you think would be interesting to pull out of the dataset.
I just finished posting some data and documentation on the Dataverse Network. This is a wonderful -- and remarkably easy to use -- resource that provides scholars with a virtual archive for their data. It allows you to make your data publicly available in a permanent format with a stable URL -- useful if, for example, you change institutions. You can even incorporate links to your data into your own website, branding the material as your own, without having to worry about hosting and maintaining it. If other scholars use or rely on your data, there is a formal citation that they can use, giving you appropriate scholarly credit.
And of course the Dataverse is a great resource for scholars who might want to replicate other people's work or rely on their data. Thanks to Harvard's Institute for Quantitative Social Science for creating and making available this resource. You can read more about it here, in a post by Gary King at the Social Science Statistics Blog.
A couple of years ago, Paul Caron flagged an obscure ABA rule change
that required law schools to report the highest LSAT score of an
admitted student versus the prior practice of averaging. Paul and
Moneylaw blogger Tom Bell
foresaw a likely surge in the number of repeat test-takers--but without any appreciable benefit. Repeat score are, in fact, less accurate in predicting
law school grades -- though, let's not kid ourselves, law schools are
not looking at the LSAT anymore for predicting 1L performance. It also
costs a lot of time and money. And if upper class white kids are
better able to afford test preparation courses, it is likely to
exacerbate the racial performance gap.
Well, the data is in for the October 2008 cycle. It is indisputable
that applicants are figuring out the implications of a low LSAT score
in the US rankings era (fewer admissions letters, few scholarship
dollars) and the potential upside/no downside of taking the exam again.
Despite a -1.7% drop in first-time takers, the repeater volume is up
16.8%. In the Northeast, where the positional competition is the most
intense, there has been a staggering 33.7% increase.
From an individual perspective, I know it makes sense to take the
test a second or third time. Indeed, it is comforting to many to have
that option. But in the aggregate, this policy really just opens the
door for a protracted zero sum game. If law schools were making better
decisions because of the second or third scores, the additional time
and expense could be justified. The second scores, however, are less
reliable than the first and second combined.
Understanding these dynamics, the regulator (the ABA Section on
Legal Education and Admission to the Bar) is supposed to set rules that
are in the best interests of the students and the profession -- not law
schools or testing agencies. Sam Stonefield of Western New England Law wrote a very detailed objection to this policy. He was prescient. I appreciate the time he took to spell it all out.
Supplementing one of the leading sources of data for state civil litigation activity in the United States, the Bureau of Justice Statistics (BJS) recently released a report, Civil Bench and Jury Trials in State Courts, 2005, which provides an important snapshot and illustrates litigation trends since 1992. The report discusses general civil cases (tort, contract, and real property) concluded by a bench or jury trial in a national sample of jurisdictions in 2005. Topics include the types of civil cases that proceed to trial, the differences between civil cases adjudicated by judges or juries, and the types of plaintiffs and defendants represented in civil trials. Also, the report covers plaintiffs. Key findings include:
In 2005 plaintiffs won in more than half (56%) of all general civil trials
concluded in state courts. The plaintiff was significantly more likely to win in
a bench trial compared to a jury trial. Among all plaintiff winners the median
final award was $28,000. Approximately 4% of all plaintiff winners won
$1,000,000 or more. Contract cases in general had higher median awards ($35,000)
than tort cases ($24,000).
The total number of civil trials declined by over 50% from 1992 to 2005 in
the nation’s 75 most populous counties. Tort cases decreased the least (40%)
while real property (77%) and contract (63%) cases registered the largest
In the nation's 75 most populous counties, some tort case categories have
seen marked increases in their median jury awards. This was particularly the
case for product liability trials, where the median awards were about 5 times
higher in 2005 than in 1992 and for medical malpractice trials, where the median
jury awards more than doubled from $280,000 in 1992 to $682,000 in 2005.
Law schools are part of a production function for entry level
lawyers. Therefore, if law schools alter their admissions practices,
the character and complexion of the hiring pool can shift
in significant ways.
On the input side, the data are crystal clear: over the last
fifteen years, the rankings arms race has pushed U.S. law schools
toward a pure numbers approach to admissions. The more interesting
question, however, is whether prestige-conscious law firms are now,
inadvertently, experiencing any fallout. First the data.
Law schools operate in an environment of supply and demand and are
famously counter-cyclical. When Silicon Valley was booming in the late
90s, law school applicants plummeted. When the economy faltered in the
early 90s or after 9/11, applicants spiked. Therefore, to examine how
admissions practices have changed over time, it is important to pay
attention to the underlying applicant pool. Below are trend lines for
median LSAT scores by USNWR rank for 1994 and 2007, which reflect
classes that entered in the fall of 1993 and 2006 respectively. During
those two admissions cycles, the number of applicants was virtually
identical: 89,600 (1994) and 88,700 (2007).
Obviously the blue line (2007) is higher than the orange line
(1994). In fact, despite slightly fewer law school applicants, the
average median LSAT increased by 2.18 points (std. dev. of 1.99). For
the record, only three schools fell out of Tier 1 between 1994 and
2007. And it cannot be explained by the ABA policy shift that
instructs law schools to no longer average LSATs for reporting 25th,
50th, and 75th percentile figures, thus slightly pumping up the volume
of high LSAT scores. That change was not enacted until the summer of 2006.
Here is the same analysis for UGPA (1994 data came from the Princeton Review, 2007 from the ABA):
Although we might chock some of the higher UPGAs (avg. of +.17, std.
dev. of +.12) on grade inflation between 1994 and 2007, it is likely
that schools were also trying to maximize this number. More after the jump ... .
I have posted on SSRN my article, forthcoming in Hastings Law Journal, Coding Complexity: Bringing Law to the Empirical Analysis of the Supreme Court. This article examines the well-known and widely-used U.S. Supreme Court Database (created by Harold Spaeth) – and most recently mentioned here – and addresses the Database’s limitations particularly for those interested in law and legal doctrine. The key point of the Article is that the Database does not contain complete or accurate information about law and legal doctrine as they appear in Supreme Court opinions. Given Harold Spaeth’s own purposes in creating the Database, these limitations may not be surprising -- although they do raise at least some challenges to his attitudinal model. Unfortunately, however, they are frequently misunderstood. Scholars all too frequently use the Database in ways that it simply cannot support, leading to the possibility of invalid or unreliable results. This post summarizes the Article’s main arguments. The primary challenges presented by the Database involve the coding for the “issue,” “issue area,” and “legal provision” variables. As the names of these variables suggest, they are frequently used by researchers interested in studying law and legal doctrine. Yet, the coding protocols for these variables (as set forth in the Codebook are not conducive to such research. Some of the limitations of these variables include: (A) The “issue” variable is not, despite its name, designed to identify any legal issues in a case. Rather, it is designed to identify the “public policy context” of a case. A case like Schenck v. Pro-Choice Network of Western N.Y. is one example. In Schenck, a group of abortion protesters challenged an injunction limiting their activities as violating the First Amendment. The only legal issue in the case involves the First Amendment and the limits it places on judicial power. But the Database codes the case as having an issue of “abortion” because that is the factual, or “public policy” context in which the case arises. (B) The coding contains a strong presumption of assigning each case only a single issue. So the Database does not add a First Amendment issue code to the coding of Schenck. (C) The issue codes are quite underinclusive and somewhat dated. For example, there are no codes for immunities, for sexual harassment, or for the dormant commerce clause. (D) Each of the approximately 260 issue codes is classified into one, and only one, of 13 “issue areas.” In some cases, the classification makes no sense. For example, in Markman v. Westview Instruments, Inc., the Court addressed the question of whether patent claims construction is a question for the judge or the jury; that is, whether there is a 7th Amendment jury right. The Database classifies Markman as a case about the right to a jury trial, but that code, which does not distinguish between civil and criminal jury rights, is located in the Criminal Procedure issue area. (E) The legal provision code does not identify cases or judge-made legal doctrines. It is limited to identifying statutes, constitutional provisions, and court rules. (F) The coding protocols provide that only legal provisions mentioned in a case’s syllabus should be identified. But the syllabus – a short summary of the case – is akin to headnotes. It is not officially part of the case, it is not written by the justices or their law clerks, and it cannot be cited by lawyers or judges. To some extent, misuses of the Database are likely due to differences in the ways that different disciplines (political science and law) use the same words. To some extent, misuses stem from scholars failing to evaluate their research design in light of the Database’s coding protocols, which are described in the Database’s Codebook. In my Article, I provide a series of examples of research project that fail to adequately take account of the Database’s limitations and that therefore produce results that may be inaccurate. To further explore the limitations of the Database and to experiment with more legally nuanced types of coding, I undertook a Recoding Project of a random sample of 10% of the cases from the last Rehnquist natural court. The details of the coding project are, of course, explained in the Article. Among other things, I redefined “issue” to mean legal issue, I expanded and rearranged the lists of issues and issue areas, I put no limit on the number of issues that could be coded per case, I redefined legal provision to include seminal cases and legal doctrines, and I identified legal provisions by looking at the opinions themselves, not just the syllabi. Some of the key findings of the Recoding Project include: (1) I identified an average of 3.7 issues and 2.4 issue areas per case, rather than the single issue and issue area per case identified in the Database. (2) I identified an average of 2.2 as many legal provisions per case as did the original Database. (3) A surprising number of legal provisions that I identified should have been identified in the Database because they were mentioned in the syllabi. (4) In both issue and legal provision coding, the “missing” codes – those that I identified but that the Database did not – disproportionately related to structural and jurisprudential issues, including procedure, the powers and operations of the federal and state governments, and the relationship between different branches of government. These and other findings have a variety of implications for researchers working with the Database. Chief among these is the importance of not drawing conclusions about the Supreme Court’s cases by looking at the numbers and types of issues, issue areas, and legal provisions coded. Researchers all too often rely on such information to draw conclusions about case complexity or about the number of issue “dimensions” in the cases. In other words, researchers sometimes point to the Database to justify their assumptions that most Supreme Court cases involve only a single issue. But as the Article demonstrates, this single-issue coding is -- or at least may well be -- an artifact of a coding protocol that presumes that each case should be assigned only a single issue, so such conclusions are circular. A second important implication is that the Database’s issues and issue areas do not accurately identify all cases involving particular legal issues and that not all cases with a particular issue or issue area code in fact involve the legal issues that a researcher might presume from the names of those codes.
Harold Spaeth has completed his coding for the Supreme Court's 07 Term. The dataset ("allcourt") is available for public download at the University of South Carolina Judicial Research Initiative (JuRI). To access the dataset please visit www.cas.sc.edu/poli/juri.
In a provocative post entitled "Is the End Near for Yale's Dominance", Brian Leiter reports one insider's assessment that Yale may lose three or four additional faculty members. If that happens, surely Harvard, with its recent lateral hiring sprees, will be the best law school in the country--right? Brian thinks that predictions of Yale's decline are premature. I agee. But in the process Brian implicitly highlights an interesting problem: what exactly does it mean to be the "best" law school?
Here are the vexing facts: on a per capital basis, Yale places more people in academia and Supreme Court clerkships than any other law school; Yale's acceptance rate is 7.3% versus 11.8% for Harvard; yet, over the last decade, the average U.S. News academic reputation score for the two schools are exactly--yes, exactly--equal: 4.840 for Harvard, 4.840 for Yale.
Is it possible that Yale is #1 because, well, Yale is #1 -- and has been every year since USN began publication? Brian refers to U.S. News "small school bias". He is right. Because of Yale's massive endowment and small student size, it enjoys a per-pupil expenditure that is roughly 1/3 its total tuition price. According to a simulation model of the 2008 U.S. News rankings, which Andy Morriss and I recently constructed, Harvard would not overtake Yale even if:
Harvard's median LSAT climbed to 180 and its median UGPA hit 4.0;
Harvard's academic and lawyer-judge reputation scores were both a perfect 5.0;
Harvard's acceptance rate plunged to less than 5%.
Yet, Yale's dominance keeps things simple. Applicants signal their elite
status by enrolling at Yale. Judges, in turn, derive prestige by hiring
Yale graduates, even though they mockingly complain that Yale clerks know very little law. And faculty favor Yale graduates because it validates
our own sense of eliteness and institutional upward movement. We can rationalize Yale's dominance in terms of scholarship, but the real endgame is the allocation of positional goods. It is so easy to get too caught up on the hamster wheel of envy and prestige without realizing that the energy expended does not necessarily produce anything of lasting social value.
Of course, each of us is free to determine our own merit criteria. I think the "best" law school is the one where faculty are willing to make inordinate personal sacrifices for the benefit of the collective enterprise--and where aspiring lawyers leave the law school skilled, confident, ethical, and ensconced in a powerful professional network that opens doors and values public service. In turn, alumnus are sufficiently grateful for the transformative experience
they received that they
are willing to underwrite the law school's mission and subsidize this opportunity
for future generations. This vision requires a greater focus on internal rather than external metrics. For us human beings, that is no easy trick.
NALP just published its 2007 edition of Jobs & JD's. One topic of interest to students, lawyers, law firms, and legal educators is the change in salary distribution from 2006 to 2007. The now famous 2006 bi-modal distribution was vivid evidence that the U.S. legal profession is undergoing significant structural change. As shown in the graph below (from this NALP webpage entitled "Another Picture Worth a 1,000 Words"), the underlying stressors are even more pronounced for the class of 2007.
The sample is based on 23,337 law school graduates from the class of 2007 who reported salary information. Note, however, that 197 ABA-Accredited law schools graduated 43,518 students in 2007. Although we know the types of jobs taken by 40,416 grads, only 57.7% of this group provided salary information. If I had to wager on the direction of underreporting, I would predict it was under-inclusive of graduates with lower salaries and those who did not pass the bar. Why? Aside from the human psychology that it is easier to share flattering rather than embarrassing information, the roughly 7,500 jobs under the second mode are fairly close to figures I have seen from ALM and NALP data, which are provided by large law firms rather than individual students. See, e.g., charts in this NLJ article.
This bias, however, is not necessarily good news. In the above graph, 32.5% of the law graduates took jobs with starting salaries in the $100K+ range; but the true percentage for the class of 2007 is probably lower. Some facts and then one normative observation. The facts first:
91.9% of 2007 graduates were employed 9 months after graduation, which compares favorably to 2006 (90.7%), 2005 (89.6%), 2004 (88.9%), and 2003 (89.0%). I would like to believe these numbers are trustworthy.
76.9% were in jobs that required bar passage. [It would be useful to disaggregate the jobs in the remaining 23.1% of law school graduates. Who are these students? How many entered law school with no intention of practicing law? ]
The median salary in the above distribution is $65,750; the mean is $86,396. But these measures of central tendency are not reliable guides of future earning power.
38% of all starting full-time salaries were less than $55,000 per year, including 18% of all jobs in private practice, 27.5% in business, and 70.0% in government (excluding judicial clerkships).
79.6% of law firm jobs in NYC, 80.3% in Washington DC, and 74.9% in Boston were in firms with 100+ lawyers. Even in Indianapolis, 50.4% were in 100+ lawyer firms. Wow! those are big numbers.
See also NALP Press Release, July 24, 2008. On the normative front, I have a simple thesis: the bi-modal distribution is bad for students, bad for law firms, bad for clients, and bad for law schools. [When I showed the 2007 distribution to one law school dean, she shielded her eyes!]:
Students. It is bad for students because at $160,000 per year, many corporate clients will ask that you not be assigned to their matters. And if your initial work experience is document review, a $160K job can quickly become a dead-end because your skill set is not growing with your billing rate (avg. 1st yr billing rate in a $160+ firm is $225 to $255/hr). So the atmosphere among associates at $160K+ firms is probably becoming more competitive. It would be better in the long run to start at $95K, learn your craft, and become a great lawyer who commands top dollar. And young lawyers should think long run.
Clients. This is bad for clients because the short term solution of requesting only midlevels and partners will eventually constrict the supply of incoming legal talent. When clients and law firms try to externalize the cost of mentoring and training--here I mean observation, contact, and feedback from partners and clients--associates are more likely to leave.
Law Firms. Actually the bi-modal distribution is only bad for firms trying to keep pace with the Am Law 200 salary pay scale. In contrast, boutiques and organizations like Axiom will find general counsel more interested in their value proposition. For Am Law 200 firms, the difficulty is getting partners to commit themselves to the future of the firm by spending more time and money investing in associates. This will reduce attrition and protect the brand. But the $160K+ cost structure provides partners with strong incentives to bill hours rather than investing in the long term future of the firm.
Law Schools. The economics of the bi-modal distribution take the pressure off elite law schools--indeed, they can raise tuition! Thus, for many law professors, the best outcome is lateraling into a Top 15 law school. But more/better law review articles--a precondition of a lateral offer--is not going to solve the difficult institutional problems of lower ranked schools. Now more than ever, all law faculty members need to understand the structural shifts taking place in our profession. When faculty at Harvard and Yale ignore these changes, it does not mean that these changes are not important. It just means that Harvard, Yale, et al. are not affected.
I don't have any solutions to these issues, though I did write up some useful insights in my prior post, "Part II: How law firms misapply the 'Cravath System.'" Our situation reflects difficult collective action and coordination/signaling problems. For example, how a firm gracefully bows out of the salary wars is an immensely difficult problem. I do think, however, that permitting nonlawyer investment would provide law firms with the financial wherewithal (and psychological courage) to experiment with more innovation. And that would be good. Larry Ribstein's scholarship is now more timely than ever. See, e.g., here and here.
When I was an interim associate at Sidley & Austin the summer after the 2000 salary wars, a partner told us that "we are all going to hell" based of the jump in salaries from $95K to $125K. I now worry that he may have been right.
In my last post, I discussed the linkage between the bimodal distribution and the emphasis on credentials under the "Cravath system". I also stated that most law firms misunderstood the internal logic of the original Cravath model and promised to elaborate in a subsequent post. This is the promised entry.
One note of context: this post is not a history lesson. The Cravath system reflects a profoundly powerful method of developing human assets. Cravath started with very good associates/inputs and turned them into truly exceptional lawyers who were in high demand by clients and other firms. Moreover, the Cravath system required lawyers to work together collaboratively to further the clients' interests. This resulted in efficient and highly effective legal services that engendered the abiding loyalty of clients and more demand for the firm's services. See Results or Résumésat 4 & n. 13 (discussing concept of firm-specific capital). In other words, under the true Cravath system, everyone comes out ahead. Two caveats: (1) the first-mover--here, Cravath 108 years ago--garners the most benefit; (2) if a firm neglects a key element--e.g., investing in associates--the model generates no competitive advantage.
[Sources: The Cravath system described below come primarily from Robert Swaine's 1948 history of the firm and other contemporaneous sources from the 50s, 60s, and 70s, which I will cite as appropriate.]
Recruiting Elite Law School Graduates
One of the hallmarks of the Cravath system is the recruitment of elite law school graduates. As of 1948, Cravath, Swaine & Moore and its predecessor firm had employed a total of 454 law school graduates as associates. Of this total, 67.7% attended Harvard (128), Columbia (124), or Yale (54). According to Swaine, "in recent years there has been an increasing number from the law schools of the Universities of Virginia and Michigan." These two schools rounded out the top five : (UVA 30, Michigan 26).
Cravath's emphasis on credentials, however, had a clear economic logic that was designed to compensate for the deficiencies of early 20th century legal education. During this period, most law schools required little or no college education. In contrast, Harvard, Columbia, and Yale grads typically had a college degree before entering law school. Swaine writes,
Cravath believed that disciplined minds are more likely to be found among college graduates than among men lacking in formal education ... .
Cravath believed in seriousness of purpose--a man with a competent mind, adapting to practicing law according to Cravath standards, should have made a good scholastic record at college. But he recognized, without full approval, the tradition of the early decades of this century--that "gentleman" went to college primarily to have a good time and make friends. Hence, while a good college record was always a factor in favor of an applicant, lack of such a record was not necessarily an excluding factor. ... [I]n the stern realities of the depression of the '30s, however, college records of applicants came to have added importance.
Yet, "[f]or a poor law school record Cravath had no tolerance." Candidates who "had not attained at least the equivalent of a Harvard Law School "B" either had a mind not adapted to the law or lacked purpose and ambition ... ." Thus, the "first choice" was a "Phi Beta Kappa man from a good college who had become a law review editor at Harvard, Columbia or Yale."
Note, however, that Cravath's emphasis on credentials had a clear business purpose designed to compensate for the limitations of legal education. During the first half of the 20th century, going to an Ivy League law school did not guaranteed legal aptitude. Prior to advent of the LSAT in 1948, college grades were the only predictors of success in law school. In 1955, The "LSAT Handbook" included cross-tab tables of LSAT scores versus law school performance for several individual law schools. At Harvard Law, roughly 1/3 of the class scored below the 50th percentile. On the west coast, UC Berkeley had a similar wide range of LSAT scores. See The Law School Admission Test and Suggestions for Its Use (ETS 1955). The clear relationship between LSAT and grades subsequently encouraged law schools to revamp their admissions criterion. But that process took decades. See Lunneburg & Radford, The LSAT: A Survey of Actual Practice, 18 J. Legal Educ. 313 (1965).
In a talk at Harvard Law School, Cravath stated that a successful "lawyer of affairs" (aka corporate lawyer) assumed "the fundamental qualities of good health, ordinary honesty, a sound education and normal intelligence." On top of these attributes, a candidate must have "character, industry and intellectual thoroughness, qualities that do not make for charm but go far to make up that indefinable something that we call efficiency. Brilliant intellectual powers are not essential."