Empirical Legal Studies

Does personality motivate job choice among lawyers and law students?

My thanks to Sara Benesh for her kind introduction and for allowing me to guest edit this post.

The government has been preparing itself for the retirement of baby boomers (the “quiet crisis”)--including legions of lawyers.  In 2007, Congress passed the College Cost Reduction and Access Act that, among other things, facilitates loan forgiveness to attract employees who have made 120 monthly payments on student loans while employed in public service.  Is the financial incentive enough to lure lawyers and law students to public service?

Beyond financial incentives, a coauthor and I have been interested in the impact of public service motivation--an intrinsic, measurable personality characteristic--on job choice.  Our articles are based on quantitative analysis of multiple data sets: (Article 1) primary data we collected from first year law students and (Article 2) analysis of secondary data “National Survey of Career Satisfaction” from the American Bar Association.  ELS readers might be particularly interested in the policy capturing design used in the first article.

Very briefly, we find that, among lawyers, public service motivation positively predicts whether one's second job is in the public sector (school debt is probably the guiding factor right out of law school). Among law students, we find that public service motivation is actually a better predictor of job-fit than sector-fit. In other words, regardless of which sector a student plans to enter, students with stronger public service motivation are more likely to accept jobs that emphasize service to others.

For government recruiters, the task ahead seems to be one of attracting applicants using service-oriented work, rather than financial perks like loan forgiveness. 

 

Guest Blogger, Robert Christensen

I am pleased to introduce Robert Christensen, assistant professor in the Department of Public Administration and Policy at the University of Georgia, as a guest blogger this week.  Most of Prof Christensen's work is on the relationship between courts and public agencies and on the impact of anti- and pro-social attitudes/actions on public and nonprofit work groups and organizations. His newest work, which he'll be blogging about, considers a personality characteristic -- "public service motivation" -- and its influence on the job choices of lawyers.  I think our readers will find it really interesting, and we welcome your comments and reactions to it.  Welcome, Rob!

Hello!

Thanks to the Empirical Legal Studies crew for letting me guest blog here. Consistent with the motto of the blog, I hope to use my time with ELS to bring some method to the madness that seems likely to swirl around Sonia Sotomayor’s confirmation hearings this summer.

 

For the past two years I have been compiling a dataset capturing all of the statements made by senators and Supreme Court nominees at the confirmation hearings held by the Senate Judiciary Committee.  The dataset starts with the Felix Frankfurter’s hearing and ends with Justice Alito’s. I have coded by issue area all statements made by the nominees and senators. Each unit of analysis includes information about the political party of the questioning senator, the appointing president, and the committee chair. The dataset also captures instances where a court case was discussed by name, and includes variables indicating whether the statement being coded addressed constitutional interpretation, statutory interpretation, or federalism.

 

I hope to use this data to provide real-time commentary on the confirmation hearings this summer.  Because Supreme Court vacancies occur so infrequently, our discussion of the confirmation process often lacks historical perspective. I hope this data can help rectify that, by bringing concrete information to conversations about what has – and has not – been “the norm” in this process. 

 

I also hope that the ELS community will provide feedback about how to use and improve this dataset. This is in many ways a ‘test run’ of the data, so I look forward to hearing your comments about it. I’m also open to suggestions regarding what type of information you think would be interesting to pull out of the dataset.

 

First up: What makes a confirmation contentious?

Discussing "Bias and the Bar" II

Sara - We definitely need to think more about how we can operationalize some of the features of "judicial temperament." I personally need more time to really think this through, so I instead thought it might be good to write about the question that keeps running through my head as I try to operationalize judicial temperament. The issue I'm struggling with, and which I'd love to hear the thoughts from others, is how we guard against imputing too much to the obvious proxies we'd have to use. For example, to play devil's advocate, I'm interpreting your suggestion of using the Congressional staff variable to reflect the quality of "open-mindedness" as implying that the political nature of the job might actually point towards close-mindedness. The question is whether that is reading too much into what it means - or what the ABA perceives it to mean - when someone has served on a Congressional staff. We thought about it in two terms: (1) an obviously political position, but also (2) a not-always-good substitution for traditional legal experience, especially given the political nature of the job. Put another way, did the person get the nomination because of their political service, or due to their qualifications? However, is the acknowledgement of the political flavor of the appointment and the nominee's background the same as measuring whether a nominee will be viewed by the ABA as "open-minded"? Alternatively, by this definition, our inclusion of the Years as Public Interest Lawyer could be seen as another measure of potential "open-mindedness" or "commitment to equal justice" given the equal justice-nature of much public interest work. But, again, are we presuming too much to suggest that the ABA views such work in this manner? Finally, what are service as a government attorney? The vast majority of people in this category - which we found the ABA viewed positively - serve as some type of prosecutor. Is this a signal of substantial courtroom and/or litigation experience, or could one possibly somehow link such experience - either positively or negatively - to some of the suggested ways of understanding "judicial temperament" in the ABA's guidelines? More broadly, how do we make sure that even our operationalization of these terms does not in and of itself make judgments? For example, does it (or should it) matter who the staffer worked for in Congress? Does it matter what issues the staffer focused on?

Of course, much of this relates back to Sara's initial post about the impact of the ABA's investigations - it's not merely that someone was a judge, but that the ABA attempts to determine whether they really were a fair and impartial judge. The problem, however, is that we have little to work with on the outside. We could obviously attempt to look into the information brought forth by others (i.e., the media, interest groups, senators, etc.), but (a) most nominees receive little attention if any, and (b) the information we hear publicly about is likely to be somewhat biased. In fact, as a side note, a big problem is the lack of information on nominees -- when I was constructing a list of nominees who were opposed by interest groups, there were a surprising number who has zero articles written about their nominations, including passing mentions that they were nominated and/or confirmed. Which brings us back to the issue of how do we attempt to explain something - or some idea - for which we only have limited information and potential proxies?

Thoughts?

Finally, we will certainly work on creating a more nuanced ideology measure. The one question, Sara, is were you referring to the distance between the ABA committee and the nominee or the distance between the Judiciary Committee and the nominee? The issue is that, and forgive me if I’ve totally misunderstood your post, is that I'm just not sure what it would mean to hypothesize that the ABA Standing Committee's ratings are influenced by how ideologically distant the nominee is from the Senate Judiciary Committee. This would in turn have to mean that we believe that the ABA is more likely to rate nominees highly in times of unified government (when the ideological distance would be closer), and rate them less well in times of divided government (when the ideological distance would be farther). And, this would suggest the ABA's ratings are in part driven by the makeup of the Judiciary Committee, rather than based on the nominee's background.  Obviously, if we could create some measure of the ABA Committee's ideology that we could then compare to the nominee's ideology, that would be great.... Last thought: yes, determining whether the bias has switched over time is definitely a worthy question; the issue, as always, is getting the data.

Sara - thanks again for really pushing me/us to think about these issues!

Re: Chris's post yesterday, the only possible response is to say thank you for such a great suggestion, and the reminder to pay close attention to how one describes what their model actually does - and for providing us with all of the information on how to put that suggestion into action! We appreciate the guidance!! And, we'll let you know what we found as soon as we're done.

Welcome Amy Steigerwalt and Rich Vining

I'd like to introduce Amy Steigerwalt and Rich Vining, who will be guest-blogging with us this week about their paper on the ABA ratings, which has been generating a lot of media attention of late.  (In fact, look for Steigerwalt's interview with Fox News Monday morning and coverage of the paper in Fox's Special Report Monday at 6pm EST.)  Their paper can be found here.  They'll start the conversation with an overall summary and discussion of the paper, and then Chris Zorn and I will engage them in some discussion of their research.  Of course, we also welcome comments from all of you "out there."  We thought this might provide a nice venue to discuss the choices we make when we conduct empricial social science, while also discussing a paper that might actually catch the attention of the general public and has immediate real-world consequences. 

Rich Vining is Assistant Professor of Political Science at the University of Georgia.  He received his PhD from Emory University in 2008, conducting most of his research on the issue of retirements from the bench.  Amy Steigerwalt received her PhD in 2004 from Berkeley and is currently Assistant Professor of Political Science at Georgia State.  Her research most often involves judicial selection and the role of interest groups.  (Their coauthor, Susan Navarro Smelcer, is a doctoral student at Emory.  Her dissertation considers the decision to dissent in collegial courts.)  Welcome, Amy and Rich!  We look forward to the discussion!   

Guest Blogger: Robert M. Howard

I'd like to take this opportunity to introduce and welcome Bob Howard, who will be Guest Blogging with us this week.  Bob is currently Associate Professor of Political Science at Georgia State University and Editor-in-Chief of the Justice System Journal (published by the National Center for State Courts).   Bob holds a JD from Union College, a MA from Suffolk University Law School, and a PhD from SUNY Stony Brook.  His research often involves specialized courts and tax policy.  He's also written two papers with Jeff Segal expressly attempting to operationalize "law" and test for its influence (exploring the effect of originalism and judicial role perceptions, respectively, on Supreme Court decision making).  Hence, he is the perfect person to talk about issues in empirical legal research and its publication, and we are anxious to discuss the issues he raises.  Bob, welcome!

Guest Blogger Joe Doherty

The ELS Blog is pleased to welcome guest blogger Joseph Doherty, who currently serves as Director the UCLA Empirical Research Group (ERG).  Joe has a Ph.D. in political science from UCLA and is an expert in quantitative research design and methods. Through ERG he works with the UCLA School of Law faculty to conceptualize, design and execute empirical research across a broad range of legal fields.  Joe has advised or collaborated with scholars in studies that include bankruptcy, legal aid, administrative law, pollution prevention, tax policy, gay rights, the living wage, campaign finance disclosure, criminal law and immigration policy.  He has co-authored research on bankruptcy with Lynn LoPucki, on the living wage with Rick Sander, and on administrative law with Jody Freeman.  In addition, Joe is leading the new UCLA-RAND Strategic Alliance for Law and Public Policy, a collaborative project between the two institutions that includes research on corporate law and civil justice. 

Joe also specializes in voting behavior and campaign finance. From 2002 to 2006, he served as Project Director of the Campaign Disclosure Project (CDP).  In this capacity, Joe created an online database of state-level campaign disclosure laws and developed an innovative method for measuring the usability of campaign finance disclosure websites across states.  The CDP is a collaborative effort of the UCLA School of Law, the Center for Governmental Studies and the California Voter Foundation, and is funded by The Pew Charitable Trusts.  It has been credited by legislators and activists across the country for providing the impetus to modernize state laws and regulations, and to make more and better campaign finance data available on the internet. 

Finally, among empiricists in the legal academy, Joe is known as a good person who is always willing to offer this time and expertise to fellow researcher.  We look forward to Joe's comments.

Today's WSJ Op-Ed, "Mismatch" Hypothesis, and ELS Blog Forums

Today's WSJ Op-Ed page includes an essay (regrettably, a direct link is not possible w/out a  subscription) that relates to a recent ELS Blog Forum featuring work by Richard Sander (UCLA) and responses to that work by Richard Lempert (Michigan).

Thanks and So Long for Now (Blumenthal guest)

Thanks to the ELS Blog hosts for the invitation to return (and for extending my guest stint), and to readers who checked out my posts and didn't laugh.  I appreciate the opportunity to share some thoughts and look forward to continued discussion about ELS.  Thanks!  ----JB

Last One on ELS and Paternalism (Blumenthal guest)

One last point about ELS and paternalism, on the third objection I identified--that paternalism reduces people's opportunities to learn.  And with this one I'll be brief.

Again, there are at least two reasons to encourage learning from mistakes.  One is so that people come to know an objectively better solution to a particular question or issue or judgment; a second is that such learning can be a rewarding, valuable experience in itself.  I referred to Klick & Mitchell's recent work on paternalism, where they discuss the learning issue; David Shapiro phrased the second theme nicely some time ago, suggesting that "the very ability to choose--which necessarily implies the ability to make poor choices by some objective standard--is critical to the growth of our diverse intellectual, emotional, and volitional capacities."

Of course, for learning to be helpful in either case there must be not only the opportunity, but also the ability and the motivation to learn.  Well-known psychological research (e.g., reviewed here), as well as more recent neuropsychological findings, show that people are prone to confirmation biases or "motivated reasoning," where individuals prefer information that will support already held beliefs or attitudes.  Personality factors (such as dogmatism), knowledge factors, and social factors (such as our perceived similarity to someone who might make a similar mistake) influence whether we learn from our mistakes--the psychologist to whom Klick & Mitchell refer suggests in his review that because of such factors, "learning is by no means automatic when people are shown the errors of their ways."

And, of course, there must be a second chance.  The New Hampshire drivers who suffer fatal accidents because of not wearing a seat belt don't learn to use one the next time.  Some scholars have suggested that paternalism may thus be more appropriate in contexts where decisions can have "irreversible consequences."

I said I'd be brief.  And in the blog context, my goal is more to raise issues for discussion and empirical research than to refute or defend particular lines of argument.  But I would suggest that for the "learning" objection, as with the knowledge-of-preferences and autonomy arguments, empirical research--rather than just ipse dixits--will be helpful in furthering such discussions.

Conference: American Psychology/Law Society

The annual conference of the American Psychology/Law Society will be held on March 5-8, 2008, in Jacksonville, FL.  Announcement here; call for papers (Sept. 21 deadline) should be posted soon.  This year's conference (July 3-8, 2007) is being held in Adelaide, Australia, in conjunction with the European Association of Psychology & Law and the Australian and New Zealand Association of Psychiatry, Psychology and Law.

The APLS conference is a great opportunity for ELS scholars, as the presentations tend to be heavily in certain areas of psychology, and law school academics are often under-represented.  It's also a great way to build interdisciplinary connections.

Empirical Work on Property Issues (Blumenthal Guest)

The variety of topics to which ELS can profitably be applied--as shown, for instance, by the recent AALS conference and by the successful ELS conferences--is, of course, both encouraging and daunting.  As one perhaps atypical example, a month or so ago there was an interesting exchange on the PropertyProfs listserv, looking for examples of empirical legal articles examining property issues.  I am certain there are more than the ones identified, as this request focused on real property--and I hope readers might identify more, please!--but the ones that were mentioned are excellent examples of the sort of useful ELS work that can be done.  Thus, in no particular order, here are the ones mentioned:

• Joyce Palomar, The War Between Attorneys and Lay Conveyancers--Empirical Evidence Says "Cease Fire!", 31 Conn. L. Rev. 423 (1999)
• N. William Hines, Real Property Joint Tenancies: Law, Fact, and Fancy, 51 Iowa L. Rev. 582 (1966)
• N. William Hines, Personal Property Joint Tenancies: More Law, Fact, and Fancy, 54 Minn. L. Rev. 509 (1970)
• Michael H. Schill et al., The Condominium v. Cooperative Puzzle: An Empirical Analysis of Housing in New York City, available at SSRN: http://ssrn.com/abstract=502362
• Vicki Been & Ioan Voicu, The Effect of Community Gardens on Neighboring Property Values, available at SSRN: http:ssrn.com/abstract=889113
• Dean Lueck & Jeffrey A. Michael, Preemptive Habitat Destruction Under the Endangered Species Act, 46 J. L. & Econ. 27 (2003)
• Nicole Stelle Garnett, The Neglected Political Economy of Eminent Domain, 105 Mich. L. Rev. 101 (2006)
• Ann E. Carlson & Daniel Pollak, Takings on the Ground: How the Supreme Court's Takings Jurisprudence Affects Local Land Use Decisions, 35 U.C. Davis L. Rev. (2001)
• Leigh L. Linden & Jonah E. Rockoff, There Goes the Neighborhood? Estimates of the Impact of Crime Risk on Property Values from Megan's Laws, NBER Working Paper No.W12253 (2006)
• Robert Ellickson, Order Without Law (1991)
• Ward Farnsworth, Do Parties to Nuisance Cases Bargain After Judgment?: A Glimpse Inside the Cathedral, 66 Chi. L. Rev. 373 (1999)
• Mark D. West, Losers: Recovering Lost Property in Japan and the United States, 37 Law & Soc. Rev. 369 (2003)
• Robert H. Sitkoff & Max M. Schanzenbach, Jurisdictional Competition for Trust Funds: An Empirical Analysis of Perpetuities and Taxes, 115 Yale L.J. 356 (2005)
• Max M. Schanzenbach & Robert H. Sitkoff, Perpetuities or Taxes? Explaining the Rise of the Perpetual Trust, 27 Cardozo L. Rev. (2006)
• Lior Strahilevitz, How Changes in Property Regimes Influence Social Norms: Commodifying California's Carpool Lanes, 75 Ind. L.J. 1231 (2000)
• Lior Strahilevitz, Exclusionary Amenities in Residential Communities, 92 Va. L. Rev. 437 (2006).

More on Grades and Success in Big Firms (Sander guest-entry 3)

I've been slow to post a third entry this week because some of the comments on my first entry made me think more carefully about the table I posted Monday (entitled "White Michigan Grades at Private Firms with 100+ Lawyers) and the conclusions I drew from it.  I went back to the data and offer below a refined analysis.

There are two significant weaknesses in Monday's table. First, I included all Michigan alumni in creating the deciles, but of course many alumni did not return surveys. These non-respondents are part of the "base" on which the rates are calculated. So the author of the first comment, who noted that the proportion of Michigan grades joining firms seemed way to low, had a good point. (Among respondents, about 37% started at firms with 100+ lawyers and 47% started at firms with 50+ lawyers. The numbers would probably be higher today because firms are bigger.) Second, the time periods covered by the first two columns and the third column are significantly different; the first column looks at graduates from 1977 to 1990, while the third (15-year column) looks at grads from 1967 to 1985. Even though I noted the data were not literal attrition rates, many readers took them to be that.

After some thought about these limitations, I concluded that a better way to look at the association between GPA and success at big firms was to look directly at survival rates for big firm associates. The Michigan alumni surveys ask respondents how long they stayed at their first job after graduation, and how many jobs they've had. It's thus possible to see how many grads who started at big firms stayed at the same firm for the next fifteen years (and, in nearly all cases, became a partner at the firm).

The table below summarizes such an analysis. Those tracked here were respondents to the 15-year alumni survey from 1987 to 2000 who reported that their first job after law school was at a firm with 50 or more attorneys; these are divided into grade deciles, and tracked by whether they are still at the same firm. (I used a 50-attorney threshold because many of these attorneys started their careers in the 1970s, when a 50-attorney firm was considered "big".)

(a) Grade Decile	(b) White respondents to 15-year survey who started at firms with 50+ lawyers	(c) Those in (b) who were still with original firm 15 years later	(d) Those in (b) no longer at original firm	% Still at Original Firm (c/b)
1	25	1	24	4%
2	32	3	29	9%
3	58	7	51	12%
4	62	12	50	19%
5	94	13	81	14%
6	136	33	103	24%
7	157	26	131	17%
8	207	55	152	27%
9	236	56	180	24%
10	185	60	125	32%
Total	1192	266	926	22.3%

To illustrate how to read this table, look at the first row of data.  Of 1192 Michigan 15-year alumni who provided information on their first job and who started their career at a big firm, 25 (2%) had grades that put them in the lowest decile of all Michigan students.  Of those 25, only 1 was at the same firm 15 years later - a 4% "survival" rate (reported in the last column, as (c/b).

The last column thus gives us an idea of how grades affect tenure among those at big firms.  The relationship is quite strong -- the correlation of grade decile and the "survival" rate is more than .90.

With this added context, let me make some observations responsive to the posted comments and other emails I've received:

1) I don't contend that high law school grades themselves have a direct causal effect on success in big firms. I contend that high grades measure various things that are related to later success. Those things might conceivably include actual learning in various law classes; more likely, they include facility with legal analysis, cognitive skills and writing ability, and also such things as ambition, drive, lack of a social life, etc.

2) Obviously another factor that affects "survival rates" to partnership are choices about career, family, and lifestyle. Those choices are certainly very different for women and men, and they might vary across racial lines, too. But in The Racial Paradox, I showed that during the early years of associate experience (the years for which the After the JD project supplies good data), white men and white women had essentially identical experiences, while those of blacks and Hispanics diverged dramatically. So for women, attrition due to personal choices or to "glass ceilings" presumably sets in later.

3) Several of those posting comments suggested that minority patterns of attrition must be far higher than those explained by these grade variations. The Michigan data can't really answer this question, because the samples of nonwhites in big firms are very small (a total of 43 nonwhites in the 15-year alumni dataset reported starting at big firms). The numbers we do have for UMLS nonwhites are roughly consistent with those for whites:

(a) Grade decile range	(b) Nonwhite respondents to 15-year survey starting at firms with 50+ lawyers	Proportion of (b) at same firm 15 years later	Comparable rate for whites
1-3	30	10%	10%
4-6	9	22%	20%
7 or higher	4	25%	25%

Again, the sample sizes are far too low to draw conclusions from this second table; the close match of percentages is largely happenstance. And of course the dynamics facing nonwhites are very different. As I observed before, the nonwhites with low grades face two extra obstacles: first, they are likely to be at more elite and challenging big firms than are the whites with low grades; second, they are likely to be "stereotyped" as second-rate even when they are not. On the other hand, minorities may benefit from some affirmative action at the promotion and partnership level -- though clearly, if this happens, it is much less than the use of racial preferences in the hiring of associates.

As this second table suggests, nonwhite hires at big firms have far lower GPAs than white hires; nearly two-thirds of the nonwhites are in the lowest two deciles, compared to 5% of the whites.

The story told in these short blog entries is a brief summary of a more complicated story laid out in The Racial Paradox. I've tried here to grapple with the responses of those skeptical of the data.  My hope (perhaps a forlorn one) is that critics will go beyond mere skepticism to try and develop an alternative explanation consistent with all of the available data.

ELS and Paternalism #2: On ipse dixits (Blumenthal guest)

A second familiar objection to paternalism is based on autonomy: simply put, people's freedom to make choices, even non-optimal ones, should be valued per se.  Alternatively, as a a sort of mixed utility and autonomy argument, we should value people's preferences for the freedom to make choices.  That is, the freedom to choose should also be protected because people value that freedom itself.

The posting title refers to a line from John Robertson, discussing precommitment strategies (i.e., a form of self-paternalism): he suggests that potential gains from such strategies "should not be shunted aside with an ipse dixit about personal liberty."  And empirical research may bring into question this second general objection to intervention as well, in at least two ways (thus preventing an ipse dixit shunt).

First, it seems that in fact, people often prefer not to have to make decisions.  We prefer to leave our options open (to "leave doors open")--that is, to retain the option of changing our minds, even when that approach comes at the expense of decision quality.  In fact, the simple act of considering alternatives and choosing between options can increase the appeal of the foregone option.  We ask others to make decisions for us.  And, of course, we procrastinate--and, despite realizing that we procrastinate, and making efforts to correct that tendency, it seems that self-imposed deadlines are often ineffective.

Second, when we do make decisions, both the preference for choice, as well as having more options from which to choose, can lead to less utility and worse decisions.  Empirical research shows that although people often want a decision to be reversible--that is, they prefer the possibility of changing their mind--they are often less satisfied with their choice when it is reversible compared to when they make a decision and are "stuck" with it.  More broadly, recent findings call into question the traditional assumptions that choice generates utility and psychological well-being.  In fact, too much choice can be conflicting, difficult, and may lead to decreased experienced utility.  Thus, the assumption that people value making decisions is often mistaken; moreover, even when it is correct it can lead to decreased, not increased, utility.  (Indeed, in some instances marketers have picked up on this: Iyengar and Lepper cite an instance of Proctor & Gamble reducing the number of available versions of a shampoo brand from 26 to 15, and then "experiencing a 10% increase in sales."  Provocative but not conclusive.)

Thus, empirical findings raise questions about another aspect of the autonomy argument.  Again, these questions do not mean the argument is invalid, or that paternalistic policies necessarily trump any claim to individual autonomy.  The data simply suggest that the standard objections must meet higher hurdles than have heretofore been set, in order to fully persuade.

ELS and Paternalism (Blumenthal guest)

As I mentioned, I think there are at least three categories of objections to paternalistic intervention, and I wanted to identify empirical research relevant to those objections.  The first centers around the traditional claim that people know their own preferences and, as a result, are best able to identify those preferences and make their choices--certainly better than a third party.  Oft-repeated, the objection is, of course, associated with John Stuart Mill: "[W]ith respect to his own feelings and circumstances, the most ordinary man or woman has means of knowledge immeasurably surpassing those that can be possessed by anyone else."

A facile response is one used to justify intervention in the first place--"look at the bad decisions people make!"  In the health context, the prevalence of obesity has doubled over the last 30 years.  Smokers underestimate their risk of dying from lung cancer, despite overestimating the link between smoking and cancer.  In the market context, cognitive biases may allow sellers to manipulate buyers; erroneous estimates of the likelihood of one's own breach, or that of the other contracting party, may lead to parties bearing risks that they did not fully consider, or to failure to consider appropriate remedies for breach.  And, of course, many people make poor financial decisions in the context of pensions or investing.

But an anti-paternalist responds--among other things--that perhaps people are not being irrational; they in fact are identifying their preferences, and deriving pleasure even from objectively detrimental activities.

One reply points to the burgeoning literature on affective forecasting.  Substantial empirical research (recounted lucidly and entertainingly by Dan Gilbert in Stumbling on Happiness) shows that people are surprisingly inaccurate in identifying their emotional reactions to future events, both positive and negative, and as a result have difficulty knowing what makes them happy or unhappy.  This research is becoming more familiar in the law, and has been imported into the paternalism discussion (as well as a number of other contexts).  Briefly, though, such findings "challenge a fundamental assumption that [supports various] forms of autonomy in decision-making . . . , namely, the assumption that people are able to make choices in their own best interests."

Simply saying that people don't always know what's best for them isn't dispositive, of course, nor does it address the Millian notion that "I know my preferences better than someone else."  But there are empirical reasons to think that in some cases, experts--or simply third parties--do better.  This may particularly be so in the context of risk perception.  For instance, ordinarily, third parties over-estimate the risk preferences of others.  However, particularly in the context of negative events, individuals' over-optimism tends to reduce their perceptions of self-relevant risks.  In a sense, the over-estimation by third-parties could in fact be relevant to bring estimates closer to the truth.  More directly, Hsee and Weber's research shows that when the target of a risk judgment by a third-party is made salient--made visible, for instance, even without actual interaction--the over-estimates become more accurate.  Thus, it may be that simply identifying the target individual to the third-party may ameliorate over-estimation by that third-party, and may result in more accurate risk estimation than when the judgment is left to the individual.  Finally, of course, is the likelihood that a third-party may be less susceptible to immediately experienced factors that bias decisions (such as mood).  More objective, "calculation"-based decision-making tends to ameliorate emotion-based "probability neglect," and might warrant placing decisions in the hands of third parties less susceptible to such immediate emotional biases.

Of course, none of these findings mandates paternalism, nor do they conclusively refute objections to paternalistic intervention.  But they are helpful in moving the ball forward and incorporating more ELS information into the discussion.  To the extent affective forecasting research suggests that we in fact do not know what makes us happy, and to the extent the risk perception research suggests that perhaps third parties can do better, the Millian criticism of intervention must say more.

Some Good News About Women in Law (Sander guest-entry 2)

I wrote The Racial Paradox to compare minorities and whites, but one of the big surprises to me in writing the piece came from comparing white men and white women in big law firms. In reviewing data from the After the JD study (based on surveys from four thousand-odd new attorneys in 2002-03, including several hundred second- to fourth-year associates in big firms) I expected the "gender" gap in satisfaction and work experiences to be smaller than the "race" gap, but still noticeable. What I found were, more often than not, statistically identical patterns for the two sexes. 51% of white men were dissatisfied with their mentoring and 37% wanted more training, compared with 47% and 33% rates for white women. 47% of white men, and 47% of white women, were responsible for keeping the client updated on half or more of the matters they worked on. 25% of the white men, and 28% of the white women, supervised other attorneys or paralegals on half or more of their cases. Despite the many problems in big firms, here was evidence of something very positive.

To investigate further, I recently used the AJD to explore "dissimilarity indices" across gender lines. The dissimilarity index is a simple measure of how integrated or segregated two populations are from one another. If men and women are distributed across occupations in exactly the same way, the index is 0; if there is complete separation, the index is 1. In the American workforce generally, the level of gender occupational dissimilarity is in the neighborhood of .5 -- a number which has fallen some in the past generation, but remained stubbornly high despite the broad influx of women into the workforce.

Traditionally, men and women lawyers have been at least somewhat segregated in two ways: the type of employer they go to work for, and the area in which they specialize. Women have tended to be overrepresented in government and non-profits; men have concentrated in private firms, especially (compared to women) in smaller firms. Within firms, women were more likely to work in family law or probate; men dominated many of the commercial and "big business" fields.

That seems to be changing.  In the AJD sample, the breakdown of men and women across sectors looks like this:

Setting	% of AJD women in setting	% of AJD men in setting
Very small firms (<5 lawyers)	10.1%	12.9%
Small firms (6-20 lawyers)	13.5%	14.4%
Medium-sized firms (21-100)	10.5%	11.3%
Large firms (101-299)	8.2%	8.7%
Very large firms (300+)	19.5%	20.3%
Government / public interest	25.3%	16.7%
Corporate, misc.	8.7%	9.7%
Solo	4.3%	6.2%

The differences have not vanished, but they are generally quite small. Overall, the index of dissimilarity from this data is .09.

If we look at areas of legal specialization, the story is (to me) even more surprising. I analyzed the distribution of AJD men and women across 24 specialized areas, adding up the proportion of time each attorney said they spent in each area. The totals are remarkably similar for men and women; here's a partial summary for associates at very big (300+ attorney) firms:

Area of practice	% of total time spent in area for:
	Women at big firms	Men at big firms
Civil litigation	21.7%	19.8%
Commercial law	5.9%	5.2%
General corporate	12.0%	13.1%
Intellectual property	7.8%	12.8%
Family law	0.2%	0.2%
Real estate commercial	4.0%	3.6%
Securities	8.2%	11.2%

Overall, the male/female index of dissimilarity in practice areas for associates at these very large firms is .13. For all eight practice settings, the average index is .18.

To get some sense of how much has changed, I examined the Chicago Lawyers dataset, which is based on the extensive interviews by Heinz et al of some eight hundred Chicago attorneys in 1994-95. I created two comparison groups: lawyers who started their careers before 1982, and lawyers who started their careers between 1988 and 1993 (this second group would thus be similar, in terms of the stage of their careers at the time of data collection, to the AJD lawyers.) Here are the results:

Lawyers compared	Index of dissimilarity across practice settings	Index of dissimilarity across 24 fields of law (average index across eight practice settings)
Chicago lawyers entering practice before 1982 (interviewed in 1994-95)	.37	.60
Chicago lawyers entering practice between 1988 and 1992 (interviewed in 1994-95)	.26	.35
AJD lawyers entering practice in 1999-2000 (surveyed/interviewed in 2002-03)	.09	.18

Some of the Chicago field-indices are a bit skewed (biased upward) because there were relatively few women in some settings. And it may be a little misleading to compare Chicago data with national data. But the trend suggests that something remarkable has happened. Over the course of a generation, it seems, gender segregation in the law shifted from being very high to being relatively low. Young women lawyers, at least, seem to be converging with young men lawyers both in their choice of paths and their ability to "get on" those paths.

None of this is to deny that women of all races are securing big firm partnerships at low rates. The first wave of the AJD (looking at lawyers 2-4 years out of school) does not allow us to understand what happens at partnership time (the second wave, now underway, should), but the problems are foreshadowed in the AJD data. Women associates at big firms are more concerned about the long hours and the conflict between their job and their family life; their attrition rates are low during the early years with the firm, but a smaller proportion of women than men see a long-term future with the firm. Those are important issues, not addressed in this note. But they should not obscure the remarkable convergence of men and women lawyers in their early careers.

Seat Belts, Paternalism, and New Hampshire--"Live Free and Die?" (Blumenthal guest)

The New Hampshire State Senate recently rejected a bill that would have mandated seat belt use for all adults.  According to the media, New Hampshire is the only U.S. state without some such law.  The discussion involved, unsurprisingly, issues of paternalism and "being told what to do"--squibs from earlier stories in the New York Times noted that "For many residents, buckling up and being required to buckle up are 2 different things," and one proponent of the bill noted that the debate "harkens [sic] to the libertarian 'don't tell me what to do' streak that characterizes much of [NH] politics."  The Times stories also noted that "New Hampshire has the lowest rate of seat belt use in the country, 49.6 percent," and that "77 percent of fatal crashes in the state involved occupants who were not wearing seat belts."

Paternalism, of course (whether asymmetric, libertarian, emotional, or other), is increasingly discussed in legal academia.  Much of the connection to ELS is in the context of evaluating whether particular interventions (as just one instance, Thaler and Benartzi's SMarT investment approach) in fact improve welfare on some specific outcome.

Less empirical analysis, I think, goes to the justification of such "paternalistic" intervention, and I hoped to post a few comments in that context, using the New Hampshire example as a vehicle [sorry, pun intended] for mentioning empirical research relevant to some of the usual objections to "telling someone what to do."  Existing empirical work may indicate that we need to look a little harder at such objections--and I think providing empirical grounds for such a discussion is a fruitful effort for ELS in any context.

Offhand I can imagine at least five such objections, falling roughly into three categories.  First is the classic Millian notion that people know their own preferences, and as a result are best at identifying and choosing their own preferences.  Second is the classic autonomy argument: people's freedom to choose should be valued per se ("don't tell me what to do!") ; relatedly, people's preference for the freedom to choose should be valued per se.  Third, people learn from their mistakes, and should therefore be allowed to make them--perhaps simply for the substantive benefit of learning what is correct, perhaps to "build character."  Jon Klick and Greg Mitchell get at this third category in their recent response to libertarian paternalism (among many other points, of course).

(At least two other objections, mentioned recently by Ilya Somin on the Volokh Conspiracy, involve the accountability of chosen "experts" who make paternalistic decisions and whether those experts should get the final word.  A related one, of course, involves public choice and capture.  I'll set those aside to the extent that they involve evaluation of the means of effectuating interventions once it's decided that paternalism is appropriate [except where Somin, for instance, notes that "Although the expert is more knowledgeable than I am about technical issues in his field, I am more knowledgeable than he is about my own values"--the first category identified above].)

Again, my goal is not to take sides, but rather to point out another instance of ELS's potential to advance a substantive policy discussion.  In short upcoming posts I'll try to identify some empirical research relevant to each of the five objections above.

New Evidence on GPA and Success In Law Firms

Can better data resolve deeply-felt issues? My entry today might be viewed as a little experiment on this question. But first, some background.

Last summer, I published The Racial Paradox of the Corporate Law Firm, a piece that explored the fate of minorities at big, elite law firms. I pointed out that while minorities are very well-represented among the starting associates at these firms, blacks and (to a somewhat lesser extent) Hispanics have far higher attrition rates than whites, leaving few black and Hispanic senior associates around when partnership decisions are made. I laid out various possible theories to explain these high attrition rates, including two different ways discrimination might affect the outcomes. And then I laid out a variety of new and fairly detailed data on the characteristics and experiences of new associates. Though I found much of the evidence consistent with more than one theory, I concluded that it was very likely that the aggressive use of racial preferences by law firms contributed to the problem. There was no doubt that firms were granting enormous preferences – in one large dataset of new associates at big firms, the median black GPA was at the 18th percentile while the median white was at the 75th percentile. Wouldn't a big disparity in grades produce a disparity in work quality? It was hard to explain many of the patterns otherwise:

--Black and Hispanic associates had experiences that closely matched those of white men on the “external indicia” of their jobs -- things like salary, hours, and representation on firm committees. But they tended to get fewer work assignments, were given less responsibility on those assignments, and were far less satisfied with the training and mentoring they received, compared with white male associates. Importantly, the problems a group had in these areas closely matched the size of their credential gap with white male associates. Blacks reported the most serious problems, and received the largest hiring preferences, then Hispanics, and then Asians. White women, who appeared to have exactly the same credentials, on average, as white men, reported virtually no differences from white men in mentoring, work assignments, or levels of responsibility -- and indeed reported very high levels of satisfaction.

--Smaller law firms appeared to be much less likely to use large racial preferences in hiring associates. I could not find any statistically significant difference in the grades of black and white associates, for example, though probably there's some disparity. Correspondingly, blacks at small firms generally are far more satisfied on issues like mentoring and work responsibility than their counterparts at big firms, and by most indicators, there's not a significant black-white gap in small-firm experiences.

Certainly, if big-firm racial preferences in hiring produce differences in performance that correlate with race, they will also produce invidious negative stereotypes in the minds of big-firm partners that could further hold back strong minority associates. But, I argue, if firms use radically different GPA thresholds for different racial groups in hiring, we ought to expect both differences in performance and negative stereotyping.

From the time I first presented this paper to a symposium at the University of North Carolina, it has evoked two types of responses. The first reaction -- always very vocally expressed -- is that my entire argument unjustifiably assumes that grades have something meaningful to do with the on-the-job performance of lawyers. The second reaction -- always expressed quietly, usually one-on-one and often from lawyers who have sat on the hiring or promotion committees of big firms -- is that my article gets it exactly right, but that any firm which publicly said so would expose itself to a lawsuit.

Public discussion of the article tends, therefore, to be a bit skewed. In November, after the New York Times ran a careful, thoughtful piece about The Racial Paradox, it ran a fluffy follow-up piece in the Week in Review, in which education reporter Jonathan Glater asked various law firm partners how important grades were for success at their firm. A typical response was 'Gee, I never would have made partner if they'd looked at my grades.'

This is disingenuous in two ways. I certainly don't contend that firms look at law school transcripts when they decide which associates to promote to partner. My point is that, other things equal, associates with high GPAs will tend to perform better, and thus will tend to build stronger reputations at the firm and succeed more often. And if this isn't true, then why would big firms set such high GPA standards in their entry-level hiring, and why would they pay such high salaries to lure the highest-GPA students?

The Racial Paradox did present some data showing that higher-GPA students tended to be more successful at big firms (e.g., Table 13), but I realized after the article was published that my presentation of the data was somewhat too abstract to really drive the point home. With further analysis, I've developed a clearer way to show the underlying patterns. The data below comes from the University of Michigan Law School's famous alumni surveys. UMLS has for many years sent out questionnaires to its alumni five and fifteen years after graduation. It links this data to actual transcript information, so that we know how each respondent performed in law school. By looking at this data, only for whites, we can accurately see the effect of grades on outcomes without the usual confounding effects of race and law school.

White Michigan Grads at Private Firms with 100+ lawyers (per 1000 graduates)

Grade Decile	Joining big firms after graduation	At big firms, 5 years out	At big firms 15 years out (gen'ly partners)	Partnership rate (per 1000 grads)
1	39	40	16	1.6%
2	80	68	15	1.5%
3	126	92	32	3.2%
4	168	130	29	2.9%
5	177	149	55	5.5%
6	190	167	86	8.6%
7	243	200	103	10.3%
8	247	219	123	12.3%
9	239	235	127	12.7%
10	221	237	169	16.9%
Number of cases	1731	1537	755
Median percentile	63	66	73

For this analysis, I grouped the white alumni from UMLS into ten equal groups based on their actual law school GPA. The table shows the rate at which these graduates (at rates per 1000 grads) worked at firms with 100 or more lawyers. Just after graduation, for example, white UMLS grads with grades in the 3rd decile had a 12.6% chance of working at a big firm (126 per 1000), while grads in the 10th (top) decile had a 22.1% chance -- a reflection, no doubt, of the easier time high-GPA grads would have getting jobs in big firms. The next two columns show how many grads were still at big firms 5 and 15 years after graduation. (I haven't tracked individual career histories for this analysis, so it is not identical to an attrition study, but it is very similar to one.) There is some erosion in numbers by the 5th year, and much more erosion between the 5th and 15th year (by which time partnership decisions have been made). And the rates of erosion are very highly correlated with GPA. Thus, for every 126 grads in the 3rd decile who are big firms in Year 1, there are 32 by year 15 (one-fourth as many). In the top grade decile, the Year 15 numbers are still about three-quarters as large as the Year 1 numbers. Indeed, in this population, there is nearly as much sorting by GPA after the initial hiring process as there is during the initial hiring process.

In other words, this data shows clearly that GPA matters a lot to one's success and longevity in the world of big firms. And, since this data is just for whites, we know that discrimination is not playing a role in these outcomes (unless, as seems unlikely, firms do pay attention to transcripts after hiring).

Moreover, if we think about this data carefully, we'll see that low GPAs will probably have even more harmful effects when they arise from racial preferences. In the table above, I can't distinguish between different sorts of big firms. But it is likely that the high-GPA whites are at tougher, more competitive firms (on average) than are the low-GPA whites. Their success rates are swimming upstream against more competitive environments and lower partnership rates. Naturally, low-GPA whites have both a better chance of being hired, and of succeeding, at a firm which is big but not super-elite. If we could control for firm quality, the GPA disparities in outcomes in the chart would probably be much greater.

For racial minorities, this caveat does not apply. The most elite firms are the ones most aggressive in using racial preferences, so huge grade disparities divide races within the same firm. So just how, exactly, can one argue that "grades don't matter" in the face of this data?

Guest Blogger: Richard Sander

This week, the ELS Blog is pleased to welcome back guest blogger Richard Sander, who is an economist and law professor at UCLA School of Law.  Sander is best known for his empirical work on racial preferences in legal education (Systemic Analysis of Affirmative Action in American Law Schools, 57 Stan L. Rev. 367 (2004)) and corporate law firms (The Racial Paradox of Corporate Law Firms, 84 N.C. L. Rev. 1755 (2006)). 

Although this work has garnered no shortage of critics, Sander's earlier scholarship and professional activities belie the perception that his work is motivated by politics or ideology.  During the last 20 years, Sander has been a ardent advocate of fair housing laws and worked on the successful campaign and transition team of Harold Washington, the first black mayor of Chicago.  Since joining the UCLA faculty in 1989, Rick has served as President of the Fair Housing Congress of Southern California, founded the Fair Housing Institute, and helped the City of Los Angeles design and implement what was, at the time, the nation's most ambitious living wage law. He also participated in the effort to develop outreach programs that sharply increased the local use of the Earned Income Tax Credit, generating tens of millions of dollars annually for LA's poorest working families. 

In addition to scholarship on housing integration and living wage laws, Sander has done important empirical work on legal education, including effective strategies in academic support programs and how law students perceive the law school experience.  At UCLA, Sander teaches courses in Property, Quantitative Methods, Urban Housing, and Policy Analysis.  We look forward to his postings over the coming week.

Follow-up on Meta-Analysis

Steven Erickson has thoughtfully raised some issues / concerns about meta-analysis in the Comments section of my previous post.  Comments often can get lost, so I wanted to bring attention to that other perspective.

Four Cheers for Meta-Analysis (Blumenthal)

One wonderful thing about this blog is the opportunity to learn about various different methodological approaches available for data analysis.  The workshops on methodologies and statistics (e.g. Northwestern's) are enormously helpful, and references for teaching and catching up on statistics have been as well.  Experimental work is very important, and relatively uncommon approaches such as MDS, network analysis, propensity score analysis, and others can give good insight into what is going on with both conventional and unconventional data.

But I'd like to highlight one approach that cuts across many of these, and in my view is one of the most important contributions ELS can make.  I'm a big fan of meta-analysis--of the quantitative synthesis of existing research studies--and advocate its use by ELS scholars, courts, agencies, practitioners, ice cream truck vendors, etc.

I've made some of my case for meta-analysis in a piece coming out in Temple Law Review, so I will be brief here.  First, descriptively, meta-analysis is just that--analysis one level up.  That is, primary research uses individual units (person, cases, courts, etc.) as the unit of analysis.  The individual units that meta-analysis involves, though, are the empirical studies themselves from a particular body of research.  Synthesizing the results of each study, the goals of meta-analysis are (1)  to identify the presence or absence of an effect in an existing empirical literature; (2) to evaluate the strength of that effect, for instance by summarizing the average effect across a set or subset of studies; and (3) to identify moderator variables, elements of the various studies that might have reliably affected their outcomes.  Thus, a good meta-analysis will identify and synthesize every study in a research area; find the average effect across all those studies in order to summarize the most current state of knowledge; and then systematically compare and contrast across studies, in order to identify methodological and substantive aspects of the various studies in a discipline that might be associated with the effect sizes they report.

Meta-analysis has a number of benefits, especially relative to the traditional narrative lit review: it is more comprehensive; it better avoids subjective judgments about what to include in a review; it is better at identifying "small" effects; it avoids over-emphasis on misleading statistical significance and p-values (focusing on effect sizes instead); emphasizes moderator variables; and, in my view, gives a better grounding for policy inferences.

Meta-analyses, unfortunately, are still rare in law reviews (Chris Guthrie and Dan Orr had one recently, but few others appear; they are cited sometimes, though).  They are used in court, usually in mass tort cases, but are often misunderstood by courts and experts.  My hope is that use of meta-analyses in ELS will increase--they serve well to summarize a body of research; they allow (or force) researchers and policy-makers to speak a "common language"--by making cherry-picking from existing studies more difficult--and, identifying moderator variables, they can prompt further research questions.

Blumenthal: Thanks for the Welcome

Thanks for the welcome!  I hope any upcoming posts won't drive readers back to grading . . . .

Guest Blogger: Jeremy Blumenthal

Since he did a great job in early March, Jeremy Blumenthal has returned as a guest blogger this week.  Jeremy is an Assistant Professor of Law at Syracuse University College of Law.   He earned his A.B., A.M., and Ph. D. in Social Psychology from Harvard University. He earned his Juris Doctor and graduated cum laude from the University of Pennsylvania Law School, where he was Editor-in-Chief of the University of Pennsylvania Journal of Constitutional Law and received several writing prizes. Most recently, he was a Faculty Fellow at Seton Hall Law School, teaching 1L Property and Law and the Social Sciences.  Previously, he clerked for the Honorable Maryanne Trump Barry of the U.S. Court of Appeals for the Third Circuit, and was an Associate for Latham & Watkins, LLP, in the Newark, NJ office.

Professor Blumenthal has published widely on topics in law and psychology, including demeanor evidence, the reasonable woman standard, victim impact statements, and more generally on the role of social science in assisting the legal system. He has also published in the areas of evidence and comparative law. His most recent publications focus on the role of emotions in the legal system; current research projects address topics in capital punishment, empirical work in law and psychology, and topics in property theory.

Since he did a great job in early March, Jeremy Blumenthal has returned as a guest blogger this week.  Jeremy is an Assistant Professor of Law at Syracuse University College of Law.   He earned his A.B., A.M., and Ph. D. in Social Psychology from Harvard University. He earned his Juris Doctor and graduated cum laude from the University of Pennsylvania Law School, where he was Editor-in-Chief of the University of Pennsylvania Journal of Constitutional Law and received several writing prizes. Most recently, he was a Faculty Fellow at Seton Hall Law School, teaching 1L Property and Law and the Social Sciences.  Previously, he clerked for the Honorable Maryanne Trump Barry of the U.S. Court of Appeals for the Third Circuit, and was an Associate for Latham & Watkins, LLP, in the Newark, NJ office.

Professor Blumenthal has published widely on topics in law and psychology, including demeanor evidence, the reasonable woman standard, victim impact statements, and more generally on the role of social science in assisting the legal system. He has also published in the areas of evidence and comparative law. His most recent publications focus on the role of emotions in the legal system; current research projects address topics in capital punishment, empirical work in law and psychology, and topics in property theory.

Guest Blogger: Timothy Johnson

This week's guest blogger is Timothy Johnson, Professor of Political Science at the University of Minnesota.  Tim's research and teaching interests include American Politics, judicial politics, Supreme Court decision-making, Supreme Court oral arguments, executive/judiciary relations, and the evolution of the norm of respecting precedent. He is the author of Oral Arguments and Decision Making on the U.S. Supreme Court (SUNY Press, 2004) and the coauthor of Religious Institutions and Minor Parties in the United States (Praeger Press 1999, with Chris Gilbert, David A.M. Peterson, and Paul Djupe).

Tim's work has also been supported by the National Science Foundation. His first grant (IIS-0324992), 'ITR SCOTUS: A Resource for Collaborative Research in Speech Technology, Linguistics, Decision Processes, and the Law' (with Jerry Goldman, Brian McWhinney, and Mark Liberman), is an effort to digitize all U.S. Supreme Court oral arguments from 1955-present, with an emphasis on using the audio to analyze how oral arguments affect the Court's decisions. The second grant (SES-0550276), 'Collaborative Research: The Establishment of Stare Decisis in the American Legal System' (with James F. Spriggs II and Paul J. Wahlbeck) focuses on the examination of the how the norm of respecting precedent affects the decisions of judges within the American legal system.

Studying Affirmative Action: What’s Going On?

In defending his Stanford Law Review article, Richard Sanders has more than once said that “It takes a theory to beat a theory,” or words to that effect. The implication is that he has offered the only theory, what he calls the “mismatch theory,” to explain the performance of black students in the Bar Passage Study Data and that his theory should stand (with the further claim that policy should be based on it) until a better one comes along. I don’t accept Professor Sander’s claim about what makes for accepted theory, even if he is quoting eminent scientists, for the lack of competition does not argue for accepting a theory poorly supported by data. The well known “theory” that the earth is held in place because it rests on the back of a giant turtle should, for example, be rejected even if no other explanation for the earth’s not falling through the firmament is advanced, simply because there is no evidence apart from assertion and the fact that the earth seems stable in space to support the theory. Nor did it take a better theory to debunk the Utah group'sclaim of having generated cold fusion; all it took was numerous failures to replicate their results. Still it is fair of Professor Sander and others to ask what does explain the relatively poor performance of black law students as revealed by the data collected in the Bar Passage Study, and what might be done about it. These are questions that were posed to Professor Sander and me in June of 2006, when we appeared jointly before the United States Commission on Civil Rights. Below is the answer (slightly edited, with footnotes omitted) that I gave in my written testimony to the USCCR. (Entire testimony available on request to [email protected].

Studying Affirmative Action: Some Unappreciated Complexities

One issue that has not been disputed to date in the litigation on race-based affirmative action has been whether schools that were sued to stop the preferential admission of minorities in fact engaged in affirmative action. In every case the schools sued had policies that espoused affirmative action, and they admitted they were following these policies. But suppose they had not admitted to affirmative action admissions. Could the plaintiffs have proved that the defendant schools were engaging in race-based affirmative action?

The question is not purely academic, or at least it might not remain so. A number of states, my home state of Michigan among them, now preclude race-based affirmative action, and the word is that those opposed to affirmative action plan ballot proposals to ban affirmative action in numbers of other states. Judging from recent referenda, if such proposals make the ballot, they are likely to pass. If they do, or where they have, should it appear that there are more blacks, Hispanics, Native Americans, or members of other minority groups on campus than race-blind admissions would bring, law suits might well ensue. Unlike earlier litigation, however, plaintiffs would be attempting to prove facts – the existence of forbidden affirmative action – rather than seeking to argue the constitutionality of an admittedly ongoing process. How hard will it be to do this?

Studying Affirmative Action: Correcting the Record

I would like to begin by thanking the Editors of the ELS blog for inviting me back for a repeat performance. I am going to use this opportunity mostly to talk about issues that arise in empirical investigations of affirmative action. I will use this kick-off contribution, however, to talk specifics and to correct misinformation that exists in the literature, which was unfortunately widely publicized in the New York Times. This misinformation stems from an article that Professor Richard Sanders published in the North Carolina Law Review that examines the career paths of black attorneys who begin their careers at large (100 lawyer +) law firms and reports that that these young attorneys leave large firms at very high rates relative to whites. This finding is later used to question the skill levels of young black attorneys and as evidence of adverse consequences, both in law school and thereafter, to affirmative action.

I wish to be clear at the outset that although I think Professor Sander’s argument misleads because crucial data are wrong, I do not mean to criticize Professor Sander for using these data when he did. At that time the data Professor Sander relied on seemed to be the best publicly available data, even if they were not the best suited to his purpose. Moreover, not only do the assumptions he made to justify relying on his data appear at least plausible, but he also attempted to check his most crucial inference with other publicly available data. Indeed, it is precisely because his efforts to be careful lend plausibility to his use of the data and consequent conclusions that it is important to correct the data record.

In his North Carolina Law Review article, Professor Sander suggests that black associates leave large (100 lawyer +) law firms in their first few years and before the partnership decision at dramatically higher rates than whites. While there are a variety of plausible explanations for this, including the arguments that blacks suffer from prejudice and hostile atmosphere and the claim that most young large firm associates are looking for ways out and blacks simply have more attractive opportunities to leave than whites, Sander’s preferred explanation for at least a large part of this disparity is that black associates are less well equipped to do the work large firms demand than whites and that this is in large measure attributable to law school and law firm affirmative action.

Guest Blogger: Richard Lempert

This week, we are pleased to announce Richard Lempert (Michigan Law School) as a guest blogger, who returns to us following an interesting and provocative set of posts back in August 2006.

Professor Lempert is particularly concerned with the problem of applying social science research to legal issues. This is reflected in much of his work, particularly his work on juries, on capital punishment and on the use of statistical and social science evidence by courts, as well as in his service as an original panelist in the National Science Foundation's Law and Social Science Program and with the National Research Council's Committee on Law Enforcement and the Administration of Justice, which he chaired from 1987-89. He is the author (with Joseph Sanders) of An Invitation to Law and Social Science and co-editor (with Jacques Normand and Charles O'Brien) of Under the Influence? Drugs and the American Work Force. From 1982-85, he edited the Law & Society Review.  He is a recipient of the Law & Society Association's Harry Kalven Jr. Prize prize for outstanding socio-legal scholarship and has held visiting fellowships at the Center for Advanced Study in the Behavioral Sciences and at the Russell Sage Foundation. In 1993 he was elected a fellow of the American Academy of Arts and Sciences. He currently serves as the secretary of Social, Economic, and Political Sciences Section (K) of the American Associate for the Advancement of Science.

Guest Blogger: Erin Kaheny

This week's guest blogger is Erin Kaheny, Assistant Professor of Political Science at Wisconsin-Milwaukee.  Her research interests include federal appellate court decision making, procedural rules of access to the legal system, judicial careers and acclimation effects, divided government and judicial selection, agenda change in the U.S. Courts of Appeals, and comparative judicial studies.  Recently, her paper draft entitled Have We Come a Long Way Baby: Female Attorneys before the United States Supreme Court has garnered much attention by both blogs and the media (perhaps she'll discuss this).

Guest Blogger: David Stras

This week's guest blogger is David Stras, Associate Professor of Law at the University of Minnesota.  Professor Stras teaches and writes in the areas of criminal law, federal courts and jurisdiction, constitutional law, and law and economics. His current research focuses on the federal judiciary and the Supreme Court of the United States.  Using a variety of methodological tools, including empirical and historical analyses, Professor Stras's research has examined a variety of issues relating to the Supreme Court.  Professor Stras is a frequent television and radio commentator on issues relating to the federal judiciary, particularly with respect to the Supreme Court of the United States.  He is also an adjunct faculty member of the University of Minnesota Political Science Department, and is the co-editor of a new SSRN journal on "Law and Politics." 

Professor Stras received his B.A. degree, with highest distinction, and his MBA from the University of Kansas. He received his law degree from the University of Kansas School of Law in 1999, where he served as Editor-in-Chief of the Criminal Procedure Edition of the Kansas Law Review. Following law school, Professor Stras clerked for The Honorable Melvin Brunetti of the United States Court of Appeals for the Ninth Circuit and then for The Honorable J. Michael Luttig of the United States Court of Appeals for the Fourth Circuit. From 2001 to 2002, he practiced white-collar criminal and appellate litigation with the Washington, D.C. office of Sidley Austin Brown & Wood. Following his year in practice, he clerked for The Honorable Clarence Thomas of the Supreme Court of the United States.

In addition, he a frequent and regular contributor to SCOTUSblog.

Multidimensional Scaling - Blumenthal #5

Another useful methodology often used in psychology -- less in social, more in cognitive -- is one alluded to here before, "multidimensional scaling" (MDS).  MDS has only been used in a small number of legal analyses, but can be quite helpful in the right context.

As I describe in a forthcoming paper, "MDS is a procedure that helps researchers uncover 'hidden structures' in data by graphically plotting respondents' perceptions of perceived similarities (or dissimilarities) among various stimuli.  When these stimuli are located on a plot based on such perceptions, underlying dimensions that respondents may have used (consciously or not) can be inferred."

So, for instance, a bunch of stimuli are identified -- crimes, Supreme Court opinions -- and every pairwise combination is created.  Each pair is rated on how similar or close they are (though the researcher usually does not articulate the scale to be used, just asks "how similar"?).  Those similarity ratings are then plotted to give the best fit to the data, yielding a "map" of the ratings that also reflects the perceived structural relationship of all those stimuli.  Finally, by examining the map, one might be able to identify the dimensions that, implicitly or explicitly, underlay subjects' ratings (and also, interestingly, look at individual differences, e.g., do men and women or Republicans and Democrats come up with different maps or weight those dimensions differently?).

As a specific, somewhat prosaic example: subjects might be asked to rate the similarity of countries in the world, and a plot made of the countries based on those similarity ratings.  The resulting map might look like a map of the world, which could suggest that subjects were, implicitly or not, using dimensions involving distance, in particular, dimensions of north/south and east/west.  But the plot might line up the countries on a single axis going from largest to smallest, or richest to poorest, or most to least economically developed, suggesting that subjects used that single dimension in their ratings.  Inferring which dimensions are important to respondents helps understand the way they see the world (literally, here), and what factors are important to them.

In the legal context, I used it in that paper to map people's perceptions of the seriousness of crimes.  MDS is broadly useful for looking at lay intuitions about legal institutions, punishments, or cases; or to assess whether different punishments may have been proportionate; or any context in which (a) similarities among stimuli may be interesting or (b) we are interested in understanding what the basis might be for people's judgments or perceptions.

A classic book on MDS is by Kruskal and Wish, from the 1970's.

Insights from the Methodology of Psychology (Blumenthal #4)

Picking up on the methodological contribution I think psychology can make to ELS, it seems as though psychology emphasizes experimental work to a greater degree than current ELS. Discussing the excellent methodology workshop that Lee Epstein and Andrew Martin run, someone recently joked that it covers 95% of all the methodology ELS scholars need to know. I’d quibble with that if, as I understand (without having attended), there’s little coverage of experimental design and analysis (e.g., analyses of variance or ANOVA’s). To be very clear, all I mean by that is I think the experimental approach could be a larger proportion of ELS work than it currently is.

Experimental research, of course, can contribute greatly to our understanding of a number of topics and phenomena (jury understanding of sentencing instructions; eyewitness accuracy; racial discrimination by judges and jurors; effects of policy interventions; different influence of legal procedures both in and out of the courtroom, etc., etc.). Researchers can manipulate specific variables and test their causal effect, tweaking particular aspects to really parse out specific effects that may be more precise than multiple regression. In some instances – when a researcher is working with existing databases – experiments can be less constrained by data limitations, can be more proactive in data collection.

Among the downsides, of course, is the perception that experiments aren’t “externally valid” – they don’t sufficiently reflect the real world that we can profitably generalize. Sometimes, true. But they sometimes do, and sometimes can be conducted in real-world settings. Other times, finding an effect in the lab suggests it might be even stronger in the real world: the example often given is with comprehension of capital sentencing instructions – when highly educated mock juror subjects from “elite universities” don’t understand them, how much more of a concern might it be for less educated jurors faced with such instructions in the real world? In yet other contexts, again with mock juries, differences between undergraduate samples and community member samples are substantially less than critics have suggested (Bornstein, 23 Law & Hum. Behav. 75 (1999)).

I don't want to oversell the experimental approach, but it could certainly be another arrow in the ELS quiver.  So to speak.

Blumenthal #3 - More on Psychology

As a follow-up, I would point out some excellent psychology-and-law resources (sorry; now that I teach it at a law school, it's "law-and-psychology" - sort of emphasizes the point, doesn't it).

First is the American Psychology/Law Society, founded in the late 1960's; with annual conferences highlighting the most current and topical research in legal psychology.  The website has links to various newsletters of the Society, which include conference programs in order to see who is doing what sort of empirical research.

Second is the Society-sponsored, peer-reviewed journal, Law and Human Behavior, published by Springer.  Publishing top-quality empirical research, LHB is consistently one of the most highly-cited non-law-review journals.  The topic areas are often limited to the "traditional" psych-and-law topics I've mentioned before, though - forensics, eyewitness, juries - and I am sure that its editors would welcome a broad range of submission topics of the sort that ELS people here investigate.

Both APLS and LHB are good resources for finding primary research and empirical, interdisciplinary collaborators (JD/PhD's are not uncommon in APLS) - IMHO, both can help as the ELS field continues to develop.

Blumenthal # 2 - A Plea for Psychology (with apologies to John Austin)

A Plea for Psychology (with apologies to J.L. Austin)

So I've gotten off to a slow start, but I'll try to make some contribution now.

As I mentioned, I wonder if psychology as a discipline has gotten short shrift from ELS. As a historical matter I think so; research in legal psychology has been conducted, off and on, for about 100 years. But until recently the legal system has been skeptical, suspicious, at times hostile to psychology, and there’s not much discussion of psychological research in modern ELS – with exceptions, much of the work stems from economics or political science.

There are historical reasons for this skepticism. The conventional story is that psychology and law as a discipline was founded around 1908, with the publication of Hugo Munsterberg’s On the Witness Stand, documenting his efforts to persuade the legal system that his empirical work in criminal justice – eyewitness research, memory, perception, lie-detection, etc. – was relevant and important. (Another interesting contribution, written in a more scholarly vein, was a book by G.F. Arnold, a British legal scholar, Psychology Applied to Legal Evidence (1906)).

Munsterberg's book was pugnacious and had a lot of 'tude -- probably too much ("The time for Applied Psychology is surely near. . . .  The lawyer alone is obdurate;" "The lawyer and the judge and the juryman are sure that they do not need the experimental psychologist. . . .  They go on thinking that their legal instinct and their common sense supplies them with all that is needed and somewhat more.")  Hugo's book prompted a now-classic response article by John Henry Wigmore, in which Wigmore staged a mock lawsuit accusing Munsterberg of libeling the legal profession.  Both tongue-in-cheek and vitriolic, Wigmore's cross-examination devastated poor Hugo, and as a result, one commentator has said, for years American psychologists "left the law rather severely alone."  Last week a wonderful conference was staged at John Jay College of Criminal Justice, though, celebrating OTWS's centenary: Off the Witness Stand: Using Psychology in the Practice of Justice."  With a keynote speech by Janet Reno, and talks by many top researchers in in psychology discussing the state of psycholegal research today--experts in eyewitness testimony, memory, jury decision-making, forensic psychology, coerced confessions, and more--it was geared toward both scholars and practitioners, and was a good state-of-the-discipline weekend.  (One concern, as I hope to discuss in another post, is the impression that criminal justice is all there is for psychology to look at.)

And there are methodological / institutional reasons for the skepticism.  As I and many others have noted, experimental psychology takes a different approach to finding answers: objective rather than adversarial, nomothetic rather than idiographic, probabilistic rather than certain, descriptive rather than prescriptive.

But this skepticism is unfortunate, and has led to a lack of communication between the disciplines of law and psychology.  I think ELS suffers from that lack, in at least two ways.  The first is simply mutual unfamiliarity with the other field’s research.  This is unfortunate historically (for instance, the explosion of heuristics-and-biases research over the last 10 years or so, which builds on psychologists’ work that has been around for 30 years and appears in any introductory cognitive psychology textbook), but also in terms of keeping up with empirical developments in psychology (and other social sciences too, of course).

*

The second, which I'll turn to next post, is methodological. . . .

*

For those who are still here, thanks!

Blumenthal Post #1

I appreciate the invitation to participate in the ELS blog.  As Jason mentioned, my empirical background is in psychology, so I’ll try to highlight a number of insights from work in that field that inform empirical – and experimental – legal studies, as well as point to some interesting ongoing work in psychology and law.  I’ll also mention a few methodological issues I think get neglected sometimes in our ELS discussion, hoping to expand the empirical approach we take to such study. And if I haven’t yet bored everyone by then, I’ll try to come up with other ELS bits as well.  Thanks again!

Guest Blogger: Jeremy A. Blumenthal

This week's guest blogger is Jeremy Blumenthal, Assistant Professor of Law at Syracuse University College of Law.   He earned his A.B., A.M., and Ph. D. in Social Psychology from Harvard University. He earned his Juris Doctor and graduated cum laude from the University of Pennsylvania Law School, where he was Editor-in-Chief of the University of Pennsylvania Journal of Constitutional Law and received several writing prizes. Most recently, he was a Faculty Fellow at Seton Hall Law School, teaching 1L Property and Law and the Social Sciences.  Previously, he clerked for the Honorable Maryanne Trump Barry of the U.S. Court of Appeals for the Third Circuit, and was an Associate for Latham & Watkins, LLP, in the Newark, NJ office.

Professor Blumenthal has published widely on topics in law and psychology, including demeanor evidence, the reasonable woman standard, victim impact statements, and more generally on the role of social science in assisting the legal system. He has also published in the areas of evidence and comparative law. His most recent publications focus on the role of emotions in the legal system; current research projects address topics in capital punishment, empirical work in law and psychology, and topics in property theory.

Numbers in the Classroom

It's been fun guest blogging this week, but this is likely my last post.  I'm going to wade into the ongoing discussion here and elsewhere on what kinds of statistical and social science background law students should get (or be offered) and lawyers need.  The easy answer to those questions, not likely to be controversial here, is more than they get!  Perhaps most useful to the students are those classes that focus on the types of numeracy and related skills that lawyers need when dealing with expert witnesses. 

But I don't think that teaching about such matters should be restricted to such classes or to classes covering, for example, empirical studies of judicial decisionmaking.  In my employment law class, for example, I cover the law governing the use of polygraph tests and other kinds of screening tests by employers.  While I'm at it, however, I walk the students through a (very simple) set of exercises aimed at helping them understand the statistics of screening tests.  What does it mean to say that a test is 90% accurate, or that it has a specificity of 95% but a sensitivity of only 70%?  Why might you want to know something about the prevalence or expected prevalence of whatever it is you are screening for?  How do you interpret a "positive" result?  Invariably, most of my students find this discussion extremely intimidating and even panic-inducing.  ("I didn't realize we'd have to do math in this class!" or "Is this going to be on the final?" are the two most common responses.)  Yet an employment lawyer advising a client on whether or not it is worth the legal hassles or potential hassles associated with using and relying on different types of screening tests simply cannot give competent advice without knowing how to think about these questions -- or at least without knowing that those questions exist and need to be answered.

I'm sure most readers of this blog wouldn't disagree and are likely to find ways to include numeracy if they teach doctrinal classes. I'm also  sure many of our colleagues who do not consider themselves ELS devotees do so as well.  It would be interesting to know how much this happens or, to the extent it doesn't, do law professors feel that more would be helpful but may not themselves feel competent in presenting the information.  I believe that statistical and social science information closely related to the doctrinal subjects already taught may be the easiest to "sell" to faculty and students alike -- certainly easier sells than, say, empirical studies of judicial decisionmaking or general statistics classes that are not directly tied to application in they study of doctrine or the practice of law.

Just the Facts?

Thanks to everyone who has commented on my posts so far.  The responses to my first post, on whether judges accurately report the facts, were extremely helpful.  I think that Sara Benesh may have put her finger on one of the real challenges of doing this kind of work – deciding what facts should have been reported. In situations where the Supreme Court has made it clear that certain types of facts must be taken into account, whether by incorporating those facts explicitly into a legal test or standard or by doing so implicitly by relying on such facts (or their absence) repeatedly, my hypothesis would be that the trial and appellate courts do a pretty good job of accurately reporting them – as Sara seems to have found. 

But my interest in this thorny question arises from a different context.  In employment discrimination cases, courts often must decide whether to grant summary judgment.  So they have to evaluate whether a reasonable jury could find for the plaintiff.  Every case presents a different constellation of facts and much of what courts must do is to weigh the facts.  Although the Supreme Court has provided a framework for this analysis (McDonnell-Douglas), in fact what courts do here is quite case-specific (and, as I’ve already suggested, quite unpredictable).  Judges in these cases don’t report every fact or piece of evidence.  They can’t.  And presumably, the decision to omit a fact is based on a determination that the fact, or piece of evidence, is not material.  So if a researcher were to go through the briefs and identify the facts mentioned by the lawyers but not the court, the researcher would still have to make a judgment whether the facts should have been mentioned.

So how to get around this problem?  One possibility is to focus on dissents.  In some cases, the dissents blow the whistle.  In Malacara v. City of Madison, for example, Judge Williams in dissent lays out the plaintiff’s substantial evidence to support his claim of discrimination, evidence that the majority simply ignores.  The problem is that such whistleblowing will point to only a fraction of the cases in which this problem arises, either because there is no dissent or because a draft dissent prompted discussion of the previously omitted facts.  This latter situation may be interesting for other reasons – does the presence of certain judges and the increased threat of whistleblowing affect the way or which facts are described?  This is a panel effect that I do not believe has been studied, but would be very difficult to observe, at least in the absence of judges’ papers.

There’s an additional facet to this problem that has to do with how the law develops, and that might provide some insight.  My colleague Mark Rosen has coined that expression “rulifying” to describe the way that the Supreme Court reliance on facts can turn those facts into elements of a legal standard or rule.  Putting these ideas together suggests an interesting possible empirical approach: is there less “whistleblowing” in areas of law that have relatively well-developed sets of facts courts are required to address versus areas of law that require judges simply to rely on their overall assessment of the evidence or facts – the totality of the circumstances?

What Judges Don't Say: The Challenge to Content Analysis

First of all, many thanks for the opportunity to guest blog here.  I've found ELSblog to be a valuable resource for information and an invaluable resource for thought-provoking reading.  I'm generally a "lurker," but guest blogging will force me out from the shadows.

Like many others, I was trained as a lawyer and practiced law before I became an academic.  My interest in empirical legal scholarship arises in part from experiences I had and things I observed as a law clerk and as a litigator. 

One of the great frustrations lawyers sometimes experience is the sense that judges are not paying attention to the specific facts and evidence of the case. And in some areas of law, at least, academics also take note.  In the world of employment discrimination law, for example, one need not look hard to find lengthy discussions and critiques of courts' inconsistent, unpredictable, or just plain wrong application of the summary judgments standards to prevent plaintiffs from taking their evidence to a jury or, in some cases, to grant judgment as a matter of law and reverse jury verdicts in plaintiffs' favor.   But these are complaints about the application of law to fact, not about inaccurate reporting of the facts themselves.

At the same time, numerous academics have undertaken important efforts to identify the facts that appear to matter to judges in different contexts. To name but two examples:  Jeffrey Segal's well-known work on search and seizure looks at the relationship of certain facts (whether there search was warrantless, for example, or whether it was a search of a home or a car) to outcome in the Supreme Court's 4th Amendment case law.  Lauren Edelman and others are studying how trial and appellate court judges in employment discrimination cases refer to employers' anti- discrimination policies and programs.  But what these efforts do not account for, by definition, is what the judges do not say, what they do not mention in their summary of the facts.

Of course, some might say, who cares what the judges don't say.  What they do say is what was salient to them, and what they do say is what often becomes important in the case law.  (This latter point is one of the arguments of Edelman et al.)  What they don't say may matter to the disappointed party and lawyers, but it is of no consequence otherwise.

But it seems to me that the question of whether judges -- consciously or unconsciously -- leave out (arguably) relevant facts is directly relevant to several points of great interest to scholars as well as to lawyers.  First, it is directly relevant to the question of how judges decide cases.  Second and relatedly, it may shed light on the extent to which they are political in their decisionmaking.  Third, it should, I think, force hard thinking about the normative question of what we want judges to do.  Do we want them to make predictions about how juries will decide?  Do we want them to announce rules of law applicable in the future?  Do we want them to focus scrupulously but narrowly on the case before them?  Are these goals inconsistent with each other?

The answers to these questions may vary depending on the case, the court, and the issue, of course.   Chief Justice Roberts has recently made some news with his call for more narrow, focused decisions -- and evoked the criticism that the Supreme Court's role requires a broader, less case-specific approach.  In fact, in my view, the question of whether judges accurately report the facts is more important in the trial and appellate courts than in the Supreme Court. 

If I'm overlooking something, of course I'd welcome references to work that does try to address the question of what gets left out of judicial opinions.   But as we in empirical legal scholarship are struggling with the question of how to adequately take into account of what is in judicial opinions (how to operationalize the law), let's not forget that opinions must distill mountains of evidence and piles of briefs into a few pages of summary.  Things must fall through the cracks.  I'm wondering -- is there a pattern to what gets left out and do those things matter?

Guest Blogger: Carolyn Shapiro

Our guest blogger this week is Carolyn Shapiro, Assistant Professor of Law at the Chicago-Kent College of Law.  Professor Shapiro earned a B.A. with general and special honors in English from the University of Chicago, an M.A. from the University of Chicago Harris Graduate School of Public Policy and a J.D. (high honors) from the University of Chicago Law School, where she was articles editor of the University of Chicago Law Review and a member of the Order of the Coif.             

After graduation, Professor Shapiro was a law clerk for Chief Judge Richard A. Posner of the U.S. Court of Appeals for the Seventh Circuit and  for Justice Stephen G. Breyer of the U.S. Supreme Court.             

Professor Shapiro's scholarly interests include federal courts and labor and employment law.  She teaches professional responsibility, employment law, and legislative process.

Guest Blogger: Jeff Yates

This week's guest blogger is Jeff Yates, Associate Professor of Political Science at the University of Georgia.  Jeff received his J.D. from the University of Tennessee and his Ph.D from Florida State University.  His research focuses on the study of judicial process, criminal justice and tort litigation.  His recent work is available here.

Guest Blogger: Yair Listokin

This week's guest blogger is Yair Listokin, an Associate Professor of Law at Yale Law School. Professor Listokin received his J.D. from Yale Law School in 2005, his A.B. magna cum laude in Economics from Harvard in 1998, and his Ph.D. in Economics from Princeton in 2002. At Yale Law School, Professor Listokin was three times named the John M. Olin Prize Winner for the best student paper in law and economics, and he was a John M. Olin Fellow in Law and Economics. In 2005-2006, he clerked for The Honorable Richard A. Posner at the U.S. Court of Appeals, 7th Circuit. His primary research and teaching interests are business organizations, bankruptcy, contracts, corporate finance, and empirical legal studies. His additional interests are in federal income taxation and secured transactions.

Guest Blogger: Greg Sisk

This week's guest blogger is Greg Sisk (St. Thomas).  Greg is a nationally-recognized scholar on the subjects of civil litigation with the federal government and empirical (statistical) analysis of judicial decisionmaking; he also writes about federal courts, legal ethics, and constitutional law. He is the author of the casebook, “Litigation With the Federal Government,” which is published by Foundation Press. Sisk’s empirical work on court decisions was honored with the Article Prize from the Law and Society Association in 1999. 

Greg's publications include: Judges and Ideology: Public and Academic Debates About Statistical Measures, 99 Northwestern University Law Review 743 (2005); Signaling and Precedent in Federal District Court Opinions, 13 Supreme Court Economic Review 63 (2005); Searching for the Soul of Judicial Decisionmaking: An Empirical Study of Religious Freedom Decisions, 65 Ohio State Law Journal 491 (2004); Above the Rules: A Response to Epstein and King, 69 University of Chicago Law Review 135 (2002).

Guest Blogger: Ed Morrison

This week's guest blogger is Edward Morrison, an Associate Professor of Law at Columbia.  Ed received his J.D. and Ph.D. (Economics) from the University of Chicago, and clerked for both Seventh Circuit Judge Richard Posner and Supreme Court Justice Scalia. Ed's areas of expertise include bankruptcy, law and economics, and corporate reorganization.  His recent scholarly work includes "Bankruptcy Decisionmaking: An Empirical Study of Continuation Bias in Chapter 11"  forthcoming in the Journal of Law and Economics.

Visualizing the United States Supreme Court

See: http://ella.slis.indiana.edu/~pahook/product/SupremeCourt_1956-2004_pajek.jpg

Please let me introduce myself. I am an academic law librarian here at Indiana University School of Law (JD, MSLIS). I am also a doctoral student in Information Science at Indiana University.

I work at the intersection of several fields: law, scientometrics, information visualization, spatialization, network science, and legal informatics. All of these fields are relevant to empirical studies of the law. Let me give you a quick gloss as to each.  Scientometrics is the study of science through scientific means.  It focuses on quantitative measures such as citation counts, the citation half-life of articles in a particular field, co-authoring proclivities and effects in different fields, etc. Information visualization is the mining of large datasets and using various display algorithms to render pictures of the data. The goal is to facilitate insights that would not be possible otherwise. Information visualization is strongly related to spatialization. This is the spatial rendering of inherently non-spatial data.  Network science is the study of large networks consisting of nodes and connections between those nodes. Finally, legal informatics is the study of how computing technology impacts the practice and study of law.

Recently, I have been producing information visualizations relevant to the work of the United States Supreme Court. My goal is to use and evaluate these visualizations in an applied educational setting.

The above visualization represents a spatial distribution of all of the justices of the United States Supreme Court that served during the 1956 to 2004 terms. The spatial layout of the justices is based on their cumulative co-voting percentages. The statistics were taken from 49 years worth of the annual Supreme Court review published in the Harvard Law Review. I used the Harvard Law Review’s O method of co-voting (“agreeing in the opinion of the Court or an opinion announcing the judgment of the Court.”)  I also normalized the raw numbers over the amount of cases any two justices heard together (N) to get the cumulative voting percentages.

The spatial distribution is the result of the spring force algorithm in a software tool known as Pajek (free for academic use). The spring force algorithm is analogous to rubber bands being placed between the justices with the higher the percentage of co-voting corresponding to a much stronger rubber band. The final distribution is the net affect of all of the different strengths of the edges (co-voting percentages) between the different nodes (justices). There is an implied element of time moving from left to right with long serving justices (co-voting with numerous different justices) being pulled to the center of the image.   

The tables in the image give the following aggregate information spanning the 1956 to 2004 terms: (1) the 10 highest cumulative voting percentages, (2) the 12 lowest cumulative voting percentages, and (3) the most cases heard together.  What interests me is the fact that Scalia and Thomas, with a cumulative voting percentage of 67%, do not even rank in the top ten. Top honors go to Warren and Marshall at 88%. [Keep in mind that the figures include unanimous cases. For instances, in non-unanimous cases from the 1994 to 2003 terms, Scalia and Thomas vote together at 78%--the highest for the time span.  Source: July 2, 2005 New York Times citing the Harvard Law Review statistics.] The lowest cumulative agreement is between Douglas and Rehnquist. They agreed in just 28% of the cases they heard together.  The mean voting agreement for all justices over the 1956 to 2004 terms is 59%.

Over the next couple days I will share more of my visualization work relevant to the Supreme Court, will mention others doing work in this area, and will talk more about the field of domain mapping which is the term that best captures my academic work.

Peter Hook
http://ella.slis.indiana.edu/~pahook/index.html

A Brief Tribute to Clifford Geertz

Sociocultural anthropology lost a major figure with the death, in late October, of Clifford Geertz. Geertz was a key founder of the subfield of sociocultural anthropology known as “symbolic anthropology,” which had its heyday in the 1960s and 70s (and whose lingering influence is felt in a number of corners of anthropology to this day). Bill Ford asked me to write a little something for the ELS Blog to mark this occasion.

When I entered graduate school in the 1970s, there was a vigorous struggle underway between symbolic anthropologists of various stripes, and scholars from competing paradigms such as economic and cognitive anthropology. (Cognitivists were then excitedly engaged in the cross-cultural study of cognitive structures such as schemas, etc. It seems as if the entire field of anthropology is periodically reduced, in the public media, to single figures and approaches – but in fact it has always had just as many internal divisions and debates as any other self-respecting social science. And recall also that “sociocultural anthropology” is just one of the four main fields into which U.S. anthropology has always been divided: the other three being archeology, physical anthropology, and anthropological linguistics.) Although sociocultural anthropologists persist in maintaining a number of distinct schools of thought, there has been a welcome rapprochement between some of those whose work emphasized the importance of economic factors, on the one hand, and the symbolic approach associated with Geertz. Clearly, a number of us would now argue, both people’s material needs and their ideas are important in understanding human societies, and the more interesting questions revolve around how, when, where, and to what degree different aspects of social and cultural systems make a difference in people’s lives.

As with many pioneers in social research, when we go back to read Geertz’s work we find a far less one-dimensional thinker than might be indicated by the label of “symbolic” or “interpretive” anthropology. Certainly Geertz pressed the discipline to think in more depth about cultural analysis and the role of symbols in organizing human experience. At various points in his career, he described this form of analysis in different ways. But a passage from his famous book, The Interpretation of Cultures, indicates Geertz’s own interest in moving to an integrative model:

“Nothing will discredit a semiotic approach to culture more quickly than allowing it to drift into a combination of intuitionism and alchemy, no matter how elegantly the intuitions are expressed or modern the alchemy is made to look. The danger that cultural analysis … will lose touch with the hard surfaces of life – with the political, economic, stratificatory realities within which men are everywhere contained – and with the biological and physical necessities on which those surfaces rest, is an ever-present one. The only defense against it, and against, thus, turning cultural analysis into a kind of sociological aestheticism, is to train this analysis on such realities and such necessities in the first place. … To look at the symbolic dimensions of social action – art, religion, ideology, science, law, morality, common sense – is not to turn away from the existential dilemmas of human life for some empyrean realm of de-emotionalized forms; it is to plunge into the midst of them.” (p. 30) 

Thus Geertz’s longterm research in Indonesia and Morocco, which began in the 1950s and 1960s, focused not only on symbols, but also on economics; not only on the symbolic importance of cock fights in Balinese culture -- but also on how industrialization created a “dualism” in the Indonesian economy. One of Geertz’s early works is devoted to an examination of the ways that politics, culture, and economic changes had interacted with ecology through a number of historical periods to create dilemmas for development in Indonesia (see Geertz, Agricultural Involution: The Processes of Ecological Change in Indonesia).

Geertz is perhaps best-known in law for his essay “Local Knowledge: Fact and Law in Comparative Perspective,” originally given as the Storrs Lectures for 1981 at the Yale Law School. Here he argues that law and anthropology share a focus on “local knowledge” – on the “artisan task of seeing broad principles in parochial facts.” Yet the two fields as often mistranslate as understand one another, because of their vastly different approaches to this task and to the “facts” themselves. Geertz argues that using comparative anthropological analysis of legal systems across cultures can offer the field of law an important insight: that one may have to exit one’s own categories and epistemological assumptions to translate accurately across diverse conceptions of law, of local reality, or of how society should be run. He argues that disciplines such as anthropology and history

“may have more to offer us in making our way through such perplexities as the shape-shifting    nature of the fact/law distinction across cultural traditions and historical phases than supposedly more ‘scientific’ enterprises, where everything that arises must converge. If there is any message …, it is that the world is a various place, various between lawyers and anthropologists, various between Muslims and Hindus, … and much is to be gained, scientifically and otherwise, by confronting that grand actuality rather than wishing it away in a haze of forceless generalities and false comforts.” (p. 234). 

Here we can detect echoes of the analysis of incommensurability (and U.S. law’s failures in coping with it) offered by present-day sociologists like Wendy Espeland.

In sum, my quick re-immersion in Geertz’s opus in order to respond to Bill’s request left me with the feeling I have when I go back to read founding thinkers like Max Weber, or Ruth Benedict – that the richness of their response to the empirical realities with which they dealt far exceeded the labels placed on their work. Geertz shared the commitment of most anthropologists to indepth lengthy fieldwork, performed with an adequate knowledge of indigenous languages. He pushed the field to examine its standards for ethnography – i.e., for our written reports of field research. In his early volume Islam Observed, Geertz undertook to test the validity of insights generated by “intensive indepth fieldwork [performed] in particular settings” in broader national arenas: “Like all scientific propositions, anthropological interpretations must be tested against the material they are designed to interpret” (p. vii). He might not have chosen to use this terminology in his later work, which showed more suspicion of the dangers of “scientism” in social science generally. (All methods are partial, all understandings somewhat contingent – and the language of science can be used to induce false certainty [which in fact leads to a very unscientific inflation of the significance of often quite limited findings].) But taken across the span of his lifetime, Geertz’s work gives us a gift to which we can return time and again for deeper understandings of law, society, culture, and the disciplines devoted to studying them.

Guest Blogger: Margo Schlanger

This week's guest blogger is Margo Schlanger of Wash U in St. Louis School of Law.  Her webpage hosts a fantastic list of ELS resources.

Professor Schlanger received her J.D. in 1993 from Yale, where she was Book Reviews Editor of the Yale Law Journal and received the Vinson Prize for excellence in clinical casework.  She then took up a two-year appointment as Law Clerk for Justice Ruth Bader Ginsburg (1993-1995).  From 1995 through 1998, she was an attorney in the Special Litigation Section of the Department of Justice Civil Rights Division, where her practice focused on police and prison civil rights issues.  Before her appointment at Wash. U., she was Assistant Professor of Law at Harvard from 1998 to 2004.  Professor Schlanger is a leading authority on prisons and inmate litigation; in addition to her teaching and research in this field, she is currently a member of the Commission on Safety and Abuse in America's Prisons.  She also runs the Civil Rights Litigation Clearinghouse.

Her recent scholarship is interdisciplinary in its perspective and often includes an empirical component.  For illustrative articles see "Civil Rights Injunctions Over Time: A Case Study of Jail and Prison Court Orders,"  81 N.Y.U. L. Rev. 550 (2006), and "Inmate Litigation," 116 Harv. L. Rev. 1555 (2003).  In 2006-07, Professor Schlanger is teaching Individual Rights & the Constitution, and a seminar on Civil Rights Injunctions; she often teaches Torts and a class on the Constitutional Law of Incarceration.

Law and Ordinary Citizens

          Continuing on my theme of “qualitative is empirical too” I want to highlight an important study about law and ordinary citizens – in this case when they are called on to be jurors in capital cases. 

          Ben Fleury-Steiner’s, Jurors’ Stories of Death comes out of the famous Capital Jury Project led by William Bowers.  We have known since the famous Baldus study that there is racial bias in the imposition in the death penalty in the United States. 

          The CJP is an ambitious qualitative and quantitative study which analyzes jurors decision making at every phase of the capital sentencing project.  In 14 states, the CJP chose 20 – 30 capital trials, selected 4 jurors from each jury and conducted extensive interviews with these 1,201 capital jurors from 354 trials.  The quantitative findings are really interesting -- one example I particularly like demonstrates jurors’ systematic misunderstanding of what will happen to the defendants if they are not executed.  But how do we explain the racial bias?

         Ben Fleury-Steiner does it excellently with qualitative research.  Based on the larger and systematically drawn sample, Fleury-Steiner’s qualitative interviews are drawn from a rigorously-drawn sample (one of the ways to evaluate qualitative research is on the quality of the sample – his is excellent because it is across state, the trials were randomly selected within the period, and he talked to multiple people of different races from the SAME juries to hear what happened in the jury room).  The shocking thing is how honestly jurors spoke of the role of race in determining death sentences.  Consider just 2 representative quotes from one of Fleury-Steiner's article:

Qualitative is Empirical Too! Law and Organizations

     Before you click back over to the New York Times or whatever you were reading before you got here, let me give you four good reasons to be interested in qualitative research in ELS.

    

     Of course, we can debate any of these that you like, but here is what I think:  (1)  Qualitative research is the kind of theory-generating research that much quantitative research is testing; (2) Many questions about law and its influence in society can only be answered in the first instance using qualitative methods.  Of course, you can operationalize them in such a way as to later test them quantitatively, but the initial understanding of various processes (I will give you examples all week) of how law works come from (a) theory or (b) qualitative research; (3) Qualitative research, when placed alongside quantitative research can teach us much more about the phenomena we study than the quantitative research alone; (4) Lot of work in law reviews right now is masquerading as “empirical qualitative research” but there is something in your gut that tells you there are problems with it.  You are probably right.  There is a science of qualitative research and there are standards by which we can evaluate qualitative research meaningfully.  More on that later in the week.

    

     This week, I hope we can talk/write about what makes it more or less convincing, how we evaluate the methods, and what we can learn from qualitative work that, although we may be able to quantitatively test it later, could not have been observed without the qualitative research in the first instance. Before I say too much more about the mechanics of qualitative research, I better give you some examples or you will totally lose interest.

One of the most important questions that I think we are almost all interested in (or why would we do this work) is something like, “what impact does law (broadly conceived -- could be statutory, judicial, constitutional or regulatory) have when it intervenes to affect social circumstances in whatever milieu we study?" 

    

     Today, I am going to highlight 2 excellent qualitative studies in the organizational context.  Law often is used in an attempt to affect individuals’ behavior in organizations.  We want executives to oversee accounting practices so that we don’t have another Enron; we want to provide incentives to individuals in organizations to refrain from discrimination; we want the police to follow certain procedures to ensure the constitutional rights of the accused; and you can name many more.     

Guest Blogger: Laura Beth Nielsen

This week's guest blogger is Laura Beth Nielsen, who currently holds a joint appointment at the American Bar Foundation and Northwestern Sociology.  Laura Beth's primary research field is the sociology of law, with particular interests in legal consciousness and the relationship between law and inequalities of race, gender, and class.  Laura Beth's recently published book, License to Harass: Law, Hierarchy, and Offensive Public Speech  (Princeton 2004) (winner of the 2000 LSA dissertation prize) uses qualitative empirical methods to examine hate speech, targets ' reactions and responses to it, and their attitudes about using law to deal with such speech.   In addition, her numerous articles, book chapters, and edited volumes have made important contributions to our understanding of employment law and the perceived versus actual autonomy of in-house legal counsel. 

Laura Beth is trained as both a lawyer and a social scientist.  She is received her Ph.D. from UC Berkeley ' s Jurisprudence and Social Policy Program (1999) and her J.D. from Boalt Hall School of Law.   During the 2005-06 academic term, Laura Beth was a Research Fellow at the Center for Advanced Study in the Behavior Sciences at Stanford University.  As a person who is fully integrated into both the legal and social science academies, Laura Beth is well situated to comment on the current and future state of the empirical legal studies movement.   We look forward to her postings over the coming week.