Empirical Legal Studies

Lay Understandings of Civil Settlements

When it comes to resolving formal civil disputes, popular imagination, partly fueled by the media and entertainment industry, is dominated by "courtroom drama" and jury trials. Empirical work over the decades, however, paints quite a different picture, and one dominated by the "disappearing trial." Thus, as Jennifer Robbennolt (Illinois) et al. make clear in Settlement Schemas: How Laypeople Understand Civil Settlement, "for all the attention paid to the real and imagined drama of the courtroom, these depictions overlook the reality of how many disputes are resolved: through any means but trial."

In the service of seeking a better understanding of how the public understands civil settlements, the authors surveyed a representative sample of more than 1,000 U.S. adults. Key findings include that while the public conveyed a widespread general understanding about civil settlements, evidence of misconceptions about the details, including the role of a jury and settlement scope, also emerged. The paper's abstract follows.

"What does the public think it means to ‘settle’ a civil case? Most legal disputes in the United States end in an agreement to settle, but little is known about what laypeople think about settlement. To fill this gap, we took a direct approach: we asked a nationally representative sample of U.S. adults – more than one thousand of them – basic questions about settlement. We found widespread understanding about the essential nature and frequency of settlement, but persistent, though not universal, misconceptions about the details, including the role of a jury and settlement scope. Because settlement is such a pervasive part of the U.S. legal system, the system’s legitimacy turns in part on how the public understands and views civil settlement. The survey reported here provides a foundational study of the understandings and framework – the schemas – that the public bring to settlement."

NYT, MTurk, and Data

Yesterday's New York Times ran a lengthy piece (here) on Amazon's Mechanical Turk ("MTurk") platform. The author joined the ranks of "Turkers" for a brief period and did some "Turking" to supplement his background research for the article. While the piece reflects a decidedly labor-economic focus, insofar as "MTurk"-created data remain a staple for an increasing slice of experimental psychological research (despite earlier concerns raised years ago by, among others, Dan Kahan (see, e.g., here and here)), the NYT''s article warrants a quick read.

Public Innumeracy

While not squarely in the typical ELS wheelhouse, the following excerpt just stopped me in my tracks.

"In her excellent book, Race to the Top, the journalist Elizabeth Green tells a story of a new hamburger that the A&W Restaurant chain introduced to the masses. Weighing 1/3 of a pound, it was meant to compete with McDonald’s quarter-pounder and was priced comparably. But the 'Third Pounder' failed miserably. Consultants were mystified until they realized many A&W customers believed that they were paying the same for less meat than they got at McDonald’s. Why? Because four is bigger than three, so wouldn’t ¼ be more than 1/3?"

To be sure, this degree of innumeracy is not typically present in law school classrooms (or I certainly hope not). That said, a general ambivalence (at best) or aversion (at worst) towards all things quantitative shapes the stream of students who self-select into law schools.

Perils of Self-Reported Data

I often caution students contemplating using self-reported data and urge them to consider observational data where possible (and appropriate to the research question and design). Why Bother Asking? The Limited Value of Self-Reported Vote Intention, by Rogers, Todd Rogers (Harvard--Kennedy School) and Masa Aida (Greenberg Quinlan Rosner), contribute empirical support to my intuition, certainly as it relates to predicted voting behavior. The findings' implications for an array of sub-fields are important. An excerpted abstract follows.

"Phone surveys from three elections, including one survey experiment, are analyzed to compare respondents’ pre-election vote intention with their actual voting behavior using administrative records (N=29,403). Unsurprisingly, many who predict that they will vote actually do not vote. More surprisingly, many who predict that they will not vote actually do vote (29% to 56%). Records of past voting behavior predicts turnout substantially better than self-prediction. Self-prediction inaccuracy is not caused by lack of cognitive salience of past voting, or by inability to recall past voting. Moreover, self-reported recall of turnout in one past election predicts future turnout just as well as self-prediction. We discuss implications for political science research, behavioral prediction, election administration policy, and public opinion."

The Polls - Torture

Both PEW and Gallup (others?) have examined public support for torture.

In a lengthy report on a variety of issues, PEW has these results:

The full discussion of these results is on page 25 of the PEW report (page 29 of the pdf file).

Gallup editor Darren Carlson discusses a November 2005 survey here (n=515). The question was, "Would you be willing -- or not willing -- to have the U.S. government torture suspected terrorists if they may know details about future terrorist attacks against the U.S.?" The results: 38% were willing; 56% were not willing; and 6% had no opinion.

Update: Here's an ABC News/Washington Post poll from May 2004 (n=1005), which found 35% of Americans thought "it's acceptable to torture people suspected of terrorism in some cases."  The poll also covered specific techniques for "getting information from prisoners," such as sexually humiliating suspects (16% support) and not allowing a suspect to sleep (66% support).

The Polls: Video Game Regulation and the First Amendment

A minor dustup occurred this week in the world of video game politics, one that implicates public thinking on First Amendment issues. In a press release, public relations firm Hill & Knowlton (H&K) reported: 

“Sixty percent of US consumers polled agree that the government should regulate the sale of violent or mature content video games while a slim majority (51%) agrees that the government should regulate mature content itself.” (emphasis added).

The Entertainment Software Association (ESA), which opposes government regulation of video games, responded to H&K’s report with some unusually harsh words. There may be some bad blood between the two organizations based on prior dealings -- the ESA’s criticisms are here [UPDATE: H&K responds here and here] -- but the ESA didn’t discuss one significant qualification to the finding of 51% support for content regulation. A comparison of the results for two of H&K’s questions suggests there is less public enthusiasm for content regulation in this survey than the reported 51%. Here are the two relevant questions:

Q11. Do you think the government should regulate violent content in video games?

Q8. If a person running for office were to take a stand on the regulation of violent or mature content in video games, to what extent would that impact your vote for him or her?*

The Polls - Presidential Favorites

The Rasmussen Reports recently asked Americans whether they have favorable or unfavorable views of every president from George Washington to The Decider. Naturally, Washington and Lincoln are the tops. Nixon and Bush are, well, not. The winner of the "not sure" category, with 74% not sure, is the guy in the picture. (Rhymes with Tohn Jyler.)

The full results are here and here. (There is clearly an error with the results for McKinley.)

UPDATE: The McKinley error has been corrected. He received a response of "very unfavorable" from 4% of respondents, not 40%.

H/T The Washington Monthly

Seat Belts, Paternalism, and New Hampshire--"Live Free and Die?" (Blumenthal guest)

The New Hampshire State Senate recently rejected a bill that would have mandated seat belt use for all adults.  According to the media, New Hampshire is the only U.S. state without some such law.  The discussion involved, unsurprisingly, issues of paternalism and "being told what to do"--squibs from earlier stories in the New York Times noted that "For many residents, buckling up and being required to buckle up are 2 different things," and one proponent of the bill noted that the debate "harkens [sic] to the libertarian 'don't tell me what to do' streak that characterizes much of [NH] politics."  The Times stories also noted that "New Hampshire has the lowest rate of seat belt use in the country, 49.6 percent," and that "77 percent of fatal crashes in the state involved occupants who were not wearing seat belts."

Paternalism, of course (whether asymmetric, libertarian, emotional, or other), is increasingly discussed in legal academia.  Much of the connection to ELS is in the context of evaluating whether particular interventions (as just one instance, Thaler and Benartzi's SMarT investment approach) in fact improve welfare on some specific outcome.

Less empirical analysis, I think, goes to the justification of such "paternalistic" intervention, and I hoped to post a few comments in that context, using the New Hampshire example as a vehicle [sorry, pun intended] for mentioning empirical research relevant to some of the usual objections to "telling someone what to do."  Existing empirical work may indicate that we need to look a little harder at such objections--and I think providing empirical grounds for such a discussion is a fruitful effort for ELS in any context.

Offhand I can imagine at least five such objections, falling roughly into three categories.  First is the classic Millian notion that people know their own preferences, and as a result are best at identifying and choosing their own preferences.  Second is the classic autonomy argument: people's freedom to choose should be valued per se ("don't tell me what to do!") ; relatedly, people's preference for the freedom to choose should be valued per se.  Third, people learn from their mistakes, and should therefore be allowed to make them--perhaps simply for the substantive benefit of learning what is correct, perhaps to "build character."  Jon Klick and Greg Mitchell get at this third category in their recent response to libertarian paternalism (among many other points, of course).

(At least two other objections, mentioned recently by Ilya Somin on the Volokh Conspiracy, involve the accountability of chosen "experts" who make paternalistic decisions and whether those experts should get the final word.  A related one, of course, involves public choice and capture.  I'll set those aside to the extent that they involve evaluation of the means of effectuating interventions once it's decided that paternalism is appropriate [except where Somin, for instance, notes that "Although the expert is more knowledgeable than I am about technical issues in his field, I am more knowledgeable than he is about my own values"--the first category identified above].)

Again, my goal is not to take sides, but rather to point out another instance of ELS's potential to advance a substantive policy discussion.  In short upcoming posts I'll try to identify some empirical research relevant to each of the five objections above.

Nw. U. L. Rev. Colloquy & Survey Methodology

Northwestern University Law Review's on-line blog, Colloquy, features a debate this week about empirical support for the belief that white Americans hold negative stereotypes associated with viewing Asian Americans as a "model minority." According to Colloquy's editors:

"Robert Chang and Rose Cuison Villazor respond to McGowan and Lindgren's prior piece, Testing the "Model Minority Myth," which argued that empirical data do not support the belief that white Americans hold negative stereotypes associated with viewing Asian Americans as a 'model minority.' Chang and Villazor argue that McGowan and Lindgren's argument is flawed because it relies heavily on data produced by asking white respondents questions about their racial attitudes in face-to-face interviews.  Chang and Villazor assert that this methodology is unlikely to yield true data about racial attitudes, as it is infected with problems of response falsification, is unable to ferret out subtler forms of racial bias, and is limited by the 'perpetrator perspective' on racial discrimination."

The Polls - Name Two Justices

I got the full report from Zogby on the recent pop culture survey. In this survey, people were asked to name only two justices, not all nine. Sixty-one percent of people could not name any justices, whereas 15% named one and 24% named two. The press release characterizes Thomas as the “most recognized justice,” and while prior surveys support this characterization (now that O’Connor is gone), the Zogby survey on its own probably supports calling him only the “most top-of-mind justice.”

Here are the percentages indicating how often each justice was named:

Thomas	20%
Scalia	11%
Roberts	9%
Ginsburg	9%
Alito	5%
Kennedy	4%
Souter	3%
Stevens	2%
Breyer	1%

Another relevant question was asking people to name the three branches of government. Forty-two percent named all three, 17% gave an incorrect “other” answer, and 41% were “not sure.” Unfortunately, it’s not clear what exactly counted as a correct answer to this question, e.g., did “President” count for the “executive” or the “courts”  for the “judiciary”? Without this information, this question is not as interesting.

n = 1213
error: +/- 2.9%

Source: Zogby International, “Pop Culture/Political Survey: Nationwide and in Ten Metropolitan Areas” (Aug. 10, 2006).

The Polls - Another Name-the-Justices Survey

Zogby released (some of) the results from a new pop culture knowledge poll yesterday. It included a couple of Supreme Court items:

When asked to name two of Snow White's Seven Dwarfs and two of the nine U.S. Supreme Court Justices, 77% of Americans polled were able to identify two dwarfs, while only 24% could name two Supreme Court Justices. Not surprisingly, Clarence Thomas, whose nomination was marked by controversy, was the most recognized Justice -- identified twice as often as his next best-known peer on the Supreme Court -- Antonin Scalia.

And the other one:

Surprisingly, despite record-breaking ratings, when asked who the newest American Idol is as of May 2006, only 23% of those polled named Taylor Hicks. Still, while that may seem low, it was double the percentage of those who recalled which Supreme Court Judge was confirmed in January 2006. Only 11% answered "Samuel Alito."

As best I can tell, Zogby has not posted the full results of this survey, but the press release is here. It contains additional results that may be interesting but that are not really ELS relevant, e.g., "Krypton was identified correctly by 60% of the respondents as Superman's planet of origin, while only 37% were able to name Mercury as the closest planet to the sun." (When the new movie came out a few months ago, I actually discussed this very question of how many people would be able to name Krypton as Superman's home planet with a friend of mine. Now, thankfully, there are data to resolve this question.)

My collection of previous name-the-justices poll results is here. I see that I still need to update it with a fourth poll from 1989; I had forgotten about it when I did the original post.

UPDATE: William Child at TortsProf points to a very nice analysis at the Language Log of how these public knowledge surveys can be spun by those who write press releases. A second Language Log post on the subject can be found here. Both posts focus on the First Amendment survey sponsored by the McCormick Tribune Freedom Museum.

The Polls - Occupational Trust

The Harris Poll has some new data on the most and least trusted occupations. Respondents were asked, "Would you generally trust each of the following types of people to tell the truth, or not?"

Doctors are at the top with 85% percent generally trusting them. Actors are at the bottom with only 26%. A few other categories:

Teachers: 83%
Professors: 75%
Judges: 70%
The President: 48%
Members of Congress: 35%
Lawyers: 27%

Additional information, including the results for all 22 occupational categories, can be found here. Unfortunately, the results do not reveal what happens when you combine a professor and a lawyer.

The Polls - Homosexuality and Gay Marriage

With the President about to make an announcement in support of a constitutional amendment to ban same-sex marriage, a just updated AEI report entitled Attitudes about Homosexuality and Gay Marriage should be of interest. Here is the abstract:

Updated for Spring 2006, this study is a compilation of public opinion polls on acceptance of homosexuality, gay marriage, civil unions, partner benefits, party identification and voting of gays, employment, and adoption. The study includes all of the latest polling data as well as important historical trends for comparative purposes.

The report is 52-pages long with numerous polling results. Assorted highlights from an earlier version of the report can be found here. The full text in pdf form of the current version can be found here.

(via Andrew Sullivan)

The Polls - NSA Surveillance

Mystery Pollster Mark Blumenthal comments on the various polls here on the CBS News PublicEye blog (rather than his own blog), and he provides links to the full results from each poll. It's a good discussion, though I question his final comparison of two poll items in the penultimate paragraph.

As a side note, view the National Security Agency Kid's Page here. Apparently, this has been around for at least a few months. Somehow, I missed it. The NSA says it's fun for adults too.

The Polls - Abortion

Gallup released an analysis today focusing on the relationship between religion and opinions on abortion.  It is not exactly full of surprises, but here is the conclusion:

[K]nowing an individual's frequency of church attendance helps predict his or her attitude toward abortion, above and beyond what can be predicted by knowing that individual's partisan affiliation. Republicans who attend church regularly are the most likely to be anti-abortion, while Democrats who seldom or never attend church are the least likely to be anti-abortion.  Both variables are important in understanding the nature of abortion attitudes in America today.

The full article is available here.

The Polls - Modern Morals

Last week, the Pew Research Center released a report last week entitled, “A Barometer of Modern Morals: Sex, Drugs, and the 1040.”  Respondents were asked to classify ten behaviors as “morally acceptable, morally wrong, or not a moral issue.”  The following results provide the percentages of people classifying each behavior as morally wrong:

Married people having an affair	88%
Not reporting all income on taxes	79%
Drinking alcohol excessively	61%
Having an abortion	52%
Smoking marijuana	50%
Homosexual behavior	50%
Lying to spare someone's feelings	43%
Sex between unmarried adults	35%
Gambling	35%
Overeating	32%
N = 745
Sampling error: +/- 4%
Interview dates: February 1-5, 2006

The complete report, with breakdowns by age, gender, education, ideology, etc., is available here.  A one-page overview that includes breakdowns only by education levels is available here.  Note that this overview lists a misleading n of 1502, which is the number of respondents for a larger survey.  Just short of one-half of these 1502 respondents were asked the “modern morals” questions on which this report is based.

Abstracts - The Journal of Politics (May 2006)

The upcoming issue of The Journal of Politics (May 2006) will include several articles in the law and courts field. It will also include a new study by Simon Jackman and Paul Sniderman on the effects of deliberation, which remains a popular topic among both political scientists and law professors. (They conclude that at least in one sense, "deliberation is for naught.") Manuscripts of most of the articles are available on the JOP website. Here are the abstracts:

Eileen Braman, "Reasoning on the Threshold: Testing the Separability of Preferences in Legal Decision-Making."

One hundred fifteen law students were given a mock brief with identical legal arguments on both sides of a standing dispute in litigation involving restrictions on political expression of public employees. The content of the expression at issue (pro-choice vs. pro-life) and the jurisdiction where the case was pending (with vs. without controlling authority on the standing issue) were experimentally manipulated. Participants' policy views on (1) abortion, (2) free speech and (3) Hatch Act restrictions were measured to assess their influence on the standing decision. In line with traditional notions of legal reasoning, participants were able to separate their views on Hatch Act restrictions from the standing decision. Opinions on free speech, however, influenced judgments consistent with attitudinal hypotheses. Also, participants’ opinions on abortion interacted with speech content to influence judgments – but in a manner not wholly consistent with legal or attitudinal accounts of decision-making.

The Polls - Constitutional Awareness

Since it shows more public knowledge about The Simpsons and American Idol than the Constitution (but with no mention of the Three Stooges), a new study released yesterday by the McCormick Tribune Freedom Museum is getting a fair amount of coverage. If you want to see the source material rather than newspaper summaries, go here for the press release or here for the full report. The Chicago Tribune got it right: "A survey released Wednesday showcases a bit of data that should surprise nobody[.]"

One highlight from the survey: About 21% of respondents thought the First Amendment guarantees the right to own and raise pets. I am always interested in finding examples of exotic constitutional claims—i.e., "rights talk" data points—and I have see this one in action before. Specifically, it was a constitutional right to own 400-pound Bengal-Siberian tigers (and maybe cougars as well): "I think it's my right as a U.S. citizen to have these cats.... If people say we can't have them, they're messing with my constitutional rights." Alex Tresniowski and Gabrielle Cosgriff, Cat Fanciers: With an Increasing Number of Texans Keeping Tigers as Pets, Some in the State Fear More Maulings and Deaths, People, April 16, 2001, at 111.

The Polls - Supreme Court Awareness

In January, FindLaw released the results of its most recent Supreme Court Awareness survey, a simple survey asking respondents to name the nine Supreme Court justices. At least one other blog (CrimProf) has posted the results, but it is useful to view the new results alongside past ones. I am aware of two other similar surveys, Findlaw's first Supreme Court Awareness survey in 2003 and Luntz Research's Three Stooges/Supreme Court survey in 1995.

Together, these three surveys reveal a fairly stable level of knowledge--or perhaps a stable lack of knowledge. Over half the population cannot name even a single sitting Supreme Court justice. Of those who can name one or more justices, the results are largely consistent in all three surveys. O'Connor was the most well-known in all three; Stevens and Breyer were the least well known. And although Thomas appears less well-known now than in 1995, most of the remaining variation in the surveys is likely within the range of sampling error. (Scalia may be slightly more well known now than in 1995.) One major difference in the December 2005 survey is that Chief Justice Roberts was a correct answer, not Rehnquist.  But despite Roberts' confirmation in September and all of the media coverage that went along with it, only 16% could recall his name by December.

Here are the results:

       8/1995                      6/2003                    12/2005
O'Connor    31%       O'Connor    25%        O'Connor   27%
Thomas       30%       Thomas       21%        Thomas      21%
Rehnquist    8%        Rehnquist   10%        Roberts     16%
Ginsburg      7%       Ginsburg        9%       Scalia         13%
Scalia            6%       Scalia            9%        Ginsburg    12%
Souter          4%       Souter           4%        Kennedy      7%
Kennedy       4%       Kennedy        4%        Souter         5%
Breyer          1%       Breyer           2%       Breyer         3%
Stevens        1%       Stevens         1%        Stevens       3%

zero              55%     zero             65%      zero            57%
one               16%     one+             35%      one+            43%
two               11%
three            17%

n=1200                     n=1000                      n=1000

While the specific ability to rattle off Supreme Court justices is not particularly important in and of itself, the public's poor performance in these surveys is a good indication that little is known about the Court in general. As for the Stooges, however, public knowledge appears more robust. In the 1995 survey, 67% of respondents could name three or more of the Stooges. Only 13% failed to name any. Unfortunately, I am not aware of any more recent Stooge data.

Sources:

1995 Survey: Joan Biskupic, Has the Court Lost Its Appeal?, Washington Post, Oct. 12, 1995, at A23; Richard Morin, Unconventional Wisdom, Washington Post, Oct. 8, 1995, at C5.

2003 Survey: FindLaw /PR Newswire

2005 Survey: FindLaw /Ipsos

The Stooges:
Moe, Larry, Curly, Shemp, and Curly Joe. (And Joe Besser?)