Empirical Legal Studies

New Thinking About an Old Problem: Selection Effects

Keith Hylton (Boston Univ.) and Sanghoon Kim (Buffalo--econ.) distill their recent paper's challenge nicely. When it comes to much of empirical work on legal (and court) outcomes, the full potential of multiple regression "has not been realized, largely because of one reason: the information contained in court opinions comes from a selected sample."

For all the obvious reasons, much of empirical legal research exploits court outcomes as data. One challenge, however, is that the comparatively smaller subset of formal legal disputes that persist to a judicial outcome almost assuredly systematically differs from the much larger subset of legal disputes that privately settle. My former mentor and colleague, Ted Eisenberg, frequently reminded me that "one cannot over-state the importance of selection effects in empirical legal research."

While the challenge endures, solutions are not entirely obvious. In Trial Selection and Estimating Damages Equations, Hylton and Kim develop an interesting suggestion. Specifically, they articulate a structural econometric model that, they argue, better accounts for litigant selections in the appellate context. The paper's abstract follows.

"Many studies have employed regression analysis with data drawn from court opinions. For example, an analyst might use regression analysis to determine the factors that explain the size of damages awards or the factors that determine the probability that the plaintiff will prevail at trial or on appeal. However, the full potential of multiple regression analysis in legal research has not been realized, largely because of the sample selection problem. We propose a method for controlling for sample selection bias using data from court opinions."

Side-By-Side vs. Overlayed Plots

At the risk of delving too deeply into Stata's graphics coding "weeds," two different ways to present the same graphs help illustrate graphing possibilities.

By convention, a "side-by-side" presentation of two distinct, though related, plots (with confidence intervals) in Stata looks like the following:

If the plots are snug enough in how they relate, however, additional coding can produce a single "overlayed" graph:

For those intrigued, the coding underneath these two possibilities is included here.

New Stata Graphing "Toys"

While Stata's graphing capabilities are robust, coding remains complex in places. While the coding "load" has come down a bit over the years, the "bells & whistles" have remained relatively stagnant over the years. One recent change on the "bells & whistles" front involves an increased capacity to "colorize" standard graphs. While the example (below) illustrates how levering color in graphics can enhance (and/or clarify) what the data are telling you, I remain curious about how journals (and their publishers) will choose to deal with the increased production costs associated with color in published papers.

Any potential cost issues aside, a recent Stata newsletter (click here) explains and illustrates (with sample coding) how one can transform a standard twoway scatter plot from this:

To this:

Plotting Coefficients

As I'm seeing more examples of coefficient plots [coefplot command] in the literature (see, e.g., below), I thought the following links might interest some readers. For brief discussions and examples (including Stata coding) click here, here, here, and here.

Plotting Kernal Density Estimates as Colored Areas

By default, kernel density estimates are plotted as lines, meaning curves:

According to recent StataList discussions (see here and here), however, plotting kernal density estimates as "areas" has become the standard in some fields. While for publication purposes the coding requires color, if such a possibility exists the resultant information conveyed is quite helpful as the following example illustrates.

Fixed Effects vs. Random Effects

While debates persist, how fixed- and random-effect estimators differ and what they seek to accomplish, however, remain somewhat muddled in the various empirical literatures. A recent discussion on StataList, however, provides a rare moment of clarity (here) by nudging us to think about fixed- and-random effects as involving different tradeoffs (or preferences) between consistency and efficiency. As one commentator helpfully notes:

"The -fe- estimator is always consistent but inefficient if -re- is the way to go.
 The -re- estimator is inconsistent if -fe- is the way to go."

Skewed Data, Histograms & Axis "Breaks"

While skewed data distributions (e.g., a large proportion of zeros) invite an array of well-known methodological complications, they also complicated data presentation needs. For example, a standard histogram for a distribution that includes approx. 85% zeros looks something like the following:

Setting aside the methodological complications imposed by such skewed data, as it narrowly relates to descriptive data presentation needs one "dissenting" approach is to insert axis "breaks" (discussion here) that generates the following histogram:

Such a strategy may provide the optimal tradeoff between the visualization of "extreme" data and compression of the non-extreme data. This approach also has the virtue of keeping the underlying data in their raw rather than a transformed scale.

"Heatplots" for Correlation Coefficients Graphs

While I've never used one myself, I detect an upward trend in "heatplots" (example below) for basic descriptive correlation coefficient graphs.

A helpful discussion (along with coding suggestions) on what is possible with Stata's heatplot command is found here.

Importing Non-Stata Data Sets Into Stata

While perhaps old-hat for some (many), a quick new Stata tutorial video (click here) walks viewers through some of the basics incident to data importation. For old-time users (including those who may reflexively turn to StatTransfer programs for conversions), increasingly easy--and a wider array of--data transfer options remain much-welcome news.

A Quick Refresher on T- vs. F-Tests

While I haven't thought about this since grad school, a recent discussion on the StataList (here) provides a helpful, quick refresher on when F- and T-Tests converge and when they do not (and why).

An Online Resource of Note

For those new to ELS, and even those with more sophisticated questions, UCLA's statistical consulting site, festooned with an array of resource links (here), remains quite helpful. While the site continues to evolve over time, some well-organized and useful links will help orient and guide those engaged in ELS-related work. Examples include a quick reference chart on "choosing the correct statistical test" (which includes command syntax for SAS, Stata, SPSS, & R), illustrated examples of how to interpret annotated output for standard commands, and robust "FAQ" links that are, again, program-specific (complemented by a "general FAQ" link (here)).

Interpreting DID Output

While one may quibble about how the questioner designed her DID estimation, the question itself--how to understand and interpret standard DID output (in Stata)--elicited some helpful replies in a recent StataList discussion (here).

Poisson Tutorial Videos

The good folks at Stata recently posted a series of video tutorials focusing on how to fit various Poisson regression models which assist when modeling count data. The various videos take up a Poisson model with: a) continuous predictors (here); b) categorical predictors (here); and c) Poisson models that include both continuous and categorical variables (here).

A Small--But Important--Logistical Matter Incident to Dataset Merging

Those who have merged (or appended or crossed) datasets may have inadvertently bumped into a not uncommon problem--where the two separate datasets share a variable with the identical variable label name. For relatively smaller datasets, and where the researcher is aware of this hurdle ex ante, one can always, by hand, rename variables prior to combining the datasets. For larger datasets with many more variables, however, while this may not be practical it nevertheless remains critical.

A recent discussion of how to manage this nettlesome problem is found here. The comments' leading suggestion references Daniel Klein's -elabel- package, available from SSC (as it involves user-written code). Put briefly, the elable command allows a user to add a suffix (e.g., "_1") to every variable name in "dataset1." An analogous manipulation to datast2 can add another unique suffix (e.g., "_2") to every variable name in dataset2. And so forth. The end result includes datasets with distinct variable label names that can then be combined.

Helpful Statistics Resources

On the better late than never front, I just stumbled across a wonderfully rich "one-stop" resource for statistics novices. Germán Rodríguez (Princeton) hosts a site (here) that includes tutorials for both Stata and R. As to the tutorials' scope, the Stata tutorial, e.g., ranges from a "gentle introduction" to coding examples for customizable tables. Many--especially students--may find this resource helpful.