Federalism and Empirical Research

I spend most of my time working on criminal law issues, and perhaps that greatly colors my thinking. But I've been wondering lately if we pay too much attention to the federal system.

First, if the answer is "yes," it is certainly not a problem limited to ELS. My gut feeling is that (almost) all of law suffers from a federal bias. Certainly criminal law does. Despite the fact that the federal system holds only 10% of the nation's inmates, it surely receives well over half of  the non-empirical scholarly attention, most likely signficantly more than half (though I don't want to say more than that, since it's probably bad form on a blog about empirical work to throw in unsubstantiated guesses based on little more than anecdotes and gut instinct). Already, for example, it seems like Booker is receiving more attention than, or almost equal attention as, Blakely, even though the latter is  more important and more far-reaching.

But I feel like this is likely the case in other areas as well. My impression is that much more is written about the behavior of federal judges (the Supreme Court in particular) than state judges,  that many more pages have been dedicated to interest group behavior at the federal level than at the state, and so on. Which leads me to a series of questions:

1. First, is my perception right? Is too much empirical work focused on the federal system? I guess first I'd have to define what "too much" is. A simple mathematical ratio might be too crude -- since 10% of all criminal sentences are handed down by federal courts, should only 10% of the empirical work on sentencing focus on the federal system? That can't be the right way to think about it. It also depends on influence: for example, maybe it makes sense to focus on the behavior of US Supreme Court justices more than state supreme court justices because the US Supreme Court opinions have more weight. But I wonder about that: do they? Certainly, on the particular issue addressed, the Supreme Court's opinion is dominant. But as we go about our daily lives, which has more of an effect on what we can or can't do, state or federal opinions? More generally, which have a greater quotidian role, state or federal legal outcomes?

But even if we could in theory develop a metric of percent-of-cases weighted by importance-of-cases and found that research was slanted towards federal outcomes, that's not necessarily a problem. Which brings me to my second question:

2. How comparable are federal outcomes to state outcomes? If we expect behavior to be similar in both situations, then maybe it doesn't matter which we study. But I'm skeptical of that claim. Interest group behavior in the states can't be the same as at the national level: states are more financially constrained; their budgets are allocated in different proportions; different groups are going to wield influence in different states due to variations in industry, location, or environment; legislatures vary in professionalism and makeup; and so on.  Or consider judicial behavior. Federal judges have life tenure and, also importantly, are drawn from a significantly different pool of possible applicants, and it seems likely that they would correspondingly behave in systematically different ways.

Certainly in the criminal context, federal results shed, I think, very little light on state outcomes. Federal jurisdiction is limited, the US Attorney can cherry-pick cases, and the BOP's share of the federal budget is nominal. Plus, seemingly even post-Booker, federal sentencing looks nothing like state sentencing, even as many state sentencing regimes are quite similar.

So if there is "too much" focus on the federal system and if federal results do not extrapolate easily to state outcome -- both of which, of course, are empirical claims that require proof -- then I have a third question:

3. Why? Is it just a bias that comes from law school, which often has a strong federal focus? Or, and this is the answer that could be more likely and is to me much more troubling, is it a matter of data? When I compare the data the US Sentencing Commission and FJC collect on federal criminal cases and compare it to the state-level sentencing data that I troll around in, I think my skin physically takes on a green hue. Is this the case in other areas as well?

And is the problem actually collection and availability (some states have almost no reporting requirements for state interest groups, making it close to impossible to compile meaningful data for those states)? Or is it consolidation? It's tough to track the legal changes occuring in 50 states, to gather data from all 50, and then to try to merge data gathered using different definitions, reporting procedures, and so on.

If the problem is the former, no easy solution presents itself, short of lobbying for greater data collection or trying to gather the data "by hand" in some way. But if it's the latter -- if the data is out there, but it's hard to collate and messy and time-consuming to use -- what are the implications?

So I guess to collapse all this to a series of simple questions, I'm really curious:

1. Do we focus "too much" on federal outcomes?
2. If we do, does this mean that we are not developing results that explain either the impact of or the forces behind the legal changes that actually play a bigger role in people's lives?
3. If so, how can we rectify this? In particular, if it's a problem of data availability, how can we get the numbers we actually need?

Comments

[Gritsforbreakfast]

I agree too much attention is paid to the federal system, but as a state-level public policy researcher on criminal justice topics, I think the lack of data issue is a cop out. Whatever happened to primary research, fer God's sake?! It's as though if data isn't handed over on CD-Rom from the US sentencing commission too many academics won't bother to gather it, much less wade through it. State court systems generate loads of data, and open records laws are typically much stronger and more timely than the federal FOIA, so go get the raw data you need and crunch it. I still do lots of research that starts with stacks of paper and ends up with an Excel spreadsheet.

By comparison, focusing on the federal system is not only easier, it's also higher-profile - basically glam scholarship by comparison to state-level work. The rarified setting allows law profs to pretend to engage the outside world while remaining above the more significant fray going on in the state courts.

That's too bad. "As the shadow of the law grows fainter," we have greater need of people with real expertise to provide analysis and insight. Too often that's when many of the nation's top legal academics seem to lose interest.

[Peter Wimsey]

The "federal bias" in law review articles (and general descriptions of criminal law in media reports as well) is something that has generally bothered me as a criminal law practitioner at the state level. I'm oversimplifying, but federal criminal law seems to focus on: (1) large scale drug or racketeering operations; (2) violations of particular federal statutes (i.e., SEC violations, tax evasion); or (3) mail fraud.

Although drug prosecutions are common in state courts (duh!), other bread-and-butter state criminal law issues (battery, robbery, burglary, murder) seem to get short shrift.

Federal policing, investigatory, and prison policies also have only a slight connection with their state counterparts.

I'm a practitioner, not an academic, so I don't know exactly how to remedy the federal law bias. However, part of my job involves advising state officials, including legislative officials, on particular approachs taken by other states in dealing with criminal law issues. While I often end up providing people with a copy of, say, Wisconsin's domestic battery statute along with a newspaper article describing it, it would be useful if there were more applicable law review articles on state criminal law issues. Sometimes this does happen - there were numerous articles dealing with Apprendi/Blakely/Booker - it can be very helpful.

Ian writes -
For me, federal criminal law and practice is much easier to write about because it is simply more law like. The cases observe the form of law much more closely and legal rules more often appear to have substantial impact on the outcomes. It seems to me that federal criminal practice is analytically more accessible to academic lawyers. Even if the best analysis takes us beyond doctrine, at least there are substantial points of contact.

The state practice with which I am familiar, on the other hand, is much harder to even begin to explain in legal terms and concepts. As the shadow of the law grows fainter, I wonder what I have to bring to the analysis.

Could you expand on this a little bit - I don't understand why state practice should be "harder to explain in legal terms and concepts" than federal law.

[David Zaring]

I doubt my observation is new, but one advantage of federal study is a narrower issue area on which your potential readers have purchase. This applies in a number of different contexts. In other areas of federal lawmaking the Supreme Court is hardly a category leader, but it's easy to read (or count) Supreme Court opinions. Does corporate law, for that matter, turn on what happens in Delaware? More so than the SEC? But I bet there's more studies (and more downloaded studies) on the decision to incorporate or reincorporate in Delaware etc. Not that there's anything wrong with such studies, but I take it your concern is one of emphasis.

[Ian Weinstein]

My colleague John's observation is both correct and important. I am a federal practitioner who has moved into a state court clinical practice and written about both federal and state law and practice. For me, federal criminal law and practice is much easier to write about because it is simply more law like. The cases observe the form of law much more closely and legal rules more often appear to have substantial impact on the outcomes. It seems to me that federal criminal practice is analytically more accessible to academic lawyers. Even if the best analysis takes us beyond doctrine, at least there are substantial points of contact.

The state practice with which I am familiar, on the other hand, is much harder to even begin to explain in legal terms and concepts. As the shadow of the law grows fainter, I wonder what I have to bring to the analysis. To the extent John is urging those with real expertise in other disciplines to pay attention to state criminal law and practice, he is absolutely right. That is where most Americans meet the criminal law. To the extent he asks why academic lawyers don't pay much attention to state practice, in addition to the excellent reasons offered above, I add my personal observation that my core expertise in doctrinal analysis and critique seems more useful in analyzing federal criminal law and less useful in the state settings with which I am familiar.

[Ross Cheit]

My view from political science is that the state politics section, of which I am a rather inactive member, is still pretty small and there's not a lot of scholarship on state politics that garners attention.

I've done a number of empirical studies, civil and criminal, using state trial court data. But I'm in a state the size of county, so the usual barriers to statewide studies are mitigated; statewide data are actually in one management information system.

State court studies definitely have the advantage of being where the action is, but the data issues even in my state are sometimes daunting.

I've also done archival research in state and federal courts. The National Archives is what you would expect from a legal archive. Research is a breeze; things are indexed and where you expect they will be. That has rarely been my experience in state courts. So I suspect the federal bias also has a lot do with access and other practical considerations.

[Chris W. Bonneau]

For years, political science exhibited the same skew toward federal courts (specifically, the U.S. Supreme Court.) However, over the past decade or so, there has been much more focus on state courts--both at the trial and at the state supreme court level. This is not to say that nobody was doing this work before then, but it seems like more people are discovering the virtues of studying politics in the states, where institutional variations have been found to have profound effects on judicial behavior.

[Dan Filler]

I think much of this analysis is right. I also suspect that some of the bias results from the fact that most people from the top law schools - the ones that produce law profs - consider federal criminal jobs more attractive. Thus, law profs are more likely to start their careers in US Attorney jobs than in state prosecutor positions. There are both good and bad reasons for this skew, but the net result is that most criminal profs think federal first. There is a load of state data out there - as Ron Wright and Marc Miller have shown - but access would surely be easier for alumni of those state systems. And those alums aren't writing very many law review articles.

I'd be curious whether research being done by non-JD's - like political scientists - exhibit the same bias.

[frankcross]

Don't know a lot about criminal law, but I suspect you are right. Though there are some legitimate reasons, in some areas of research. There are data reasons -- I suspect more data is available at the federal level and there may be comparability problems when using data from multiple states. Also, there are significance reasons. A USSC justice is more important than a state justice. I suspect more people paid attention to the Alito confirmation than to their own state's supreme court selection, not to mention that of other states.

In law reviews, there might be a publication bias. If I wanted to study Texas courts, it might be tougher to interest, say, the Virginia Law Review.

[Ron Wright]

Yes, it's a persistent problem. Some of it derives status concerns: will a top-tier law review publish my piece if it deals with only one or two states (and not the home state of the journals in question)? As you know, in lots of other disciplines this is not a problem. In criminology, for instance, data from one state can form the basis for serious theoretical and empirical work with implications that could apply in lots of places.

How to rectify this? In the criminal justice area, part of the answer comes from state sentencing commissions that produce pretty good data. Not as rich as the federal system, but plenty of interesting work is possible based on what they're already collecting.

[Michael Heise]

John: Good questions, all. My quick $.02 follows.

1. Yes.
2. Yes.
3. Yes, for me, anyway, it's largely a function of data availability. As to the "how" sub-part of your query, I haven't a clue. All I know (or *think* I know), however, is that high quality empirical work requires high quality data.