Instrumental Variables versus Matching Techniques

A few weeks ago, I posted on the Advanced Empirical Methods Workshop at Washington University.  I briefly mentioned one of the presenter's comments that non-parametric matching is a better way to deal with endogeneity than instrumental variables (IV) in a linear model.

That post prompted an email from one of the leading econometricians in the legal academy, who noted that the pros and cons of IV versus matching was a matter of open debate in which the two camps were roughly divided by discipline--political science favoring matching and economists relying on IV.  [For the record, my correspondent claimed that both techniques had their strengths and their relative superiority depended upon the research question and the available data.]

The basic problem that both techniques are grappling with is accurate causal inference from observational data.  In a controlled experiment in which subjects are randomized and divided into treated and untreated groups, we can be relatively confident that subsequent differences (Y) among the two groups are attributable to the treatment (X).  But many social phenomena are not amenable to controlled experiments.    Thus, it is difficult to ascertain whether some unobserved or unmeasurable attribute is really driving our results, thus biasing our parameter estimates for X. 

Matching techniques attempt to achieve the same effect as experimental randomization by "balancing" a sample on all observable attributes.  [Ed Morrison (Columbia Law) blogged on this technique a couple of months ago.] The key assumption is that all unknown/omitted attributes are similar within the matched sample.  In contrast, when there is a good theoretical basis to believe that a particular variable (X) is correlated with a missing/unobservable variable (Z), IV attempts to find an instrumental variable that is correlated with X but uncorrelated with Z.  By substituting the IV for X, the potential endogenous relationship is eliminated.

Critics of IV claim that good instruments are very difficult (or impossible) to find.  But, as my econometric colleague pointed out, occasionally some unusual event occurs that creates an natural experiment--and this "shock" can supply the ideal instrumental variables. 

All of the above is a long way of introducing this terrific article from the Wall Street Journal (subscription required) on how economists have used natural experiments and instrumental variables to make great headway in explaining the causal mechanisms of a wide array of social phenomena  (Hat tip to Tom Ulen at the Law & Economics Blog). 

Under the fold is a highly informative WSJ chart [click to enlarge] summarizing some of the leading IV studies.   

Comments

[Mike Guttentag]

It seems to me that one of the most interesting applications of an instrumental variable in the context of the empirical study of the law is the series of articles by LaPorta, Lopez de Silanes, Shleifer, and Vishny. In these articles, these authors argue that the quality of legal protections is an important determinant of the strength of a country’s capital markets. They offer an instrumental variable analysis to support their argument, using the source of law as an instrument. These are exciting arguments, but legal academics have failed to my knowledge to seriously engage the issues as to the validity of their instrument. I think this is a question that deserves further discussion.

[Michael Heise]

Interesting post, Bill. Your observation about the discipline-specific (economists v. political scientists) aspect of the (albeit rough) opinion split on IVs and matching reminded me a bit about the old SPSS v. SAS "battles."