New Article on Affirmative Action in Higher Education

In light of Richard's discussion last week on affirmative action, I thought I would post the abstract of a new paper by John Lott and Mark Ramseyer (that I hope to personally read soon), see here, which challenges some of the traditional hypotheses about peer effects with respect to affirmative action in higher education.  The SSRN abstract sounds interesting:

In the Grutter case, Justice O'Connor suggested that universities could justifiably try to enroll a critical mass of minority students. Enroll fewer than that critical mass, reason some observers, and minority students will feel too marginalized to perform at their highest levels. In this article, we test whether minority students perform better with other students from their ethnic group in a class or school. To do so, we assemble data on the ethnicity and performance of each student in all classes at two law schools - for three years at one, and for sixteen years at the other. We find no consistent evidence that having additional students from one's ethnic group raises a student's performance. Instead, we find some evidence that having additional ethnic peers lowers performance - albeit by a very small amount (US, Canada).

Comments

[Richard Lempert]

The Lott-Ramseyer paper, which in some ways I like, nonetheless further illustrates my theme last week of needed cautions in writing on and reading about affirmative action. In this case the two cautions I would add for readers of this and empirical studies more generally are: (1) Don't reach conclusions from abstracts and (2) ask how variables are operationally defined and realize that concepts as operationalized may not be the concepts one is interested in or how concepts are used in policy discussions.

What's good about the paper in my view is (1) that it takes seriously a question often assumed in the affirmative action debate about the value of a "critical mass," (2) it has asembled a unique data set for analysis, (3) the authors have set out to test hypotheses and nothing about the analysis suggests they were seeking to make a case either for or against affirmative action, (4) the authors are properly restrained in presenting their results, (5) the article considers all affirmative action minorities and not jsut blacks, and (6) the authors seek to apply methods appropriate to their data and hypotheses.

Reported results are essentially that the number of fellow ethinics a student has has virtually no effect on how black (and other minority students, but I shall use blacks as the example) perform because grades either do not vary significantly with the number of minority students in a class, course or school or, to the extent that relationships are significant, effects are so small that they matter little.

What's probelmatic about the study is that it doesn't and cannot address the issue highlighted in the abstract and the introduction of the paper; that is, it does not and cannot provide the promised assessment of the importance of a "critical mass" of black students. The data set consists of grade, class, race and gender information for students at two law schools, one for three classes and the other for 16. The study cannot address the question of interest, however, because neither school has what one might call a "critical mass" of black students. At the school with 3 years of data, blacks accounted for only slightly more than 3% of all students, or about 9 in a class. At the school with 16 years of data, blacks accounted for about 1.9% of all students or an average of about 3 in a class. Thus at neither school is the number and proportion of black students high enough so that there is what one would call a critical mass, either in the school as a whole or in all but a handful of individual classes. Hence the study should not style itself as a study of the effects of a critical mass, which implies a number that exceeds a certain threshhold that is certainly aove 3 in an admissions cohort, and in my view also above 9. (e.g. at the larger school this means that across the three cohorts attending at the same time 28 of 873 students are black; at the smaller school there would be an average of only about 9 black students across 3 cohorts.) This paper would be more properly styled as a study of how increases in the number of blacks (and other minorities) over a small, low range is affected by increasing number.

A second issue would persist even if the schools studied contained critical masses of black (and othe rminority) students. This issue involved how the results attributed to the presence of a critical mass are operationally defined. Here they are defined solely in terms of grades. But those who argue for the importance of having a critical mass of different groups of minority students are typically thinking not just (or even mainly) of how the presence of a certain number of fellow ethnics affects a student's grades, but also of the comfort level of black students in a school, their willingness to speak up in class, how whites think about blacks, the degree of within school support for journals and clubs oriented to issues of race, civil rights, etc. There is nothing wrong with operationalizing one possible and hypothesized benefit of critical mass as graded classroom performance, but the limited nature of that operational definition should always be front and center. No study that deals only with this aspect can claim to have resolved the issue of whether a critical mass benefits white students, black students,classroom discussion, etc..

Finally the study raises an interesting statistical issue. Smaller numbers of minorities can be expected in smaller classes, especially seminars. Grades are typically higher in such classes than in larger ones. The authors try to control for this with a fixed effects model, which at the individual class level takes account of the tendency for grades to be higher for everyone in seminars and other small classes. This is a strength of the paper, but I don't think it does all the work intended. Seminars and small classes are more likely than larger ones to be graded on the basis of papers, oral participation, partnered projects, etc.. If all students were equally advantaged or disadvantaged by this, there would be no problem for the fixed effects model used. But at least some people with some reason (e.g. stereotype threat) believe that blacks may be differentially disadvantaged relative to whites and to their own inherent ability in high stakes, one time-limited test for a semester grade, than they are in small courses graded on the basis of papers, oral participation, etc. In other words, grade effects associated with class size may not be the same for all students but may differ within courses by ethnic group. The fixed effect control for individual class and controls for class size will smooth out this interaction effect. If so, the fact that the number of blacks in a class does not affect grades positively as black representation increases and appears to have a very small but statisticlaly significant negative effect may nonetheless reflect biases associated with the likely correlation between class size and course nature and the number of blacks in a class. This, of course, need not be true, but the model does not help us decide.

A major difficulty the authors faced is that they only had data on two law schools, both of whcih had relatively small absolute and proportionate black enrollments. One hopes that they will be able to find numbers of other scholls willing to provide the data they need because with many more schools they may be able to shed light on the interesting question they tackle. But even if they acquire more data, one must still keep in mind the limited nature of any grade-based assessment of the value of having a "critical mass." Complementary data on minority student attitudes, participation, drop out rates, extracurricular activities etc. also bear on the critical mass issue, and to assess the value of critical mass for policy purposes it is as important that data on these kinds of outcome measures be collected.

Rick