Grades and the mismatch effect - Sander 6

Previously: Part 1, 2, 3, 4, 5

In my third post, I summarized the “grades” argument: preferences cause black students to have much lower grades; low grades lead ineluctably to a variety of bad outcomes; and grades matter more than school prestige in predicting outcomes. See that post for more detail and the theoretical context of the argument. Here I’d like to give a few empirical examples and discuss the critics.

The two tables below provide a nice intuitive illustration of how low grades produce the mismatch. The first breaks out, for each law school tier, the relationship between law school grades and the odds of graduating and passing the bar on the first attempt. For example, 27% of the students in the bottom 5% of Tier 3 (Duodecile 20) graduated and passed the bar on their first attempt. It’s pretty obvious from this table that the bottom of one’s class is a lousy place to be.

(Note that in both Table 1 an 2, below, the lowest grade demidecile is 20; smaller-numbered demideciles mean higher grades.)

Table 1. Proportion of matriculants, by school eliteness and grade demidecile, who graduate and pass the bar on their first attempt, BPS data

Grades demidecile	Tier (1 = most elite)
	6	5	4	3	2	1
20	10%	10%	22%	27%	33%	59%
19	21%	36%	40%	51%	68%	74%
18	19%	42%	56%	67%	71%	81%
17	26%	48%	67%	77%	73%	84%
16	31%	54%	70%	84%	81%	85%
15	35%	58%	82%	83%	82%	90%
14	30%	72%	79%	88%	87%	96%
13	41%	73%	86%	85%	88%	95%
12	45%	88%	84%	91%	89%	96%
11	43%	81%	90%	94%	91%	99%
10	62%	85%	89%	94%	96%	98%
9	67%	90%	93%	95%	93%	98%
8	77%	92%	93%	93%	96%	97%
7	79%	92%	93%	96%	95%	98%
6	77%	87%	95%	94%	96%	100%
5	90%	88%	96%	97%	97%	98%
4	86%	89%	96%	98%	97%	96%
3	86%	94%	97%	97%	97%	98%
2	93%	94%	96%	96%	98%	98%
1	97%	95%	97%	98%	99%	100%

This next table (Table 2) contains the same categories as Table 1, but each cell shows the average admissions index (a weighted average of LSAT and undergraduate GPA) of students with those law school grades at that school.  (Both of these tables omit students who graduated but never took a bar.)Obviously, there’s a relationship (though a far from perfect one) in each tier between index and grades.

Table 2. Average index of matriculants, by school eliteness and law school grade demidecile, BPS data

Demidecile	Tier (1 = most elite)
	6	5	4	3	2	1
20	503	567	627	628	680	745
19	538	603	650	683	723	793
18	545	630	667	712	749	805
17	535	637	675	730	753	818
16	509	642	687	752	768	853
15	514	647	694	751	766	846
14	586	645	702	762	784	843
13	565	649	707	759	781	853
12	528	647	711	771	783	855
11	563	647	711	769	765	862
10	565	662	716	770	791	857
9	610	669	723	785	803	858
8	557	660	722	781	801	873
7	598	672	722	788	807	876
6	618	683	732	789	817	864
5	599	684	739	797	814	874
4	583	701	737	799	822	876
3	642	692	748	799	823	877
2	660	720	755	812	819	863
1	676	727	777	827	829	854

Here’s the point: compare someone with the same index at differing tiers (in Table 2), and look at the outcomes of those cells in Table 1. For example, the cohort with an average index of 680 in Tier 2 schools has, in the aggregate, a 33% chance of graduating and passing the bar on the first attempt. A similar cohort (683) in Tier 3 has a success rate of 51%. A similar cohort (675) has a success rate of 67% in Tier 4, and the closest cohort in Tier 5 (683) has a success rate of 87%. In each case, lower tiers are associated with better outcomes for what appear to be similar cohorts.

This doesn’t mean, of course, that it’s always better to go to a less elite school – once one’s cohort has an index close to a tier’s mean index, the benefits (in terms of graduation and bar passage) of going to a less elite school are small. But a cohort associated with low grades at a given school will do much better at a less elite school.

There have been two sorts of critique of the “grade” argument. The first was from Dan Ho, who argued here that my regressions comparing the power of law school grades and law school eliteness were invalid, because one’s grades are influenced by the eliteness of one’s school. I think Ho simply didn’t realize that law school grades in the BPS dataset are standardized by school (so my measures of GPA do accurately measure performance relative to one’s peers within each school); if one digests that fact, the argument goes away. Moreover, one can use many different methodologies to demonstrate that grades trump school eliteness in predicting outcomes – Tables 1 and 2 above, for example, as well as analyses here and in "Mismeasuring the Mismatch" (posted here).

The other critique of the grade argument is that credentials are poorly correlated with grades; thus, there’s little if any relationship between the admitted “credentials gap” caused by racial preferences and the “grade gap” between blacks and whites.

This really shouldn’t be a debatable issue anymore. No one has tried to dispute that, for groups (as opposed to individuals) the correlation between credentials and grades is extremely high (.88 and upwards; see Systemic Analysis, Part IV). No has disputed Anthony & Liu’s finding (which can be found here , report TR-00-02) that black underperformance is a very small factor in explaining black grades (their finding implies that about one-tenth of the black-white grade gap might be due to underperformance). Moreover, it appears that when one takes into account differences in college quality, even that small underperformance diminishes or disappears.

Many critics won’t concede this issue because, if they do, it concedes the entire debate on the mismatch effect.

Comments

[Rick Lempert]

Rick,

Can you redo Tables 8 and 9 with just black students, as it is their performance we are interested in, and include the cell sizes so one gets an idea of how stable the figures might be.

Rick