I would like to begin by thanking the Editors of the ELS blog for inviting me back for a repeat performance. I am going to use this opportunity mostly to talk about issues that arise in empirical investigations of affirmative action. I will use this kick-off contribution, however, to talk specifics and to correct misinformation that exists in the literature, which was unfortunately widely publicized in the New York Times. This misinformation stems from an article that Professor Richard Sanders published in the North Carolina Law Review that examines the career paths of black attorneys who begin their careers at large (100 lawyer +) law firms and reports that that these young attorneys leave large firms at very high rates relative to whites. This finding is later used to question the skill levels of young black attorneys and as evidence of adverse consequences, both in law school and thereafter, to affirmative action.
I wish to be clear at the outset that although I think Professor Sander’s argument misleads because crucial data are wrong, I do not mean to criticize Professor Sander for using these data when he did. At that time the data Professor Sander relied on seemed to be the best publicly available data, even if they were not the best suited to his purpose. Moreover, not only do the assumptions he made to justify relying on his data appear at least plausible, but he also attempted to check his most crucial inference with other publicly available data. Indeed, it is precisely because his efforts to be careful lend plausibility to his use of the data and consequent conclusions that it is important to correct the data record.
In his North Carolina Law Review article, Professor Sander suggests that black associates leave large (100 lawyer +) law firms in their first few years and before the partnership decision at dramatically higher rates than whites. While there are a variety of plausible explanations for this, including the arguments that blacks suffer from prejudice and hostile atmosphere and the claim that most young large firm associates are looking for ways out and blacks simply have more attractive opportunities to leave than whites, Sander’s preferred explanation for at least a large part of this disparity is that black associates are less well equipped to do the work large firms demand than whites and that this is in large measure attributable to law school and law firm affirmative action.
To support his claim, Professor Sander presents data indicating that the proportion of large firm associates who are black drops from 8.1% at initial hiring to 4.7% by about the 3rd year to somewhat over 1% at the partnership level. The partnership data are not in my view terribly consequential as a measure of lawyerly capacity, since, to judge by turnover statistics and considerable anecdotal evidence, most lawyers who start in law firms, whatever their race, want out long before the partnership decision, and rain making, social acceptability and other factors likely to be biased by race figure importantly in who makes partner. In addition, partnership proportions are distorted by cohort effects, since many current partners attended law school when black representation in schools that feed large firm practice was sparse. However, the early year record seems consequential, and the disparity between white and black dropout rates as Professor Sander reports them seems sufficiently large that explanations suggesting black associates are less able than others to handle large firm work have an intuitive appeal, even if hostile atmospheres, other opportunities and similar explanations might also explain a share of the drop.
There is, however, another explanation for the drop. Professor Sander had no data on the proportion of large firm starting associates who were black. Rather he had data on the proportion of very large firm (Am Law 100) summer associates who were reported by their firms to have been black. He then argued, not unreasonably, that since large firms commonly extend permanent offers to those who work for them in the summer, the summer associate data provided an accurate and unbiased estimate of the proportion of starting associates who were black. To reinforce the assumption of accuracy, he noted that the proportion of all summer clerks in his data who were reported by firms as minorities (blacks, Hispanics, Native Americans and Asians) was close to (only 1.5% higher than) NALP’s published figures on the proportion of starting associates at firms with more than 100 lawyers who are minorities. He further suggested that the small discrepancy in overall rates could be explained by the fact that his summer associate data were from very large firms and that these firms could be expected to hire a somewhat higher proportion of minority associates than large firms in general.
Sander’s numbers and argument for extrapolating from the summer associate numbers appeared plausible to me, as I expect they did to most readers. However, reading a relevant NALP report, I realized that NALP had the capacity to break down its starting minority associate figure further by race, though its relevant published data had lumped all minorities together. I asked NALP if they could provide me with the relevant break downs, and they did. What I found was that according to the NALP data, the proportion of starting large firm associates who in the years Sander studied were black was not 8.1%, as Professor Sander assumed, but rather 6%. Moreover, when one looked only at very large firms, like those that had provided Sander’s summer clerk data, the proportion of starting associates who were black was slightly lower than it was in all 100+ firms rather than higher, as Sander had assumed. I also realized that Sander’s argument did not account for the fact that if no new associates left firms in the first two years, the proportion of black associates would likely still diminish since whites are proportionately more likely than blacks to hold judicial clerkships, and many students enter large firm positions after their clerkship. Putting this information together we see that about 2/3 of the fall off that Professor Sander pointed to as evidence that blacks are less able than others to keep up with the demands imposed by large firm practice is illusionary, a result of Professor Sander’s understandable but mistaken reliance on summer associate statistics for his starting associate proportions.
The difference between the proportion of blacks who start at large firms and the proportion of associates who, by about the 3rd or 4th years, are black could still, in some measure, be attributable to Sander’s preferred causes, but the fact that his explanations cannot be ruled out do not make them compelling. The data are equally consistent with a variety of rival hypotheses related to racially structured differences in the experience of large firm practice and the availability of other, more attractive alternatives. Moreover, whatever differences exist between the career patterns of young black and white associates within large firms, they appear to be far smaller than Sander’s North Carolina Law Review article would lead one to believe. My own sense is that because so many young lawyers – white and minority – want out of large law firms (I saw a statistic reporting that large firm turnover rates are 80% in the first 5 years, but I cannot vouch for its accuracy) and because there are so many reasons why lawyers leave large firms, the study of large firm drop out rates has little or nothing to say about the legal competence of lawyers and nothing at all to say about the implications of law school affirmative action.
I should point out that Professor Sander seems still to believe that the estimate he provided in his NCLR article is accurate. In a letter to the reporter who wrote the New York Times article, he acknowledged the fact that the NALP survey found that only 6% of starting large firm attorneys were black, but he argued that these data were nonetheless consistent with his much larger summer associate-based estimate. His claims were (1) in the years he examined (2001-04) firm size information was missing on proportionately more starting black lawyers than starting lawyers of other races and that when respondents with missing firm size data were distributed in proportion to the number of lawyers in firms of different sizes the proportion of black associates in large firms rose from 6.0 to 6.2%; (2) that black graduates were less likely to respond to the survey than non-black graduates and that when non-respondents were distributed across firm sizes in the same proportion as respondents, the proportion of starting associates who were black rose from 6.2 to 6.8% (Professor Sander also argued that the NALP data were biased by school non-response. I question his argument, but as he did not offer a specific correction based on it, I shall not discuss it further.); (3) that self-identified multiracial graduates were reported separately in the NALP data but were most likely counted as black in the summer associate data, and that if the estimated 1/3 of multi-racial lawyers who are part black were counted in the NALP data as black, the figure of starting associates who were black would rise from 6.8% to 7.0%; (4) and finally, and most importantly, that NALP reports employment status as it exists at the time of their survey in the February after graduation. If, he argued, one assumes the law firm employment year begins after law school on July 1 and that 6.4% of large firm associates after 1 year are black, then the “data best fits (sic) an attrition model in which blacks make up 8.1% of the associates” who started their employment on July 1. In short, he claims, the summer associate data from very large firms provides a better measure of the proportion of starting large firm associates who are black than the NALP data which seek to directly measure this.
I assume Professor Sander’s math is accurate, but I find his arguments unpersuasive. The first three proposed adjustments have the same weakness. They assume that non-respondents will be distributed in the same way that respondents are. It seems plausible to me that lawyers of all races would be more likely to respond to the NALP survey and more likely to indicate firm size if they were working in large firms than if they were working in smaller firm settings. After all large firms are prestigious – having a job at one is something to crow about. One might as easily speculate that white liberals are ashamed of working in large firms and so are particularly likely to refrain from responding or, if they respond, from listing firm size. If so, the ability to correctly assign non-respondents to firms might lower the proportion of starting associates who are black. I would be astonished if the liberal non-response hypothesis were true, but I would also be surprised if employment and firm size had no influence on propensities to respond. (Note in the NALP data the difference between blacks and other respondents in response rates largely reflects a difference between blacks, on the one hand, and whites, who are numerically dominant in this survey, on the other. Black non-response rates are not always higher than the non-response rates of other identified minorities, so it is a mistake to think that blacks are always the minority least likely to respond.)
Possible error in the third adjustment works somewhat differently. Here Sander assumes that someone who identifies as a part black multiracial is as likely to be working in a large firm as someone of a different mixed racial heritage. But this assumption too is tenuous. Asians, for example, are more likely to take large firm jobs than lawyers with other backgrounds, and they are a group that often intermarries. It is possible, if not likely, that those large firm attorneys who self-identify as multi-racial are disproportionately part Asian, with no part black. Moreover, the AM Law 100 firms may have classified some of their part black summer associates as Asian, Hispanic or white or failed to provide any race data for them.
But it is Professor Sander’s fourth correction that does the heavy lifting, and it is this correction I find most problematic. One problem is that Professor Sander starts his hypothetical employment year before most young law school graduates will have taken a bar exam, and perhaps 4 to 6 months before they will have been admitted to the bar. A far more serious flaw is that the argument assumes that 30% of blacks who start at large firms will drop out each year over a 7 year period, a figure that Professor Sander admits in his NCLR article is “made up.” To the extent this assumption is justified at all, the justification seems to be that a 30% annual attrition rate from an assumed starting proportion of 8.1% yields a black proportionate retention figure after 2 to 3 years much like that found in other NALP survey data, and after 7 years a black retention proportion similar to the proportion of partners in Am Law 100 firms who were black in a 2002 survey. Although the 30% attrition rate is “made up,” it does have the nice feature that if 8.1% of large firm associates were black and if only 6.4% of these associates were black after 1 year, and if the chance that a person would leave a large firm was the same on the first day of the year as on the last and on each day in between, then in February at about the time of the NALP survey one would expect about 7% of the associates remaining in large firms to be black, which is the figure Sander manages to reach through his first three adjustments.
This same argument is used in Table 22 in Professor Sander’s UCLA article to extend the extrapolation to 7 years, at which time 1.7% of large firm associates are predicted to be black. The weaknesses of the extrapolation can be better appreciated in the context of this longer time frame, but they apply to the first year as well all as to all seven. The fundamental flaw is that the model assumes that the chance an associate will quit or be fired is the same on his or her first day on the job or the day before a bonus is distributed or new partners announced as it is on any other day during the first seven years a lawyer is with a firm. It assumes that people are as likely to leave large firm practice before they have even passed the bar, or before they have acquired marketable skills and connections as they are in their third or fourth year. It is inconsistent with older studies that show associates staying until the partnership decision and then quitting or being shown the door if they haven’t made partner as well as with anecdotes about numbers of lawyers leaving firms shortly after bonuses are distributed. Moreover the front loading of departures that the model implies is inconsistent with Professor Sander’s argument that the attrition of blacks from large farm practice occurs because they lack the skills this practice demands. In their first nine months, and especially before they have passed the bar, new lawyers are learning the ropes and much of what they do will involve such routine activities as reviewing records for discovery or accompanying partners at depositions. Whether black or white, if they leave their recently acquired positions so early, it is unlikely they are leaving because their intellectual shortcomings have been exposed to either themselves or others but rather because they have discovered that the life they must lead – and would expect to be leading for the next several years – is not worth the money they are earning. In short, Professor Sander’s fourth “correction” of the NALP data is no correction at all, and there is no reason to believe that using the black summer associate data to estimate of the proportion of first year associates who are black is anywhere near correct.
If the NALP figure on the proportion of starting large firm associates who are black is on the low side, it is likely that it is only slightly low, and as of today it constitutes the best estimate we have. In examining the relatively higher departure rates of blacks than whites from large firms, the best currently available bookends are the NALP estimates that about 6% of attorneys who start at firms with more than 100 lawyers are black and, based on another NALP survey, by early in the 3rd or 4th year about 4.6% of those associates remaining are black. (If we look just at very large firms, relative attrition seems to be a few tenths of a percent less). As measures of relative attrition, even these figures are ambiguous, however. Not only are there potential biases and some degree of sampling error in these data, but they are also consistent with a world in which black and white associates leave large firms at the same rate, but whites are proportionately more available (returning judicial clerks) or willing than blacks to be recruited as replacements. In short, we who wish to study attrition from practice need better data, and until such data are available we should not rest arguments heavily on speculation.
None of this is to deny that black associates almost certainly leave large law firms earlier and at higher rates than whites and that there most likely is a problem here that deserves attention. But the degree of black attrition is not nearly so great – relative to that of whites – as Sander’s NCLR data suggested, nor do the data provide convincing evidence that blacks beginning at large firms are not up to the tasks of an elite practice or that black departures from large firms relate in some way to affirmative action. The overall rate at which young lawyers of all races leave large firm practice suggests that it is more useful to look for causes of leaving elsewhere – in the cultures of large law firms, the ways partners treat associates, dissatisfaction with the clients and causes most large firms serve, and perhaps, above all, the time demands large firm practice places on young associates and the degree to which these conflict with family and social life. If young black attorneys are more sensitive to any of these features than whites or if the culture of large firms is less hospitable to blacks than to whites or if partners are less supportive and less likely to mentor them, this could be the entire story. Indeed, many young attorneys are soon so eager to leave the large firms where they begin that one could make a case that the relatively greater attrition of black attorneys is a sign that they are more able than their white counterparts, perform better in practice and so have earlier and better opportunities to move on. I wouldn’t make such a case. It would be pure speculation. But viewed in the light of the most reliable data available, this conclusion is no more speculative than ones Professor Sander reached in his North Carolina Law Review article about the capacity of young black attorneys to handle the demands posed by large firm practice and the implications of affirmative action.