« ELS and Paternalism #2: On ipse dixits (Blumenthal guest) | Main | Empirical Work on Property Issues (Blumenthal Guest) »

June 07, 2007


Rick Lempert

Professor Sander's analysis in this blog is obviously a first cut with the data he is using, as evidenced by the fact that it revises and improves upon an analysis reported in his initial blog in this series. While the data, on whites especially, seem strongly supportive of the Sander hypothesis that law school grades reflect traits essential to success in large law firms, there is in fact substantial ambiguity in the data as reported. This ambiguity will hopefully be resolved through deeper analyses.

First, the Michigan alumni data set that is the basis for the table aggregates data from a successive cohorts of Michigan alumni surveyed 15 years after graduation. However, for Prof. Sander's purposes, it is essential that cohort graduation year be controlled. This is because, if my recollection of the changing marketplace for Michigan graduates is correct, in the earlier part of the time series large law firm recruitment was concentrated toward the top of the Michigan graduating class and at that time (e.g. my graduation year of 1968) the chances of making partner at the firm one started with were reasonably good and known within 7 years. As the years passed, large firms began to recruit from further down in the class and to make partnerships more difficult to achieve. Moreover, in more recent years law firms have done much more lateral hiring of partners, meaning that a partnership in the firm one started with relates less strongly to one's performance than it once did. Thus a good part of the correlation between law school class rank and original firm persistence (meaning in most cases partnership) may be spurious, a result of the fact that large firms began recruiting more people with lower class ranks during the years when partnership promotions became less likely and lateral moves to partnerships or positions in other firms increased.

Also time sensitive is the definition of large firm employment. While firms of 50 attorneys might be considered large in the early years of the Michigan time series, by the later years firms of this size were considered boutiques. Rather than define large firms with a constant number, changes in firm size might be better captured if for each cohort Professor Sander took the firm size that encompassed the median Michigan graduate (with a minimum of 50 attorneys in the early years) and considered any graduate in a firm of that size or larger to be in a large firm. The measure cannot be ideal because, as I recall, the alumni survey does not report numbers of attorneys but just ranges. Still defining "large firm" by cohort seems a better strategy than choosing a figure that is numerically constant but changes in meaning, such that it no longer defines a large firm, over time.

Equally important is a control for gender, as women seem less likely to remain in large firm practice than men for reasons that most likely have nothing to do with different skill levels. (Prof. Sander's second blog posting which shows women distributed more or less like men in the AJD data set may or may not be a cause for optimism since the situation after a decade may be quite different from what it is after two years. He might crudely check on the likelihood of this possibility by looking at how dissimilar initial positions for men and women in the Alumni data set are after 5 years and compare this with position dissimilarity in the same cohorts after 15 years. Similar levels of dissimilarity would suggest that initial distributions persist. An increase in dissimilarity would suggest that not too much should be made of initially similar job distributions.) The gender control is particularly important if one is seeking to get a handle on minority performance as a far higher proportion of minorities than whites were women. (Over the 1970-1996 time period about 37% of minorities were women as opposed to about 24% of whites.) Since most blacks have relatively low grades compared to Michigan's white alumni, in an uncontrolled analysis gender bias in large law firm drop out could be mistakenly attributed to grades.

Anther helpful adjustment would be to exclude from the base those who began their careers in large firms but are currently in teaching, judgeships, elective office or (possibly) general counsel or associate general counsel positions with large companies. Large firm practice is often a stepping stone to the first 3 of these. While the fourth was at one time a consolation prize for those who didn't make partner in large firms, it was also suggestive of considerable accomplishment since firms would not cooperate to place their associates who did not make partner into the general counsel's offices of their clients if they did not think these lawyers would reflect well on the firm and maintain close relations with the firm. Thus if grades are hypothesized to relate strongly to whatever it takes to be a success in an elite law firm practice, one should not include as negative cases those whose career paths from large firms suggest either that they were reasonably successful within them or that they were using the large firm as a jumping off point for different careers. Certainly no one would assume that those who left firms to enter teaching had poorer legal analytic skills than those who didn't make this move, and movements to judicial or political careers have such attractions for certain people that no inference of an inability to succeed in the large firm context should be made from making this move. The latter is particularly important in work that seeks to examine the success of minorities in large firms, for the visibility of minorities employed by large firms often makes them prime candidates for movement to these kinds of prestigious public service.

I have never worked closely with the Michigan Alumni Data set but I thought it contained information on current partnership status and firm size. If so rather than looking at original large firm persistence after 15 years as it relates to class rank, one can look at those who started in one large firm and 15 years later are partners at that firm or another large firm. (Note that for reasons indicated above, the definition of large firm need not and probably should not be the same for original and current position.) In a world where law firm mergers and the lateral hiring of partners has become more common, any large firm partnership after 15 years seems a better measure of success in large firm practice than a partnership in the original firm after 15 years. Indeed, since one can persist in an original firm as a part timer or associate, the equivalent measure would be any membership in a large firm after 15 years.

Finally Professor Sander is correct that there are too few blacks in the UM alumni data set who began in large firms to make much of the grade/partnership correlation in just th black population. This is even truer when one applies necessary cohort and gender controls. Thus I doubt if the table on black grades and law firm performance that Prof. Sander presents in his blog above is worth presenting. But since he has presented it, I would ask that he present the information for blacks as he does the information for whites, by all deciles and let readers make their own aggregations. If an N of 4 is sufficient for the top 4 deciles and an N of 9 for the middle 3, surely the N of 30 for the lowest three deciles is sufficient to allow the disaggregation of these deciles. Since it is in the bottom 3 deciles where blacks hired by large firms are concentrated, it would be interesting to know if within the ranks of blacks hired by large firms, there was a strong relationship between law school grades by decile and persistence to partnership. (One might also note that any judgments of the relationship between grades and advancement to partnership for those with low grades is made more difficult by selection bias. For example, whites hired by large firms in the lowest deciles may be hired because they are the children of partners in other large firms where they cannot be hired because of nepotism rules. Apart from this they may be weaker than even their grades suggest, or if they succeed their parent's firm may be willing to bring them in at the partner level although they would not have hired them initially. Blacks with low grades could, conversely, have aspects to their records that suggest they will be stronger performers than their grades indicate.)

One final comment for the moderators as much as anyone. I note that the prior commentator on this blog and most on Prof. Sander's first blog, use pseudonyms or first names. I don't think this kind of anonymity has any place in an academic blog like this one, except in rare circumstances that should be approved by the moderators on an as needed basis. I don't think people should be able to criticize academic work while holding their own identities secret, and I know that I might be helped in evaluating criticism and comments by knowing who the critic or commentator is.

Rick Lempert

Captain Obvious

As I observed before, the nonwhites with low grades face two extra obstacles: (1) they are likely to be at more elite and challenging big firms than are the whites with low grades; (2) they are likely to be "stereotyped" as second-rate even when they are not.

The reason for much of the skepticism is your lack of substantiation for (1); your presumption that (1) significantly outweighs (2); and your failure to investigate (2) with any rigor or attention even coming close to your unhealthy obsession with (1).

The comments to this entry are closed.


April 2014

Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30      

Site Meter

Creative Commons License

  • Creative Commons License
Blog powered by Typepad