Associate Satisfaction by Metropolitan Area

David Zaring and I are finishing up a book review on two novels that feature large law firm associates (Kermit Roosevelt's In the Shadow of the Law and Nick Laird's Utterly Monkey).  In a nutshell, we compare the fictional accounts of misery in large law firms with associate satisfaction data published by The American Lawyer.

The editors at Michigan Law Review have imposed a strict 10,000 word limit.  As a result, it looks like the table below, which analyzes various dimensions of associate satisfaction by metropolitan area, is headed for the cutting room floor.  It suggests that desirable work conditions in large corporate law firms are partially a function of market size/geography.

Some methodological notes:  In this table, associates (years 3-5) at 160+ law firms rated their firm on a 1 to 5 scale (5 = top/best category).  The geographic category was broken down by the top 10 cities in terms of lawyers in the sample: NYC (1,415 associates); DC, Chicago, LA, San Fran (1,834 associates); Boston, Atlanta, Houston, Dallas, Philadelphia (955 associates).  This breakdown tracks the "Global Cities" typology I have discussed in previous posts (see here and here).   The firm means are calculated at the branch office level; in turn, the metropolitan means are weighted by the number of mid-level respondents at each firm.  A large number of firms are represented in all three geographic categories.   

The most striking feature of this table is that, in ALL categories, working conditions  appear to be better in the smaller corporate law markets.   An interesting subsidiary observation is that the mean scores for "overall rating as a place to work" (in red) are typically higher than the working conditions.  This seems at odds with the intuition that these work conditions would be determinants of the "overall rating" score.  One possible explanation, which David and I explore in the Michigan essay, is that,  because of exposure to more high-end non-commodity legal work, associates at prestigious firms in large markets (a) enjoy much greater outplacement options than their smaller market counterparts, and (b) higher expected lifetime earnings.

I would be interested in hearing other alternative explanations for these results.

NOTE:  If you are an academic researcher interested in working with similar law firms data, see announcement for the Law Firms Working Group.

Comments

[More info please]

Can you provide a sense of the effect sizes for the differences? P values can only tell us so much.

[anon]

Perhaps there's a selection bias in associates working in NYC versus those working in the other cities. How an associate working in NYC rates say, Training and Guidance, is different from an associate in Boston rating Training and Guidance. Thus rendering the observations across various cities not comparable, even in aggregate form...

[Mike Guttentag]

A couple of questions/thoughts. First, I see no reason to expect that the “Overall Workplace Rating” needs to be defined by the seven topics that the survey also included. You mention future compensation, but what about present compensation? Also, it would seem that one’s attitude would be shaped by the expected likelihood that you would make partner, and I would suspect this varies systematically depending on whether you are at a main office or regional offices, which, in turn, may be associated with the size of the legal market.

Another issue that certainly complicates all of this analysis is a self-selection bias. Initially, I thought this would tend to lead your results to underestimate the actual effect size. If only those willing to tolerate an unpleasant environment went to New York, say, then it would have to be really awful to exceed their expectations for unpleasantness. On the other hand, we are talking about individuals who choose to work for law firms in New York. Maybe their self-reporting of their work environment is negatively biased, as these may be the type of hyper-competitive people who are never satisfied with the status quo. Thoughts from someone whose legal work experience consists of some halcyon summers as an over-paid, under-worked summer associate.

[William Henderson]

Bill,

You raise some fair points. A few responses:

First, the ALM rankings of firms is not very instructive, primarily because it is the product of weighted and aggregated inputs, just like US News. It is the inputs that are relevant here, not some formula announced by Am Law. Further, this is a very competitive market for associates, so the work conditions in the broad middle should be (and are) very similar. Ranking them magnifies tiny changes among very similar firms. And unlike US News, law firms cannot control what they report.

Second, the response rates are not that bad: an average of 44 percent in 2004 and 42 percent in 2005.

Third, I am weighting/aggregating specific input scores of branch offices in particular markets. These are, in essence, average of over several hundred 3-5 year associates working in those markets.

Fourth, the metro-level analysis reveals a remarkably consistent pattern from NYC to the large market to the middle markets. It is highly unlikely that response rates are going to produce this pattern.

bh.

[Bill Ford]

Bill,

The annual changes in the American Lawyer rankings makes me wonder how good the data are for comparing individual law firms.

I took a quick look at the 2004 and 2003 midlevel associate rankings, and for the 125 firms ranked in both years, the average change for individual firms is about 29 ranks.

For example, the biggest move from 2003 to 2004 involved Chapman & Cutler. It was ranked 145th out of 159 firms in 2003, but it was ranked 21st out of 149 firms in 2004. Many other firms also moved way up or way down.

It seems unlikely that law firm policies and cultures are really changing enough from year to year to explain these variations. More likely, the high turnover rates at many law firms and the low response rates produce the extreme variations from year to year.

Now, you are aggregating the firms together from various regions, which might overcome the problem. For comparing individual firms, the rankings might mean very little, but across many firms, meaningful patterns for the various regions might emerge.

Thoughts?

-Bill

AmLaw Links: http://www.law.com/jsp/tal/PubArticleTAL.jsp?id=1096473926557
http://www.law.com/special/professionals/amlaw/2003/associates_survey/national_rankings.shtml