The LSAT is a controversial test for many reasons. One of the main critiques is that the skills that it measures--three closely related forms of verbal reasoning ability--are only a subset of the abilities relied upon by successful lawyers. In many legal contexts, for example, engagement, empathy, affect recognition, oratory skills, sense of humor, discretion, and integrity (+ many more) are probably more important that raw reasoning ability. Of course, the LSAC reference manuals are crystal clear than the LSAT is only valid for predicting first-year law school grades. Yet, for a variety of reasons related to nature of law school competition, the LSAT has become the primary gatekeeper for the profession.
Fortunately, an ambitious empirical project by Majorie Schultz (Berkeley Law) and Sheldon Zedeck (Berkeley Psychology) seeks to improve the current system of law school entrance exams (see ABA Journal story). The project, which began in 2001 with grant funding from the LSAC, explores the possibility of creating a test that captures a broader array of characteristics relevant to success as a lawyer. Note that the most difficult threshold task of the project is to operationalize into measurable attributes what it means to be a "successful" or "competent" lawyer.
According to the story, Schulz and Zedeck have conducted extensive interviews with hundreds of law alumni, faculty, judges, clients, and hiring partners to generate a list of 26 "effectiveness factors" for practicing lawyers. Remarkably, this research has lead to a pilot test (starting this month) open to 25,000 graduates of Boalt Hall and UC Hastings.
The next threshold will be demonstrating the test's validity--i.e., empirical evidence that the scores really do measure the putative attributes in a reliable way. Anyway, this is a truly visionary project that was difficult to get off the ground. Whether the project successes or fails, the world is better off from their efforts to address this complex, high-stakes issue. Kudos to Schultz and Zedeck!