Fred Krebs, in his President’s message, in the January/February 2010 issue of the Docket, encouraged measurement of law department performance. The central themes of his comments are:
The more we measure our successes, our challenges and even our failures, the more we will learn how to carry these experiences forward in a meaningful way into a new year.
Allow me to express some reservations, but first I should tell you my bias—I am Mr. Measurement. So what is my problem?
When ADR was touted as the certain means to reducing litigation costs and study after study appeared supporting that fact, I complained we should measure these claims in a “scientifically valid manner.” The key is scientifically valid. There was little in the deluge of materials from such notable organizations as the ABA and others that was scientifically valid. There were explanations as to why ADR just had to be more cost efficient and numerous opinion surveys claiming it clearly saved money.
In the context of the Civil Justice Reform Act, I had the opportunity to persuade the Federal District Court for the Northern District of Ohio to let me study these claims. Unknown to me, the Administrative Offices of Federal Courts had engaged Rand to do a similar study. Using two different methodologies, but both looking at actual costs with controls we reached the same result—ADR had no impact on costs.
Measuring is not a simple task and my experience is that much of the measuring done by legal consulting firms is irrelevant at best and misleading at worse, since it involves questionable comparisons and often is not measuring the right thing.
I am in the process now of conducting an enhanced study of ADR with my co-chair a law professor at Case Law School, with the assistance of a professor in social science methodology. We find ourselves continually challenged to insure we are measuring the right thing, the right way and for the right reason.
So before you charge off, inspired by Fred’s challenge to measure, let me suggest you think through three issues. First, why am I measuring? Second, what am I measuring? And third, how am I measuring it? I will explain the importance of each issue in future blogs.
.jpg)
