Be a Lawyer, You Love to Argue
(ACC Docket — May)
A quick straw poll at a recent ACC MiniMBA showed that all the former litigators in the room eventually moved out of this area of their profession for one reason: they got tired of always arguing. If all the good litigators leave, who will represent you in court? Is there a way that in-house counsel can protect the company and reduce the amount of arguing the litigators must do?
This month’s ACC Docket features a variety of articles on areas that can impact your litigation efforts, including: ediscovery, internal investigations, personal injury claims, patent infringement, the Consumer Legal Remedies Act and Six Sigma.
In addition, you will find an interview tailored just for you in this month’s “Tips & Insights” section — one of the top-read departments in the magazine according to ACC Docket’s 2008 Readership Survey. Editor Maggy Baccinelli interviews Amar Sarwal from the National Chamber Litigation Center on hot button litigation issues and current cases affecting business. As general litigation counsel, Sarwal provides insights on how to improve your litigation game, stressing the importance of cost benefit analysis prior to pursuing litigation, a pointed brief that tells a story, and careful management of outside counsel.
Maybe not all lawyers like to argue, but I bet you like to win. This issue of ACC Docket helps you get just that much closer to victory.
The article from the May 2010 ACC Docket, Why My Human Document Review Is Better than Your Algorithm, is ill-informed and does corporate legal departments a disservice by suggesting, without supporting data, that human review is better than computer-assisted review. Reviewing electronic records for production purposes is often the single largest expense element in litigation. Decisions on how to best accomplish that ought to be based on empirical data, not by blindly using a process that has not changed since photocopying or microfilming were the most advanced technologies available.
There are several criteria that could be used in evaluating different processes, and in all of them computer-assisted review is the clear winner:
Cost: We do not have to tell ACC members about the pressures to cut legal bills.
Replicability: Studies by the Text Retrieval Conference (sponsored by the National Institute of Standards and Technology) and the eDiscovery Institute (a 501(c)(3) nonprofit research organization) have shown that when two teams of human reviewers examine the same collections of records, the second team seldom agrees with the first team more than 60-80 percent of the time, while computer-assisted review will often surpass that level of agreement.
Time: Every day a case is open there are ongoing costs; the review costs are just the most obvious. Cutting months off review time can appreciably shorten the time cases are open.
Transparency: When human reviewers make relevancy decisions, all that is known is the result, not the reasoning.
Confidentiality: The less efficient the review process is the more bodies have to be thrown at the problem increasing chances for misuse of the information in the records.
The net effect of all these factors is that brute force linear review can cost several times what computer-assisted review would cost while yielding results that are only poorly replicable, take longer to deliver and can not be explained.
We would encourage ACC members to study the results of the TREC Legal Track and the eDiscovery Institute and to participate in them. Furthermore, consider investing just a percentage of your legal spend in studies using your own data. Do not just rely on law firms or vendors who bill by the hour for advice.
We think that a discussion of effective ediscovery is critical. Accordingly, we appreciate the recently submitted comments on our article about increasing the effectiveness of the human element in the computer assisted discovery process. We would like to offer some further thoughts.
It is not clear that the authors of this letter aiming to rebut our view of the importance of the human element in the computer assisted discovery process read the article we actually wrote. We can only assume that they used an algorithm to search the article and formulate a computer-generated response. It also appears that the algorithm was designed to search for key words that would allow the authors to set up a straw man that they could knock down, while at the same time generating self-serving support. Oddly enough, but for the outright misstatement they use to set up their straw man, we are hard pressed to see where we even disagree with them.
Their outright misstatement is that - [t]he article suggest[s] that human review is better than computer-assisted review. But the article says no such thing - explicitly or implicitly. Nowhere in our article do we claim that a review process without computer assistance is better than a review process with computer assistance. That is nonsense and a straw man. As for the empirical data they claim we do not have, we did not feel much need to provide empirical support for contentions we did not make.
They then go on to knock down the straw man by setting forth criteria for evaluating different processes. In this process they suggest that one could conduct an entire document review with the use of empirical data and computers. We know of no one who would conduct discovery without human review of documents.
We do agree that computer assisted review is best; the key word here being assisted. You can do discovery without computers, but you cannot do discovery without humans.
Our erstwhile rebutters repeatedly choose to ignore the heart and soul of our article at every turn: all the computer assisted ediscovery process ultimately does is find documents. That is all it does. An algorithm can find documents that meet the search parameters, and it can even priority-rank documents from hot to not, but it can not tell you how those documents will ultimately impact the case. That key duty falls to the oft undertrained, underpaid and undervalued human document reviewer.
In short, our article believes in the power of computer assistance, it just does not worship it. Otherwise, you will end up producing a set of documents that responds to a straw man; one your opposition will surely knock down.
The title of their article (“Why My Human Document Reviewer is Better than Your Algorithm”) certainly suggests that human review is superior to an automated approach. The article cautions that no search tool is 100 percent effective, seemingly implying that humans are. This ignores data that shows that even the best human approach is rarely more than 80 percent effective if effectiveness is measured by being able to have a second team produce the same records from the same collection using the same criteria – and the agreement rate is often much lower.
Our response did not claim that humans could be taken out of the equation, hence our use of the term “computer assisted.” However, data strongly supports the notion that electronic records can be produced without going “eyes on” each and every record that is produced. By contrast their article suggests using technology to cull records but uses “eyes on” human review for the last phase. That’s much more than a minor difference in approach as measured in dollars, time and consistency.
I am writing in response to the May 2010 ACC Docket article entitled "Law School Didn't Prepare You For This — Tips for the Internal Investigation." Aside from many good points made in the article, the author addresses the issue as to whether corporate counsel should retain a law firm or an investigative firm to assist and/or conduct an internal investigation.
The author cites three criteria — privilege, quality and "optics" - leading to his conclusion that the selection of a law firm is a better choice. I respectfully dissent for the following reasons:
Privilege. The author states that attorney client privilege and work product may be better protected when utilizing a law firm. The distinction between outside counsel and an outside investigative firm is a thin one. The U.S. Supreme Court, in U.S. v. Nobles 422 U.S. 225, 239 (1975) held and other courts have held that the work of an investigator retained by legal counsel is covered by the work product doctrine. The attorney client privilege would be no less applicable where general counsel retained an investigative firm.
The author further stated that "attorneys are more likely to be sensitive to privilege concerns." In fact many investigative firms are led by lawyers and former prosecutors who are acutely attuned to privilege and work product. . Bear in mind too that privilege and work product are not widely recognized outside the United States.
Quality. The author states that "Personnel at investigative firms can vary widely in talent and experience." I respectfully point out that the same is quite true of law firms.
Optics. The author states that "employees may be more comfortable talking to an attorney than a 'private eye.'" I can confidently say from 35 years of conducting investigations, that employees can be quite intimidated when interviewed by a lawyer as opposed to an investigator. Furthermore, the use of the phrase "private eye" conjures an image not representative of the best firms in the investigations profession. One might err similarly by using the cliches commonly applied to lawyers.
The article also presumes that internal investigations are conducted within the four walls of an office. Most investigations call for information from outside sources — former employees, vendors, clients/customers and many other such parties. This calls for knocking on doors and persuading prospective witnesses to talk (legally and ethically). This is generally not a law firm's sweet spot.
In sum, professional investigative firms are frequently the first port of call, particularly for chief legal officers who want to lead the investigation, and just as frequently for outside counsel who recognize that a good investigative firm complements their legal skills.
I greatly appreciate the time you took to write. With respect to your comments, I would suggest the following:
First, on the issue of privilege, as I noted in my article "investigators may work at the direction of in-house counsel, thus subjecting the results to a claim of privilege." However, the law in this area can be complex and varies greatly among jurisdictions. Particularly in non-US jurisdictions (as Mr. Karson notes), protecting the privilege is difficult, as there are jurisdictions that simply don't recognize a privilege for in-house counsel, though some of those jurisdictions recognize an attorney-client privilege for outside counsel. Thus, in the context of privilege, though using an investigative firm may be viable, outside counsel generally represents a safer alternative.
Second, in representing a firm as reputable as Kroll Associates, I would think that Mr. Karson would fully agree with my observation that "investigative firms vary widely in quality." Indeed, as an Assistant US Attorney, I was pleased to recommend Kroll to serve as a court-appointed monitor in the context of a civil RICO action. And while it is true that law firms also vary in quality, hiring lawyers guarantees certain minimum quality standards, including attending law school, passing the bar exam and state licensing.
Third, I fully agree that interviewees may feel more comfortable talking to the right investigator than they would being interviewed by an attorney. Indeed, I have investigators on my staff who would outperform any lawyer in conducting an investigative interview. All things being equal, however, business people may be less intimidated by meeting with a lawyer, an ordinary business function, as compared to meeting an investigator, which could suggest that the individual is under investigation. My use of the term "private eye" — in quotation marks — was only intended to represent a potential perception by the interviewee. It wasn't meant as a pejorative.
As to the remainder of Mr. Karson's points, I agree that there are certain types of investigations for which investigative firms — particularly a firm with the resources of Kroll — are better suited. Such a determination — which may not be easy to make particularly in the early phases of an investigation — has to be made on a case-by-case basis.