A Tale of Two Cases: TAR Trouble Arises from ESI Protocol

Valsartan and Livingston stand as a warning of the risks of including provisions in an ESI protocol that serve to relinquish a party’s right to determine an appropriate review methodology.

With a nod to Dickens, it was the best of times for the defendant in a recent case seeking to change its document review process from search terms alone to using technology assisted review (TAR) on top of the search term hits. It was the worst of times for a defendant trying to do the same in another case.

Why was it wisdom in the eyes of one court and foolishness in the eyes of the other? The answer: The absence and presence, respectively, of an ESI protocol provision in which the parties agreed to “cooperate in good faith regarding the disclosure and formulation of appropriate search methodology.”

The primary lesson learned is to be careful what you put in an ESI protocol. A feel-good provision regarding cooperation can operate as a significant relinquishment of rights otherwise available under the rules of civil procedure.

In In re: Valsartan, Losartan, and Irbesartan Prods. Liab. Litig., a federal multidistrict litigation (MDL) case being managed in the District of New Jersey, the court, in a decision dated December 2, 2020, pointed to the ESI protocol in denying the defendant’s request to cease review of search term hits that its TAR algorithm predicted to be non-responsive.

In Livingston v. The City of Chicago, also a federal case, this one in the Northern District of Illinois, the court in a decision dated September 3, 2020, by contrast, allowed the defendant to switch from reviewing search term hits to using TAR on top of the search terms, pointing to the absence of any requirement in a prior order or in the federal rules that the defendant collaborate with plaintiffs in developing a review process.

In both cases, the courts did not have a problem with the defendants layering TAR on top of search terms—a win for the defendants on that oft-disputed issue. But the courts diverged on the appropriateness of the mid-stream pivot to using TAR without involving plaintiffs in the process.

In Valsartan, the parties negotiated search terms pursuant to the requirements of the ESI protocol that they meet and confer as early as possible and cooperate regarding the “disclosure and formulation” of the search methodology.

Approximately a year after the parties commenced negotiating the search terms and seven months after the court’s order approving them, the defendant revealed that it had been using TAR to prioritize its review. The defendant further stated that based on proportionality grounds, it wanted not to review approximately 260,000 remaining documents that the TAR algorithm predicted would be non-responsive.

Prioritizing the review of search term hits most likely to be responsive is a relatively common use of TAR. Its benefits include expediting the production of the most responsive documents and avoiding drawn-out negotiations over a TAR protocol, which often include demands from the requesting party to participate in training the TAR algorithm and to participate in its validation by reviewing a sample of documents predicted to be irrelevant—accommodations that are frequently unappealing to producing parties.

The court in Valsartan observed that there would have been no problem if the defendant had stuck with using TAR solely for prioritization. It viewed the defendant’s pivot to using TAR to exclude a quarter million documents from human review, however, as a violation of the defendant’s commitment and the court’s order in the ESI protocol to cooperate and meet and confer in good faith as early as possible regarding the disclosure and formulation of a search methodology.

The court held that requirement of the protocol trumped both the principle that the producing party has the right to decide the manner in which it reviews documents and proportionality considerations. It found that a party’s unilateral adoption of a TAR process “late in the game” and its presentation to the other side as a “fait accompli” violated the ESI protocol’s requirements.

While the court stated there was no evidence that the defendant delayed disclosing its use of TAR “for strategic reasons or for a nefarious purpose,” such as an end run around negotiating a TAR protocol, the court appears to have at least been suspicious of that. Given the nature of the case and that ESI discovery was likely to be extensive, the court repeatedly emphasized that “the use of TAR was or should have been reasonably contemplated at the outset of the litigation,” such that the defendant should have “collaborated” with the plaintiffs regarding the TAR process at the outset.

Nevertheless, stating that it was “reluctant to impose a harsh penalty,” the court, in the end, allowed the defendant to use TAR but only pursuant to the TAR protocol sought by plaintiffs, which provided for them to review 5,000 documents “of their choosing“ from the “elusion set,” i.e., the documents predicted to be non-responsive—an outcome that many responding parties would find unpalatable. The outcome could have been worse—for example, having to review over 260,000 likely irrelevant documents.

The court in Valsartan was explicit in stating that its ruling was based entirely on the requirements of the ESI protocol and that it was not deciding what would have been appropriate in its absence. The latter scenario was presented in Livingston.

The defendant in Livingston, as in Valsartan, wanted to use TAR on top of search terms in its review process and to stop review at a certain responsiveness threshold. Plaintiffs objected on the grounds that the parties had agreed they would identify responsive documents through keyword searches and the defendant had never mentioned using TAR while litigating the search terms for over a year.

Plaintiffs also argued that if defendants wanted to use TAR, they must do so on the entire population of documents collected instead of the search term hits (over 9 million pages of documents instead of 1.3 million that hit the search terms).

The Livingston court rejected those arguments. It agreed with the defendant that there was nothing in the court’s prior order regarding the review process requiring it to negotiate with plaintiffs regarding its review method. Moreover, it held that plaintiffs’ insistence that the defendant “must collaborate with them to establish a review protocol and validation process has no foothold in the federal rules regarding discovery.”

Citing Sedona Principle 6, the court held that the responding party is “best situated to decide how to search for and produce” documents responsive to plaintiffs’ requests. It found that using TAR on the search term hits was appropriate given the low richness of the 9 million-page document population, only 15% of which hit plaintiffs’ own search terms. The court also found that the defendant was appropriately transparent in disclosing the TAR software it was using and how it would validate the results.

As a result, the court in Livingston held the defendant’s review process using TAR on the search term hits “satisfies the reasonable inquiry standard and is proportional to the needs of this case under the federal rules.”

Valsartan and Livingston thus stand as a warning of the risks of including provisions in an ESI protocol that serve to relinquish a party’s right to determine an appropriate review methodology. In these cases, it meant the difference between having to proceed in a manner that the producing party found objectionable versus in a manner that the producing party viewed as appropriate.

Additionally, Valsartan illustrates that TAR should be seriously considered early in cases that are likely to involve high volumes of ESI. Both cases illustrate the challenges parties are likely to face when they seek to change their review process late in the case.

To borrow again from Dickens, ‘tis a far, far better thing to be able to determine one’s own review process (while being transparent about it) than to cede that right in an ESI protocol. It’s also wise in a case likely to involve large volumes of ESI to plan from the outset to use TAR along with search terms, rather than trying to pivot late in the game.

Gareth Evans, is a partner in the Los Angeles office of Redgrave LLP, a national law firm focused on the complex legal challenges that arise at the intersection of the law and technology.

The opinions expressed are those of the author and do not necessarily reflect the views of the firm or its clients. This article is for general information purposes and is not intended to be and should not be taken as legal advice.

Published in Legaltech News (January 2021) and in The Legal Intelligencer 2021 E-Discovery Special Section (March 2021)