Page 9 - E-Discovery

NYLJ.COM |
E-Discovery | MONDAY, OCTOBER 5, 2015 | S9

benefits of an effectively implemented predic- must share with the opposing party the docu- pricing. The good news is that pricing can be predictive coding protocol is sound, as the pre-

tive coding protocol.
ments used in the seed, training and valida- renegotiated sometimes.
dictive model (like search terms) is designed
These benefits are becoming increasingly tion sets; (5) whether predictive coding may to eliminate documents from being reviewed.
important as volumes of electronically stored be used in combination with keywords or Will Your Costs Actually Be Lower?
Moreover, a significant risk of non-disclosure
information skyrocket. Meeting court-ordered other search methodologies; and (6) whether is that it may expose the predictive coding
deadlines, and the often short deadlines that court approval is necessary.
With cost savings being one of the principle process to challenge by hindsight, similar to
governmental investigators require, has professed benefits of predictive coding, par- unilaterally using search terms without hav-
become increasingly challenging. Indeed, ties can be surprised when it does not actu- ing sought and obtained advance agreement
we have seen a rise in sanctions for missed Importance of Vendor Selection
ally yield substantial savings. It is therefore on search terms with the requesting party.
deadlines.
A party’s ability to use predictive coding important to analyze the likely overall costs Disclosing and seeking a stipulated proto-

The costs of document review can also be often depends upon the capabilities of its in advance.
col, however, can often lead down a path of
extraordinarily high. Traditional search terms e-discovery services provider (aka vendor). The costs that a party will incur for pre- protracted negotiations and motion practice.
often yield high numbers of irrelevant docu- Parties often do not consider predictive cod- dictive coding consist not just of technology Under some circumstances, the requesting
ments. By substantially reducing the number ing until they are well downstream in a case costs—i.e., the vendor’s charges for using the party may actually seek to obstruct the pro-
of irrelevant documents, fewer documents and find themselves faced with the burden predictive coding tool (discussed above)— ducing party’s use of predictive coding by
require review, which can significantly reduce and expense of a massive document review. but also, importantly, professional fees, pri- demanding unreasonably high recall figures
costs. A study by the RAND Institute in 2012 At that point, if they are already committed marily attorney fees for the document review or demanding to participate in the training
found that savings from predictive coding to a vendor that lacks predictive coding capa-
involved in training the predictive model and
process in a manner that most producing

ranged from 20-30 percent at the low end to parties will find unacceptable.
77 percent at the high end.
One approach that some producing par-
Producing parties can use predictive cod- ties have taken to mitigate or avoid the risks
ing in a variety of ways. For example, it can of non-disclosure, while also avoiding the Benefits of predictive coding can also extend to the party
speed up considerably the review of large downsides of disclosure, is to use predictive requesting documents. A more efficient process—with reviewers
numbers of documents set aside as poten- coding as a means of prioritizing review—for
tially privileged.
example, reviewing all the documents hitting having to review fewer irrelevant documents—can result in faster
search terms, but using predictive coding to
prioritize review of documents that are most productions.
What’s in It for the Requesting Party?
likely to be relevant.
Benefits of predictive coding can also bilities or they are locked into a contract in validating the results. Additional attorney fees
extend to the party requesting documents. A Agree to Share Seed and Training Sets?
which predictive coding pricing is prohibitive, may be incurred in negotiating a stipulated
more efficient process—with reviewers having it is often too late.
predictive coding protocol, if you decide to
to review fewer irrelevant documents—can In the early predictive coding cases, in Many vendors offer predictive coding seek one, and in motion practice if the par-
result in faster productions.
order to obtain both agreement with the software that is not their own, but rather is ties are unable to reach agreement and you
Additionally, requesting parties in large requesting party and court approval, produc- licensed from a software vendor. A poten- nevertheless want advance court approval
cases often complain that productions can ing parties were often willing to share seed tial issue with this arrangement is that the of the protocol.
amount to “document dumps,” with large and training documents with the requesting e-discovery vendor passes through to the The attorney fees that will be incurred in

numbers of irrelevant documents pro- party (including irrelevant documents). Since end user the software vendor’s pricing. That training the predictive model, validating the
duced. This can be a product of document then, this issue has become one of the prin- pricing is usually fixed (i.e., there is little or no results, and conducting a final pre-production
requests that are not narrowly tailored. But cipal hurdles for predictive coding, as many flexibility), based on the volume of documents review often depend on the number of docu-
to the extent it results from reviewers erring producing parties have become much more to which predictive coding is applied, and ments that must be reviewed (and on the
on the side of caution when reviewing large reticent to do so.
the rates are often relatively high. Because rates of those conducting the review). Many
volumes of irrelevant documents, it can yield There is some judicial support for not of this relatively high, volume-based pricing, contend that “expert reviewers” should con-
a production that is more narrowly focused disclosing the seed and training sets. Mag- predictive coding can become impractical duct training and validation.
on relevant documents.
istrate Judge Andrew Peck of the Southern in large document volume cases—the very Where the prevalence of relevant docu-

Unfortunately, the reality is that some District of New York recently pointed out in cases where it may be needed most.
ments in the document population is very
requesting parties—particularly in asym- Rio Tinto v. Vale that there are alternatives to Using predictive coding in such situations low, or the targeted recall and confidence
metrical litigation (e.g., an individual or a producing seed sets and training documents may only be practical if one reduces the size level high, it will typically be necessary to
class against a large corporation)—often seek and coding decisions—“such as statistical of the document population to which predic- review significantly larger samples to train
to use burdensome document requests for estimation of recall at the conclusion of the tive coding is applied, e.g., by first using search the predictive model and validate the results,
leverage. Consequently, such litigants may review as well as by whether there are gaps terms before applying predictive coding. In this which likely will drive up costs. Here, the
oppose a producing party’s efforts to make in the production, and quality control review scenario, the overall recall of the predictive predictive coding tool used can make a dif-
document search and review more efficient of samples from the documents categorized coding output will be reduced to the extent ference, as vendors with active learning tools
and less costly.
as non-responsive.”1 Additionally, with CAL that the search terms miss relevant documents.
claim that training and validation is much

It is in precisely these types of cases that tools, there are no discrete training sets to Vendors that have developed their own pre- more efficient in low prevalence situations
producing parties are more likely to want to share and studies have shown that seed sets dictive coding software generally will have a than with random sampling based tools.
use predictive coding. In negotiating predic- have much less impact on the results.
greater ability to be flexible and creative with Predictive coding may not yield significant
tive coding protocols, requesting parties have Some parties also have agreed to a middle pricing. In addition to lower pricing generally, savings where the prevalence of relevant
been known to seek to impose hurdles to deter ground, where the producing party has pro- we are increasingly seeing such vendors offer- documents is very high. Under those circum-
the producing party from using or realizing vided access to samples from the seed and ing either a flat fee for predictive coding or bun- stances, predictive coding is less likely to
the benefits of predictive coding. Examples training sets rather than the entire sets.
dling it with other technology-based charges bring substantial efficiency gains over manual
include demanding unrealistically high recall (e.g., processing, use of a review platform, and review. In other words, there may not be suf-
and confidence levels (which can substantially hosting of data). Such vendors also are more ficient document review savings to offset the
Seek Advance Court Approval?
increase the number of documents that must likely to have skilled personnel experienced cost of using the predictive coding tool.
be reviewed in training) and access to the doc- Finally, whether to seek approval from in successfully developing and implementing
uments (including irrelevant documents) in the court before using predictive coding is a predictive coding protocol.
Disclose to the Other Side?
the seed, training and control sets.
another decision that must be made. While Carefully negotiating predictive coding
advance approval is not strictly required— pricing when first engaging a vendor can be Whether to disclose your intention to use
What Issues Will You Face?
one court recently commented that such a very important. Even with vendors that have predictive coding to the other side, and to
request was “somewhat unusual”2—it can be their own predictive coding software, pricing seek agreement on a stipulated predictive
Common issues when considering predic- helpful to submit disputes about the predic- can be prohibitive. Parties often do not focus coding protocol, is another important deci-

tive coding include (1) whether your e-discov- tive coding to the court for resolution.
on the pricing for predictive coding when sion point.
ery vendor has predictive coding capabilities; they negotiate the vendor contract. When Courts generally encourage parties to dis-
(2) whether predictive coding will actually Conclusion
they are later confronted with the challenges close and seek agreement on a protocol (but
yield cost savings; (3) whether you should of a large document review, they may face acknowledge that such “cooperation” is not
disclose to the opposing party your intention Predictive coding can offer substantial a Hobson’s choice between high traditional strictly required). Requesting parties may
to use predictive coding; (4) whether you
potential benefits, but also » Page S10
document review and high predictive coding
have legitimate interests in ensuring that the