IP Library › Granted Patent US 12,299,051
Granted Patent B2
US 12,299,051 · App. 17/151,031 · Granted May 13, 2025

Systems and methods of predictive filtering using document field values

Inventors: Jan Puzicha (Bonn, DE); Joe Federline (Mansfield, MA)
Assignee: Open Text Holdings, Inc.
G06F16/93G06F16/23G06F16/38
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,299,051
App. No.
17/151,031
Filed
Jan 15, 2021
Granted
May 13, 2025
Kind
B2
Art Unit
2169
USPC
707/754
Abstract

Electronic discovery using predictive filtering is disclosed herein. An example method includes providing a filtering interface that includes a field value input, a predicted values selector, and a predictor type selector; receiving at least a pivot selected from the field value input and a predicted value from the predicted values selector; searching a plurality of documents based on the pivot and the predicted value selected for any of predictive phrases or predictive concepts; calculating a predictive value for each of the predictive phrases or predictive concepts; and generating a graphical user interface that includes the predictive phrases or predictive concepts in conjunction with their respective predictive value.

Claims (48)

1. A method, comprising:

receiving a plurality of documents;

receiving a selection of a pivot, the pivot being a field value or a set of field values for the plurality of documents;

generating a prediction of responsive phrases, responsive concepts, or other meta-data, the prediction based at least in part of evaluating the plurality of documents based on the pivot, the pivot serving as an input for the generating of the prediction of the responsive phrases, the responsive concepts, or the other meta-data;

further evaluating the plurality of documents based on a new pivot, the new pivot being one of the predicted responsive phrases or the predicted responsive concepts;

generating a prediction of other responsive phrases or other responsive concepts;

calculating a predictive value for each of the other predicted responsive phrases or the other predicted responsive concepts, the predictive value being indicative of a likelihood that the other predicted responsive phrases or the other predicted responsive concepts are to be associated with documents of the plurality of documents that are tagged with the pivot;

generating a graphical user interface that comprises automatically generated filter criteria based on the predictive value for each of the other predicted responsive phrases, the other predicted responsive concepts, or the other predicted meta-data, the automatically generated filter criteria being based on key phrases identified in a preview frame of the graphical user interface, the key phrases comprising selectable filter parameters;

receiving a selection of at least one of the filter criteria from the graphical user interface;

building and applying a filter based on the selection; and

displaying within the graphical user interface, documents from the plurality of documents that were selected, using the at least one of the filter criteria.

2. The method according to claim 1 , wherein calculating the predictive value for each of the other predicted responsive phrases or the other predicted responsive concepts comprises using any of chi-squared statistic or pointwise mutual information.

3. The method according to claim 1 , wherein the pivot is updated.

4. The method according to claim 1 , wherein the plurality of documents have been tagged with field values.

5. The method according to claim 1 , further comprising building a predictive filter, wherein building the predictive filter comprises generating and displaying a filtering interface that comprises a field value input, a predicted values selector, and a predictor type selector.

6. The method according to claim 5 , wherein the filtering interface further comprises a preview panel that comprises a summary comprising the predictive value for each of the predicted responsive phrases or the predicted responsive concepts identified.

7. The method according to claim 6 , wherein the predicted responsive phrases or the predicted responsive concepts are ranked within the graphical user interface according to their respective predictive value.

8. The method according to claim 7 , wherein each of the predicted responsive phrases or the predicted responsive concepts are associated with a frequency count value that indicates how many times the predicted responsive phrases or the predicted responsive concepts appear in the plurality of documents.

9. The method according to claim 8 , wherein the frequency count value is displayed next to the predictive value for each of the predicted responsive phrases or the predicted responsive concepts.

10. The method of claim 1 , wherein the pivot and the new pivot are picked by a user.

11. The method of claim 1 , further comprising computing a predictive score for each of the predicted responsive phrases or predicted responsive concepts based on a pair wise count matrix.

12. The method of claim 11 , wherein computing the predictive score for each of the predicted responsive phrases or the predicted responsive concepts based on the pair wise count matrix is performed in a single pass over the plurality of documents that are tagged with the pivot.

13. The method according to claim 1 , further comprising computing a similarity score for one or more of the plurality of documents, based on the pivot, against documents that have been previously tagged.

14. A system, comprising:

a processor; and

a memory for storing executable instructions, the processor executing the instructions to:

receive a plurality of documents;

receive a selection of a pivot, the pivot being a field value or a set of field values for the plurality of documents;

generate a prediction of responsive phrases, responsive concepts, or other meta-data, the prediction based at least in part of evaluating the plurality of documents based on the pivot, the pivot serving as an input for the generating of the prediction of the responsive phrases, the responsive concepts, or the other meta-data;

further evaluate the plurality of documents based on a new pivot, the new pivot being one of the predicted responsive phrases or the predicted responsive concepts;

generate a prediction of other responsive phrases or other responsive concepts;

calculate a predictive value for each of the other predicted responsive phrases or the other predicted responsive concepts, the predictive value being indicative of a likelihood that the other predicted responsive phrases or the other predicted responsive concepts are to be associated with documents of the plurality of documents that are tagged with the pivot;

generate a graphical user interface that comprises automatically generated filter criteria based on the predictive value for each of the other predicted responsive phrases, the other predicted responsive concepts, or the other predicted meta-data, the automatically generated filter criteria being based on key phrases identified in a preview frame of the graphical user interface, the key phrases comprising selectable filter parameters;

receive a selection of at least one of the filter criteria from the graphical user interface;

build and apply the filter based on the selection; and

display within the graphical user interface, documents from the plurality of documents that were selected, using the at least one of the filter criteria.

15. The system according to claim 14 , wherein the processor further executes the instructions to receive a selection of at least one of the predicted responsive phrases or the predicted responsive concepts from the graphical user interface.

16. The system according to claim 15 , wherein the processor further executes the instructions to build and apply a predictive filter based on the selection.

17. The system according to claim 16 , wherein the processor further executes the instructions to display, within the graphical user interface, documents from the plurality of documents that were selected using the predictive filter.

18. The system according to claim 17 , wherein the processor further executes the instructions to update the pivot with a selection and the predictive filter is rebuilt, the rebuilt filter identifying additional documents found in an updated search using the rebuilt filter.

19. The system according to claim 18 , wherein the processor further executes the instructions to generate and display a filtering interface that comprises a field value input, a predicted values selector, and a predictor type selector.

20. The system according to claim 19 , wherein the filtering interface further comprises a preview panel that comprises a summary comprising the predictive value for each of the predicted responsive phrases or the predicted responsive concepts identified.

21. The system according to claim 20 , wherein the predicted responsive phrases or the predicted responsive concepts are ranked by the system within the graphical user interface according to their respective predictive value.

22. The system according to claim 21 , wherein each of the predicted responsive phrases or the predicted responsive concepts are associated with a frequency count value that indicates how many times the predicted responsive phrases or the predicted responsive concepts appear in the plurality of documents, the frequency count value is displayed next to the predictive value for each of the predicted responsive phrases or the predicted responsive concepts.

23. The system according to claim 16 , wherein the processor further executes the instructions to utilize the filter to exclude documents from the plurality of documents based on the pivot.

24. The system according to claim 14 , wherein the processor further executes the instructions to compute a predictive score for each of the predicted responsive phrases or the predicted responsive concepts based on a pair wise count matrix.

25. The system according to claim 24 , wherein computing the predictive score for each of the predicted responsive phrases or the predicted responsive concepts based on the pair wise count matrix is performed in a single pass over the plurality of documents that are tagged with the pivot.

26. The system according to claim 14 , wherein the processor further executes the instructions to compute a similarity score for one or more of the plurality of documents, based on the pivot, against documents that have been previously tagged.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2026
From: OPEN TEXT HOLDINGS, INC.
To: OPEN TEXT INC.
Reel/Frame 074362/0745 →
CORRECTIVE ASSIGNMENT TO CORRECT THE EXECUTION DATE FOR RECOMMIND GMBH FROM 06/08/2018 TO 08/06/2018 PREVIOUSLY RECORDED ON REEL 55195 FRAME 706. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 28, 2025
From: RECOMMIND GMBH
To: OPEN TEXT HOLDINGS, INC.
Reel/Frame 072712/0384 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2021
From: FEDERLINE, JOE
To: OPEN TEXT HOLDINGS, INC.
Reel/Frame 055194/0829 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2021
From: PUZICHA, JAN
To: RECOMMIND GMBH
Reel/Frame 055195/0195 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2021
From: RECOMMIND GMBH
To: OPEN TEXT HOLDINGS, INC.
Reel/Frame 055195/0706 →
Continuity (2)
Continuation 16042293 · Jul 23, 2018
Related Publication 20210133255A1 · May 6, 2021
References Cited (122)
US 4839853A · Deerwester et al. · 1989 [cited by applicant]
US 6687696B2 · Hofmann et al. · 2004 [cited by applicant]
US 7051017B2 · Marchisio · 2006 [cited by applicant]
US 7089238B1 · Davis et al. · 2006 [cited by applicant]
US 7107266B1 · Breyman et al. · 2006 [cited by applicant]
US 7328216B2 · Hofmann et al. · 2008 [cited by applicant]
US 7376635B1 · Porcari et al. · 2008 [cited by applicant]
US 7428541B2 · Houle · 2008 [cited by applicant]
US 7454407B2 · Chaudhuri et al. · 2008 [cited by applicant]
US 7519589B2 · Charnock et al. · 2009 [cited by applicant]
US 7558778B2 · Carus et al. · 2009 [cited by applicant]
US 7657522B1 · Puzicha et al. · 2010 [cited by applicant]
US 7933859B1 · Puzicha et al. · 2011 [cited by applicant]
US 7945600B1 · Thomas et al. · 2011 [cited by applicant]
US 8015124B2 · Milo · 2011 [cited by applicant]
US 8196030B1 · Wang et al. · 2012 [cited by applicant]
US 8250008B1 · Cao et al. · 2012 [cited by applicant]
US 8296309B2 · Brassil et al. · 2012 [cited by applicant]
US 8433705B1 · Dredze · 2013 [cited by examiner]
US 8489538B1 · Puzicha et al. · 2013 [cited by applicant]
US 8527523B1 · Ravid · 2013 [cited by applicant]
US 8554716B1 · Puzicha et al. · 2013 [cited by applicant]
US 8577866B1 · Osinga et al. · 2013 [cited by applicant]
US 8620842B1 · Cormack · 2013 [cited by applicant]
US 9058327B1 · Lehrman et al. · 2015 [cited by applicant]
US 9223858B1 · Gummaregula et al. · 2015 [cited by applicant]
US 9269053B2 · Naslund et al. · 2016 [cited by applicant]
US 9558265B1 · Tacchi · 2017 [cited by examiner]
US 9595005B1 · Puzicha et al. · 2017 [cited by applicant]
US 9607272B1 · Yu · 2017 [cited by applicant]
US 9785634B2 · Puzicha · 2017 [cited by applicant]
US 10062039B1 · Lockett · 2018 [cited by applicant]
US 10691760B2 · Pattabiraman · 2020 [cited by examiner]
US 10902066B2 · Puzicha et al. · 2021 [cited by applicant]
US 11023828B2 · Puzicha et al. · 2021 [cited by applicant]
US 11282000B2 · Puzicha et al. · 2022 [cited by applicant]
US 20010037324A1 · Agrawal et al. · 2001 [cited by applicant]
US 20020032564A1 · Ehsani et al. · 2002 [cited by applicant]
US 20020080170A1 · Goldberg et al. · 2002 [cited by applicant]
US 20020164070A1 · Kuhner et al. · 2002 [cited by applicant]
US 20030120653A1 · Brady et al. · 2003 [cited by applicant]
US 20030135818A1 · Goodwin et al. · 2003 [cited by applicant]
US 20040167877A1 · Thompson, III · 2004 [cited by applicant]
US 20040210834A1 · Duncan et al. · 2004 [cited by applicant]
US 20050021397A1 · Cui et al. · 2005 [cited by applicant]
US 20050027664A1 · Johnson et al. · 2005 [cited by applicant]
US 20050262039A1 · Kreulen et al. · 2005 [cited by applicant]
US 20060020571A1 · Patterson · 2006 [cited by examiner]
US 20060161423A1 · Scott · 2006 [cited by examiner]
US 20060242190A1 · Wnek · 2006 [cited by applicant]
US 20060259475A1 · Dehlinger · 2006 [cited by examiner]
US 20060294101A1 · Wnek · 2006 [cited by applicant]
US 20070226211A1 · Heinze et al. · 2007 [cited by applicant]
US 20080086433A1 · Schmidtler et al. · 2008 [cited by applicant]
US 20090012984A1 · Ravid et al. · 2009 [cited by applicant]
US 20090043797A1 · Dorie et al. · 2009 [cited by applicant]
US 20090083200A1 · Pollara et al. · 2009 [cited by applicant]
US 20090106239A1 · Getner et al. · 2009 [cited by applicant]
US 20090119343A1 · Jiao et al. · 2009 [cited by applicant]
US 20090164416A1 · Guha · 2009 [cited by applicant]
US 20090306933A1 · Chan et al. · 2009 [cited by applicant]
US 20100014762A1 · Renders et al. · 2010 [cited by applicant]
US 20100030798A1 · Kumar et al. · 2010 [cited by applicant]
US 20100097634A1 · Meyers et al. · 2010 [cited by applicant]
US 20100118025A1 · Smith et al. · 2010 [cited by applicant]
US 20100250474A1 · Richards et al. · 2010 [cited by applicant]
US 20100250541A1 · Richards et al. · 2010 [cited by applicant]
US 20100293117A1 · Xu · 2010 [cited by applicant]
US 20100312725A1 · Privault et al. · 2010 [cited by applicant]
US 20100325102A1 · Maze · 2010 [cited by applicant]
US 20110023034A1 · Nelson et al. · 2011 [cited by applicant]
US 20110029536A1 · Knight et al. · 2011 [cited by applicant]
US 20110047156A1 · Knight et al. · 2011 [cited by applicant]
US 20110135209A1 · Oba · 2011 [cited by applicant]
US 20120101965A1 · Hennig et al. · 2012 [cited by applicant]
US 20120191708A1 · Barsony et al. · 2012 [cited by applicant]
US 20120278266A1 · Naslund et al. · 2012 [cited by applicant]
US 20120296891A1 · Rangan · 2012 [cited by applicant]
US 20120310930A1 · Kumar et al. · 2012 [cited by applicant]
US 20120310935A1 · Puzicha · 2012 [cited by applicant]
US 20130006996A1 · Kadarkarai · 2013 [cited by applicant]
US 20130124552A1 · Stevenson et al. · 2013 [cited by applicant]
US 20130132394A1 · Puzicha · 2013 [cited by applicant]
US 20140059038A1 · McPherson et al. · 2014 [cited by applicant]
US 20140059069A1 · Taft et al. · 2014 [cited by applicant]
US 20140156567A1 · Scholtes · 2014 [cited by applicant]
US 20140207786A1 · Tal-Rothschild et al. · 2014 [cited by applicant]
US 20140310588A1 · Bhogal et al. · 2014 [cited by applicant]
US 20150347576A1 · Endert · 2015 [cited by examiner]
US 20160019282A1 · Lewis et al. · 2016 [cited by applicant]
US 20160110826A1 · Morimoto et al. · 2016 [cited by applicant]
US 20170132530A1 · Puzicha et al. · 2017 [cited by applicant]
US 20170270115A1 · Cormack et al. · 2017 [cited by applicant]
US 20170322931A1 · Puzicha · 2017 [cited by applicant]
US 20180121831A1 · Puzicha et al. · 2018 [cited by applicant]
US 20180341875A1 · Carr · 2018 [cited by applicant]
US 20190138615A1 · Huh et al. · 2019 [cited by applicant]
US 20190205400A1 · Puzicha · 2019 [cited by applicant]
US 20190325031A1 · Puzicha · 2019 [cited by applicant]
US 20200005218A1 · Cheung · 2020 [cited by examiner]
US 20200026768A1 · Puzicha et al. · 2020 [cited by applicant]
US 20210216915A1 · Puzicha et al. · 2021 [cited by applicant]
US 20210224693A1 · Puzicha et al. · 2021 [cited by applicant]
US 20210224694A1 · Puzicha et al. · 2021 [cited by applicant]
US 20220036244A1 · Puzicha et al. · 2022 [cited by applicant]
US 20220188708A1 · Puzicha et al. · 2022 [cited by applicant]
EP 2718803A1 · 2014 [cited by applicant]
WO WO2012170048A1 · 2012 [cited by applicant]
Joachims, Thorsten, “Transductive Inference for Text Classification Using Support Vector Machines”, Proceedings of the Sixteenth International Conference on Machine Learning, 1999, 10 pages. [cited by applicant]
Webber et al. “Assessor Error in Stratified Evaluation,” Proceedings of the 19th ACM International Conference on Information and Knowledge Management, 2010. p. 539-548. [Accessed Jun. 2, 2011—ACM Digital Library] http:/… [cited by applicant]
Webber et al. “Score Adjustment for Correction of Pooling Bias,” Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, 2009. p. 444-451. [Accessed Jun. 2, 2011—… [cited by applicant]
Buckley et al. “Bias and the Limits of Pooling for Large Collections,” Journal of Information Retrieval, Dec. 2007. vol. 10, No. 6, pp. 1-16 [Accessed Jun. 2, 2011—Google, via ACM Digital Library] http://www.cs.umbc.edu… [cited by applicant]
Carpenter, “E-Discovery: Predictive Tagging To Reduce Cost and Error”, The Metropolitan Corporate Counsel, 2009, p. 40. [cited by applicant]
Zad et al. “Collaborative Movie Annotation”, Handbook of Multimedia for Digital Entertainment and Arts, 2009, pp. 265-288. [cited by applicant]
“Extended European Search Report”, European Patent Application No. 11867283.1, Feb. 24, 2015, 6 pages. [cited by applicant]
“Axcelerate 5 Case Manager Guide”, Recommind, Inc., [online], 2016 [retrieved Aug. 12, 2020], retrieved from the Internet: <URL: http://axcelerate-docs.opentext.com/help/axc-main/5.15/en_us/content/resources/pdf%20guide… [cited by applicant]
“Axcelerate 5 Reviewer User Guide”, Recommind, Inc., [online], 2016 [retrieved Aug. 12, 2020], retrieved from the Internet: <URL: http://axcelerate-docs.opentext.com/help/axc-main/5.15/en_us/content/resources/pdf%20guid… [cited by applicant]
“Discovery-Assistant—Near Duplicates”, ImageMAKER Development Inc. [online], 2010, [retrieved Aug. 12, 2020], retrieved from the Internet: <URL:www.discovery-assistant.com > Download > Near-Duplicates.pdf>, 14 pages. [cited by applicant]
Doherty, Sean, “Recornrnind's Axcelerate: An E-Discovery Speedway”, Legal Technology News, 20 Septernber 2014, 3 pages. [cited by applicant]
Youtube, “Introduction to Axcelerate 5”, OpenText Discovery, [online], uploaded Apr. 17, 2014, [retrieved Jul. 10, 2020], retrieved from the Internet: <URL:www.youtube.com/watch?v=KBzbZL9Uxyw>, 41 pages. [cited by applicant]
“Axcelerate 5.9.0 Release Notes”, Recommind, Inc., [online], Aug. 17, 2016 [retrieved Aug. 12, 2020], retrieved from the Internet: <URL: http://axcelerate-docs.opentext.com/help/axc-user/5.9/en_us/content/resources/pdf%… [cited by applicant]
“Axcelerate 5.7.2 Release Notes”, Recommind, Inc., [online], Mar. 3, 2016 [retrieved Aug. 12, 2020], retrieved from the Internet: <URL: http://axcelerate-docs.opentext.com/help/axc-user/5.9/en_us/content/resources/pdf%2… [cited by applicant]
Cited By (3)
US 12,547,944 US 12,572,857 US 12,664,481