IP Library Granted Patent US 11,977,964
Granted Patent B1
US 11,977,964 · App. 18/135,117 · Granted May 7, 2024

Systems and methods for using a machine learning model to document inconsistencies

Inventor: Rees Winstead (Austin, TX)
Assignee: PJM PRO, LLC
G06N20/00G06F16/215
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,977,964
App. No.
18/135,117
Granted
May 7, 2024
Kind
B1
Abstract

Disclosed herein are exemplary implementations of systems and methods for utilizing a machine learning model to identify logical inconsistencies. An aspect of the disclosed embodiments includes a method comprises receiving, via a network, a document related to a user; receiving, from the machine learning model, a predetermined label and a value associated with the predetermined label wherein the value is associated with a location on the document; storing the predetermined label and the value associated with the predetermined label in a structured data object; identifying, based a consistency rule, a logical inconsistency in the structured data object; generating a visualization based on the logical inconsistency and providing, at a display device, the structured data object, wherein the structured data object includes the visualization of the logical inconsistency.

Claims (50)

1. A method for using a machine learning model to identify a logical inconsistency, comprising:

receiving, via a network, a document related to a user;

receiving, from the machine learning model, a predetermined label and a value associated with the predetermined label wherein the value is associated with a location on the document, wherein: the machine learning model is initially trained using training data comprising labeled inputs, labeled outputs, and unlabeled data; the machine learning model groups the data of the unlabeled data into group clusters based on one or more patterns; and the machine learning model is subsequently trained using reinforcement learning to improve prediction accuracy of the machine learning model;

storing the predetermined label and the value associated with the predetermined label in a structured data object;

identifying, based a consistency rule, a logical inconsistency in the structured data object;

generating a visualization based on the logical inconsistency;

providing, at a display device, the structured data object, wherein the structured data object includes the visualization of the logical inconsistency;

storing an identification inconsistency, where the identification inconsistency is identified when a confidence score generated by the machine learning model for a prestored location of a value associated a predetermined label in the structured data object is less than a first predetermined threshold and greater than a second predetermined threshold; and

receiving, from the user, a selection of the location of the value associated with the predetermined label for the machine learning model and storing the value associated with the predetermined label.

2. The method of claim 1 , further comprising:

generating a consistency rule when the user interacts with the visualization and selects a consistency resolution option; and

storing the consistency rule and resolving a future inconsistency based on the consistency rule.

3. The method of claim 1 , wherein the document is one of: a price quote document, an invoice document, and a delivery confirmation document.

4. The method of claim 1 , wherein the consistency rule may include: comparing a quantity ordered with a quantity delivered, comparing a quoted price per unit to an invoiced price per unit, and comparing an invoiced price per unit to an industry average price per unit.

5. The method of claim 1 , further comprising:

generating, based on monitoring of the structured data object content, a report about changes to the structured data object content over a predetermined interval; and

sending, to a remote computing device, the report.

6. The method of claim 1 , wherein storing the value in the structured data object further includes: generating a first unique identification value for the document and a second unique identification value for the value.

7. The method of claim 1 , further comprising:

sending, in response to identifying a logical inconsistency, a message to a remote computing device based in part on the logical inconsistency and the consistency rule associated with the logical inconsistency.

8. A system for using a machine learning model to identify a logical inconsistency, comprising:

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to:

receive, via a network, a document related to a user;

receive, from the machine learning model, a predetermined label and a value associated with the predetermined label wherein the value is associated with a location on the document, wherein: the machine leaning model is initially trained using training data comprising labeled inputs, labeled outputs, and unlabeled data; the machine learning model groups the data of the unlabeled data into group clusters based on one or more patterns; and the machine learning model is subsequently trained using reinforcement learning to improve prediction accuracy of the machine learning model;

store the predetermined label and the value associated with the predetermined label in a structured data object;

identify, based a consistency rule, a logical inconsistency in the structured data object;

generate a visualization based on the logical inconsistency;

provide, at a display device, the structured data object, wherein the structured data object includes the visualization of the logical inconsistency;

store an identification inconsistency, where the identification inconsistency is identified when a confidence score generated by the machine learning model for a prestored location of a value associated a predetermined label in the structured data object is less than a first predetermined threshold and greater than a second predetermined threshold; and

receive, from the user, a selection of the location of the value associated with the predetermined label for the machine learning model and storing the value associated with the predetermined label.

9. The system of claim 8 , wherein the instructions further cause the processor to:

generate a consistency rule when the user interacts with the visualization and selects a consistency resolution option; and

store the consistency rule and resolving a future inconsistency based on the consistency rule.

10. The system of claim 8 , wherein the instructions further cause the processor to:

send, in response to identifying a logical inconsistency, a message to a remote computing device based in part on the logical inconsistency and the consistency rule associated with the logical inconsistency.

11. A tangible, non-transitory computer-readable medium storing instructions that, when executed, cause a processing device to:

receive, via a network, a document related to a user;

receiving, from the machine learning model, a predetermined label and a value associated with the predetermined label wherein the value is associated with a location on the document, wherein: the machine learning model is initially trained using training data comprising labeled inputs, labeled outputs, and unlabeled data; the machine learning model groups the data of the unlabeled data into group clusters based on one or more patterns; and the machine learning model is subsequently trained using reinforcement learning to improve prediction accuracy of the machine learning model;

store the predetermined label and the value associated with the predetermined label in a structured data object;

identify, based a consistency rule, a logical inconsistency in the structured data object;

generate a visualization based on the logical inconsistency;

provide, at a display device, the structured data object, wherein the structured data object includes the visualization of the logical inconsistency;

store an identification inconsistency, where the identification inconsistency is identified when a confidence score generated by the machine learning model for a prestored location of a value associated a predetermined label in the structured data object is less than a first predetermined threshold and greater than a second predetermined threshold; and

receive, from the user, a selection of the location of the value associated with the predetermined label for the machine learning model and storing the value associated with the predetermined label.

12. The computer-readable medium of claim 11 , wherein the instructions further cause the processor to:

generate a consistency rule when the user interacts with the visualization and selects a consistency resolution option; and

store the consistency rule and resolving a future inconsistency based on the consistency rule.

13. The computer-readable medium of claim 11 , wherein the document is one of: a price quote document, an invoice document, and a delivery confirmation document.

14. The computer-readable medium of claim 11 , wherein the consistency rule may include: comparing a quantity ordered with a quantity delivered, comparing a quoted price per unit to an invoiced price per unit, and comparing an invoiced price per unit to an industry average price per unit.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2023
From: WINSTEAD, REES
To: PJM PRO, LLC
Reel/Frame 063344/0714 →
Cited By (4)
US 12,493,904 US 12,614,147 US 12,639,745 US 12,711,535