Clause taxonomy system and method for structured document construction and analysis
A system and method for clause and clause cluster classification for structured document analysis, interpretation, annotation, versioning and construction. The system and method can analyse one or more structured documents and parse the documents by clause, and assign each clause a clause category and clause cluster category. The clauses of similar structured documents can then be compared and documents can be flagged for variance in clause text, clause content, and the presence or absence of clauses in expected categories for the document type. By organizing clauses and clause language in a digital platform, the present system is capable of rapid document analysis and presentation of like clauses and related clause and clause cluster information for expediting document drafting, version tracking, and document analysis.
1 . A computer-implemented method for structured document analysis comprising:
segmenting a structured document into a plurality of clauses;
parsing each of the plurality of clauses from the structured document, each clause comprising clause text and a clause taxonomy comprising contextual metadata and extrinsic metadata associated with each clause, the parsing comprising using natural language processing of the clause text and a multidimensional mapping of the clause taxonomy to provide a vector description of each clause defining a relationship between clause taxonomy and clause text;
classifying each of the plurality of clauses in the structured document by clause cluster by classifying the structured document by document category and matching each of the plurality of clauses in the structured document to a standard clause category in the document category in a clause library by comparing the vector description of each of the plurality of clauses to vector descriptions of other clauses and clause clusters in the clause library, the clause library in a clause database comprising a plurality of previously classified clauses from a plurality of classified structured documents;
displaying, on a graphical user interface, a particular clause in the structured document next to at least one clause from the clause library from at least one of the plurality of previously classified structured documents with a similar vector description and in the same clause category as the particular clause along with the contextual metadata and extrinsic metadata relevant to the at least one clause from the clause library;
identifying a weak clause from the plurality of clauses that presents a potential risk in the structured document; and
providing an alert to the weak clause on the graphical user interface with a set of clause rules to avoid the potential risk, the clause rules based on the extrinsic metadata and contextual metadata of the clause and comprising a rationale for certain phrases in the clause category and at least one previously classified clause in the standard clause category from the clause library.
2 . The method of claim 1 , wherein segmenting the structured document into a plurality of clauses comprises identifying one or more of a carriage return, comma, semi colon, period, conjunction, paragraph structure, and other punctuation.
3 . The method of claim 1 , wherein parsing each of the plurality of clauses is done using word to vector analysis to identify vector based clauses and clause clusters in the structured document.
4 . The method of claim 1 , wherein the method identifies the absence of one or more standard clause categories in the structured document.
5 . The method of claim 1 , further comprising tracking versioning of each clause in the structured document to previous versions of the same clause category in the clause library by comparing the clause with previous clause variants having similar extrinsic metadata or intrinsic metadata.
6 . The method of claim 1 , further comprising performing a gap analysis between at least one of the plurality of clauses or clause clusters in the structured document and similar clauses and clause clusters in the clause library of classified clauses and clause clusters and identifying one or more of a missing clause, missing clause cluster, and missing clause variant.
7 . The method of claim 1 , wherein the structured document is a legal document, contract, sales proposal, policy document, agreement, court case, report, scientific paper, or online text.
8 . The method of claim 1 , wherein the weak clause is matched to the at least one standard clause based one or more organization rule, legal decision, and legal jurisprudence.
9 . The method of claim 1 , further comprising editing clause text of one of the plurality of clauses in the structured document and updating the clause taxonomy of the clause based on the editing.
10 . The method of claim 1 , wherein the extrinsic metadata associated with each clause comprises one or more of an occurrence of particular language, misuse of language, clause category occurrence, type of document, importance of document, inconsistencies within document, internally conflicting language, a conflict situation within clauses in the document, a conflict situation between clauses in the document, clause category, legal precedents, external comments on the clause, external history of use of the clause, gap analysis of potentially missing clauses or clause clusters, and clause variants.
11 . The method of claim 1 , wherein the contextual metadata for each clause comprises one or more of firm identification, author identification, editor identification, editing timestamp, drafting timestamp, clause edit history, document location, internal discussion, comments, answers, escalations, rule development, comparing against a rule, internal approval of the clause, internal disapproval of the clause, notes on improvements to be made to that clause, and document occurrence identification.
12 . The method of claim 1 , wherein the potential risk is a loophole that may result in illegality or unenforceability, a procedural risk caused by introducing inadequately researched or improperly or unclearly defined terms or clauses, an external regulatory risk, an internal contractual risk, or a negotiation risk.
13 . The method of claim 1 , wherein the potential risk is classified by a risk level category comprising one or more of financial burden, time pressure, asset risk, and integrity of the contract.
14 . The method of claim 1 , wherein the clause rules are based on one or more of legal precedent, organization rules, and organization precedents based on prior legal analysis and risk tolerance.
15 . The method of claim 1 , wherein the weak clause comprises one or more of use or misuse of language, document inconsistency, internally conflicting language, internally conflicting situation, prior dispute over clause or clause cluster language, disapproval of the clause, notes on improvements to be made to the clause, and document occurrence identification.
16 . A system for constructing a structured document comprising:
a clause library comprising a plurality of clauses classified by clause category and clause clusters classified by cluster category, each clause and clause cluster comprising a clause taxonomy comprising contextual metadata and extrinsic metadata associated with each clause;
a clause parsing engine comprising a natural language processing algorithm for parsing clauses and clause clusters in a structured document to identify the clause category to which the clause belongs and a multidimensional map of the clause taxonomy for each parsed clause in the structured document to provide a vector description of the clause comprising one or more vector expressions defining the relationship between clause taxonomy and clause text, the clause taxonomy comprising contextual metadata, and extrinsic metadata for the parsed clause;
a clause matching engine for matching parsed clauses and clause clusters to similar clauses and clause clusters in the clause library using the vector description of the clauses and clause clusters which incorporates the clause and clause cluster taxonomy, the clause library in a clause database comprising a plurality of previously classified clauses from a plurality of classified structured documents; and
a graphical user interface for displaying the structured document deconstructed into clauses and clause clusters in the structured document next to similar clauses and clause clusters from the clause library from at least one of the plurality of previously classified structured documents to identify at least one weak clause in the structured document that presents a potential risk based on the contextual metadata and extrinsic metadata relevant to the at least one weak clause, and providing an alert to the at lease one weak clause on the graphical user interface with a set of clause rules, the clause rules based on the extrinsic metadata and contextual metadata of the at least one weak clause and comprising a rationale for certain phrases in the clause category and at least one previously classified clause in the standard clause category from the clause library to avoid the potential risk.
17 . The system of claim 16 , wherein each clause and clause cluster has one or more document type that the clause and clause cluster can be applied to.
18 . The system of claim 16 , wherein the system identifies the absence of one or more clause categories in the document and clause variants that presents the potential risk in the document.
19 . The system of claim 16 , further comprising a translation database matching each of the plurality of clauses with clauses having the same intent in another language.
20 . A system for structured document analysis comprising:
at least one processor; and
at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the system to perform operations comprising:
segmenting a structured document into a plurality of clauses;
parsing each of the plurality of clauses from the structured document, each clause comprising clause text and a clause taxonomy comprising contextual metadata and extrinsic metadata associated with each clause, the parsing comprising using natural language processing of the clause text and a multidimensional mapping of the clause taxonomy to provide a vector description of each clause defining a relationship between clause taxonomy and clause text;
classifying each of the plurality of clauses in the structured document by clause cluster by classifying the structured document by document category and matching each of the plurality of clauses in the structured document to a standard clause category in the document category in a clause library and comparing the vector description of each of the plurality of clauses to vector descriptions of other clauses and clause clusters in the clause library, the clause library in a clause database comprising a plurality of previously classified clauses from a plurality of classified structured documents;
displaying, on a graphical user interface, a particular clause in the structured document next to at least one clause from the clause library from at least one of the plurality of previously classified structured documents with a similar vector description and in the same clause category as the particular clause along with the contextual metadata and extrinsic metadata relevant to the at least one clause from the clause library;
identifying a weak clause from the plurality of clauses that presents a potential risk in the structured document; and
providing an alert to the weak clause on the graphical user interface with a set of clause rules, the clause rules based on the extrinsic metadata and contextual metadata of the clause and comprising a rationale for certain phrases in the clause category and at least one previously classified clause in the standard clause category from the clause library to avoid the potential risk.