Textual encoding and analysis with a large graphical language model
The techniques discussed herein enhance the operation of content generation and analysis systems. Namely, textual content applications such as technical documentation, creative writing, and content moderation. This is accomplished through generating a graphical representation of a body of text (e.g., a document). The graphical representation can comprise a plurality of nodes representing the words of the document and a plurality of lines that join the nodes representing a level of association between individual words. As such, the graphical representation can capture the semantic and syntactical structure of the associated document while omitting the original textual content. The graphical representation can be subsequently evaluated for complexity based on the density of nodes and lines. Accordingly, the disclosed system can assign a score to a document based on the evaluation of the graphical representation. In addition, various documents can be ranked based on such scores.
1 . A method comprising:
retrieving a document including a plurality of words;
analyzing the plurality of words included in the document;
generating a graphical representation of the document based on the plurality of words, wherein:
the graphical representation of the document includes a plurality of nodes and a plurality of lines joining the plurality of nodes;
each node of the plurality of nodes represents a word of the plurality of words;
a line joining two nodes represents a level of association between two words represented by the two nodes; and
the plurality of words is omitted from the graphical representation of the document;
determining a number of the plurality of nodes and a number of the plurality of lines included in the graphical representation of the document;
calculating, by utilizing a model without access to the plurality of words included in the document and based on the number of the plurality of nodes and the number of the plurality of lines included in the graphical representation of the document, a numerical evaluation of the graphical representation of the document that quantifies a characteristic of the document;
ranking the graphical representation of the document against a plurality of other graphical representations of other documents based on the numerical evaluation of the graphical representation of the document and a plurality of other numerical evaluations of the plurality of other graphical representations of the other documents; and
generating an identification of a top number of documents based on a corresponding number of top ranked graphical representations.
2 . The method of claim 1 , wherein the document is generated by a multimodal model.
3 . The method of claim 1 , wherein:
a thickness of the line joining the two nodes indicates the level of association; and
the level of association is determined based on at least one of a syntactic connection or a semantic connection.
4 . The method of claim 1 , further comprising:
generating a simplified graphical representation of the document based on the graphical representation;
comparing the simplified graphical representation against a plurality of other simplified graphical representations of the other documents that are generated based on the plurality of other graphical representations; and
determining a writing style for the document based on the plurality of other simplified graphical representations of the other documents.
5 . The method of claim 1 , wherein:
the plurality of words of the document is analyzed by a first multimodal model;
the graphical representation is generated by the first multimodal model; and
the model that calculates the numerical evaluation is a second multimodal model.
6 . The method of claim 1 , wherein the characteristic of the document is a complexity of the document.
7 . The method of claim 1 , wherein the characteristic of the document is a simplicity of the document.
8 . A system comprising:
one or more processing units;
computer-readable storage media storing instructions that, when executed by the one or more processing units, cause the system to perform operations comprising:
analyzing a plurality of words included in a document;
generating a graphical representation of the document based on the plurality of words, wherein:
the graphical representation of the document includes a plurality of nodes and a plurality of lines joining the plurality of nodes;
each node of the plurality of nodes represents a word of the plurality of words;
a line joining two nodes represents a level of association between two words represented by the two nodes; and
the plurality of words is omitted from the graphical representation of the document;
determining a number of the plurality of nodes and a number of the plurality of lines included in the graphical representation of the document;
calculating, by utilizing a model without access to the plurality of words included in the document and based on the number of the plurality of nodes and the number of the plurality of lines included in the graphical representation of the document, a numerical evaluation of the graphical representation of the document that quantifies a characteristic of the document;
ranking the graphical representation of the document against a plurality of other graphical representations of other documents based on the numerical evaluation of the graphical representation of the document and a plurality of other numerical evaluations of the plurality of other graphical representations of the other documents; and
generating an identification of a top number of documents based on a corresponding number of top ranked graphical representations.
9 . The system of claim 8 , wherein:
a thickness of the line joining the two nodes indicates the level of association; and
the level of association is determined based on at least one of a syntactic connection or a semantic connection.
10 . The system of claim 8 , wherein the operations further comprise:
generating a simplified graphical representation of the document based on the graphical representation;
comparing the simplified graphical representation against a plurality of other simplified graphical representations of the other documents that are generated based on the plurality of other graphical representations; and
determining a writing style for the document based on the plurality of other simplified graphical representations of the other documents.
11 . The system of claim 8 , wherein:
the plurality of words of the document is analyzed by a first multimodal model;
the graphical representation is generated by the first multimodal model; and
the model that calculates the numerical evaluation is a second multimodal model.
12 . The system of claim 8 , wherein the characteristic of the document is a complexity of the document.
13 . The system of claim 8 , wherein the characteristic of the document is a simplicity of the document.
14 . A computer-readable storage medium having encoded thereon instructions that, when executed by one or more processing units, cause a system to perform operations comprising:
analyzing a plurality of words included in a document;
generating a graphical representation of the document based on the plurality of words, wherein:
the graphical representation the document includes a plurality of nodes and a plurality of lines joining the plurality of nodes;
each node of the plurality of nodes represents a word of the plurality of words;
a line joining two nodes represents a level of association between two words represented by the two nodes; and
the plurality of words is omitted from the graphical representation of the document;
determining a number of the plurality of nodes and a number of the plurality of lines included in the graphical representation of the document;
calculating, by utilizing a model without access to the plurality of words included in the document and based on the number of the plurality of nodes and the number of the plurality of lines included in the graphical representation of the document, a numerical evaluation of the graphical representation of the document that quantifies a characteristic of the document;
ranking the graphical representation of the document against a plurality of other graphical representations of other documents based on the numerical evaluation of the graphical representation of the document and a plurality of other numerical evaluations of the plurality of other graphical representations of the other documents; and
generating an identification of a top number of documents based on a corresponding number of top ranked graphical representations.
15 . The computer-readable storage medium of claim 14 , wherein:
a thickness of the line joining the two nodes indicates the level of association; and
the level of association is determined based on at least one of a syntactic connection or a semantic connection.
16 . The computer-readable storage medium of claim 14 , wherein the operations further comprise:
generating a simplified graphical representation of the document based on the graphical representation;
comparing the simplified graphical representation against a plurality of other simplified graphical representations of the other documents that are generated based on the plurality of other graphical representations; and
determining a writing style for the document based on the plurality of other simplified graphical representations of the other documents.
17 . The computer-readable storage medium of claim 14 , wherein:
the plurality of words of the document is analyzed by a first multimodal model;
the graphical representation is generated by the first multimodal model; and
the model that calculates the numerical evaluation is a second multimodal model.
18 . The computer-readable storage medium of claim 14 , wherein the characteristic of the document is a complexity of the document.
19 . The computer-readable storage medium of claim 14 , wherein the characteristic of the document is a simplicity of the document.