Architectures for modeling comment and edit relations
Generally discussed herein are devices, systems, and methods for determining a relationship between an edit and a comment. A system can include a memory to store parameters defining a machine learning (ML) model, the ML model to determine a relationship between an edit, by an author or reviewer, of content of a document and a comment, by a same or different author or reviewer, regarding the content of the document, and processing circuitry to provide the comment and the edit as input to the ML model, and receive, from the ML model, data indicating a relationship between the comment and the edit, the relationship including whether the edit addresses the comment or a location of the content that is a target of the comment.
1. A system comprising:
a memory to store parameters defining a hierarchical neural network (NN) machine learning (ML) model trained using a supervised learning technique, the ML model to determine a relationship between an edit, by an author or reviewer, of content of a body of a pre-edit version of a document that alters the content of the body of the pre-edit version of the document resulting in a post-edit version of the document and a comment in the post-edit version of the document, the comment separate from the body of the post-edit version of the document, by a same or different author or reviewer, regarding the content of the body of the pre-edit version of the document; and
processing circuitry to:
determine, based on the pre-edit version of the document and the post-edit version of the document, an action encoding indicating whether the content is the same, removed, or added between content of the pre-edit version and post-edit version of the document by associating content only in the pre-edit version with a first label of removed, associating content only in the post-edit version with a second, different label of added, and associating content in both the pre-edit version and the post-edit version with a third, different label of same;
provide the comment, the pre-edit version of the document, the action encoding, and the post-edit version of the document as input to the ML model, the post-edit version of the document including the pre-edit version of the document after the edit by the author or reviewer; and
receive, from the ML model and based on the pre-edit version of the document, the action encoding, and the post-edit version, data indicating whether the edit to the content of the body addresses the comment or a location in the body of the content of the body that is a target of the comment.
2. The system of claim 1 , wherein the data from the ML model indicates at least one of (a) the comment most-related to the edit or (b) a location of the post-edit version of the document that is the target of the edit, given the comment.
3. The system of claim 2 , wherein the ML model is configured to determine a relevance score between the edit and the comment and provide the data based on the relevance score.
4. The system of claim 1 , wherein the ML model includes an input embed layer that projects words in the edit and the comment to one or more respective vector spaces, a context embed layer to model sequential interaction between content based on the projected edit and comment, a comment-edit attention layer to model a relationship between the edit and the comment based on the modeled sequential interaction, and an output layer to determine the relationship between the edit and the comment based on the modeled relationship.
5. The system of claim 4 , wherein the context embed layer determines a similarity matrix based on the edit and the comment, wherein the similarity matrix includes values indicating how similar content of the edit is to content of the comment.
6. The system of claim 5 , wherein the comment-edit attention layer determines a normalized probability distribution of the similarity matrix combined with the action encoding.
7. The system of claim 1 , wherein the processing circuitry is further to provide a signal to an application that generated the document, the signal indicating a modification to the document.
8. A method of determining a relationship between an edit of a body of a pre-edit version of a document that alters content of the body of the pre-edit version of the document resulting in a post-edit version of the document and a comment in the document and separate from the body of the document, the method comprising:
labelling unchanged content between a pre-edit version of the document and a post-edit version of the document with a first label;
labelling content in the pre-edit version of the document that is different from the content in the post-edit version of the document with a second, different label;
labelling the content in the post-edit version of the document that is different from the content in the pre-edit version of the document with a third, different label; and
determining, based on the content in the pre-edit version of the document, the content in the post-edit version of the document, the comment, and the first, second, and third labels, and using a machine learning (ML) model that receives the content in the pre-edit version of the document, the content in the post-edit version of the document, the comment, and the first, second, and third labels, the relationship between the comment and the edit, the ML model trained based on at least one of a comment ranking loss function or an edit anchoring loss function.
9. The method of claim 8 , wherein the ML model is trained based on both the comment ranking loss function and the edit anchoring loss function.
10. The method of claim 8 , wherein the ML model is a hierarchical neural network (NN) trained using a supervised learning technique.
11. The method of claim 10 , wherein the ML model includes an input embed layer that projects the edit and the comment to a vector space, a context embed layer to model sequential interaction between content based on the projected edit and comment, a comment-edit attention layer to model a relationship between the projected and embedded edit and the projected and embedded comment based on the modeled sequential interaction, and an output layer to determine the relationship between the edit and the comment based on the modeled relationship.
12. The method of claim 11 , wherein the context embed layer determines a similarity matrix based on the edit and the comment, wherein the similarity matrix includes values indicating how similar content of the edit is to content of the comment.
13. A non-transitory machine-readable medium including instructions that, when executed by a machine, configure the machine to perform operations comprising:
receiving pre-edit content of a document, post-edit content of the document that includes an edit to a body of the pre-edit content of the document resulting in the post-edit content of the document, and a comment of the document separate from the body of the document;
operating a machine learning (ML) model on the pre-edit content, post-edit content, and the comment to determine a relevance score indicating the relevance of content in the post-edit content that is not in the pre-edit content and the comment, the ML model trained based on at least one of a comment ranking loss function or an edit anchoring loss function; and
providing data indicating a relationship between the content in the post-edit content that is not in the pre-edit content and the comment.
14. The non-transitory machine-readable medium of claim 13 , wherein the operations further comprise:
labelling unchanged content between the pre-edit content of the document and the post-edit content of the document with a first label;
labelling content in the pre-edit content of the document that is different from the content in the post-edit content of the document with a second, different label;
labelling content in the post-edit content of the document that is different from the content in the pre-edit content of the document with a third, different label; and
wherein operating the ML model includes further operating the ML model on the first, second, and third labels to determine the relationship between content in the post-edit content that is not in the pre-edit content and the comment of the document.
15. The non-transitory machine-readable medium of claim 13 , wherein the ML model is trained based on both the comment ranking loss function and the edit anchoring loss function.
16. The non-transitory machine-readable medium of claim 13 , wherein the ML model is a hierarchical neural network (NN) trained using a supervised learning technique.