Systems and methods for construction, maintenance, and improvement of knowledge representations
In one aspect, the present disclosure relates to a method which, in one example embodiment, can include reading text data corresponding to messages, creating semantic annotations to the text data to generate one or more annotated messages, and aggregating the annotated messages and storing information associated with the aggregated annotated messages in a message store. The method can further include performing, based on information from the message store and associated with the one or more messages, one or more global analytics functions that include: identifying an annotation error in the semantic annotations created using the trained statistical language model, updating the respective semantic annotation to correct the annotation error, and back-propagating corrected data corresponding to the updated semantic annotation into training data for further language model training.
1. A computer-implemented method, comprising:
reading text data corresponding to one or more messages, wherein the text data comprises natural language content;
creating one or more semantic annotations to the text data to generate one or more annotated messages, wherein creating the one or more semantic annotations comprises generating, at least in part by a trained statistical language model, one or more predictive labels as annotations corresponding to language patterns associated with the text data, and wherein the language patterns comprise at least one of part-of-speech, syntactic role, and sentiment associated with the text data;
aggregating the one or more annotated messages and storing information associated with the aggregated one or more annotated messages in a message store; and
performing, based on information from the message store and associated with the one or more messages, global analytics functions that include:
identifying an annotation error in the created semantic annotations,
updating the respective semantic annotation to correct the annotation error, and
back-propagating the updated semantic annotation into training data for further language model training;
wherein aggregating the one or more annotated messages and storing the information associated with the aggregated one or more annotated messages comprises constructing a global knowledge graph representation corresponding to the aggregated one or more annotated messages, and wherein identifying the annotation error, updating the respective semantic annotation, and back-propagating the updated semantic annotation comprises:
(a) identifying the annotation error in the created one or more semantic annotations,
(b) updating the respective semantic annotation in the knowledge graph representation to correct the annotation error,
(c) back-propagating the updated semantic annotation into the training data for the further language model training, and
(d) performing steps (a)-(c) repeatedly until a predetermined level of accuracy of the annotations has been reached or a predetermined number of iterations have been performed.
2. The method of claim 1 , wherein identifying the annotation error from the knowledge graph representation and updating the respective annotation comprises receiving an annotation correction from at least one user via a user interface.
3. The method of claim 1 , wherein updating the respective annotation comprises overriding values derived from the knowledge graph representation, and wherein the overriding of the values is performed at least in part based on values from predetermined information in structured reference data.
4. The method of claim 1 , further comprising:
recording the update to the respective annotation in a change log prior to back-propagating the corrected data.
5. The method of claim 1 , wherein at least one of updating the respective semantic annotation and back-propagating the corrected data is performed by an autonomous trainer agent.
6. The method of claim 1 , wherein the message store is configured to provide for at least one of data insertion and data querying for at least one of entities, concepts, relationships, and metadata associated with the one or more messages.
7. The method of claim 1 , wherein identifying the annotation error comprises identifying a categorization error from a named entity recognition (NER) model.
8. A system, comprising:
one or more processors;
a memory device coupled to the one or more processors and storing instructions which, when executed by the one or more processors, cause the system to perform functions that include:
reading text data corresponding to one or more messages;
creating one or more semantic annotations to the text data to generate one or more annotated messages, wherein creating the one or more semantic annotations comprises generating, at least in part by a trained statistical language model, one or more predictive labels as annotations corresponding to language patterns associated with the text data;
aggregating the one or more annotated messages and storing information associated with the aggregated one or more annotated messages in a message store; and
performing, based on information from the message store and associated with the one or more messages, global analytics functions that include:
identifying an annotation error in the created semantic annotations,
updating the respective semantic annotation to correct the annotation error,
and
back-propagating the updated semantic annotation into training data for further language model training,
wherein aggregating the one or more annotated messages and storing the information associated with the aggregated one or more annotated messages comprises constructing a global knowledge graph representation corresponding to the aggregated one or more annotated messages, and wherein identifying the annotation error, updating the respective semantic annotation, and back-propagating the updated semantic annotation comprises:
(a) identifying the annotation error from the knowledge graph representation,
(b) updating the respective semantic annotation in the knowledge graph representation to correct the annotation error,
(c) back-propagating the updated semantic annotation into the training data for the further language model training, and
(d) performing steps (a)-(c) repeatedly until a predetermined level of accuracy of the annotations has been reached or a predetermined number of iterations have been performed.
9. The system of claim 8 , wherein identifying the annotation error from the knowledge graph representation and updating the respective annotation comprises receiving an annotation correction from at least one user via a user interface.
10. The system of claim 8 , wherein updating the respective annotation comprises overriding values derived from the knowledge graph representation.
11. The system of claim 10 , wherein the overriding of the values is performed at least in part based on values from predetermined information in structured reference data.
12. The system of claim 8 , wherein the stored instructions, when executed by the one or more processors, further cause the system to record the update to the respective annotation in a change log prior to back-propagating the corrected data.
13. The system of claim 8 , wherein at least one of updating the respective semantic annotation and back-propagating the corrected data is performed by an autonomous trainer agent.
14. The system of claim 8 , wherein the message store is configured to provide for at least one of data insertion and data querying for at least one of entities, concepts, relationships, and metadata associated with the one or more messages.
15. The system of claim 8 , wherein identifying the annotation error comprises identifying a categorization error from a named entity recognition (NER) model.
16. The system of claim 8 , wherein the text data corresponding to one or more messages comprises at least one of natural language content and metadata.
17. The system of claim 8 , wherein the language patterns comprise at least one of part-of-speech, syntactic role, and sentiment associated with the text data.
18. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause a computer to perform functions that include:
reading text data corresponding to one or more messages, wherein the text data comprises at least one of natural language content and metadata;
creating one or more semantic annotations to the text data to generate one or more annotated messages, wherein creating the one or more semantic annotations comprises generating, at least in part by a trained statistical language model, one or more predictive labels as annotations corresponding to language patterns associated with the text data, and wherein the language patterns comprise at least one of part-of-speech, syntactic role, and sentiment associated with the text data;
aggregating the one or more annotated messages and storing information associated with the aggregated one or more annotated messages in a message store; and
performing, based on information from the message store and associated with the one or more messages, global analytics functions that include:
identifying an annotation error in the created semantic annotations,
updating the respective semantic annotation to correct the annotation error, and
back-propagating the updated semantic annotation into training data for further language model training;
wherein aggregating the one or more annotated messages and storing the information associated with the aggregated one or more annotated messages comprises constructing a global knowledge graph representation corresponding to the aggregated one or more annotated messages, and wherein identifying the annotation error, updating the respective semantic annotation to correct the annotation error, and back-propagating the updated semantic annotation comprises:
(a) identifying the annotation error from the knowledge graph representation,
(b) updating the respective semantic annotation in the knowledge graph representation to correct the annotation error,
(c) back-propagating the updated semantic annotation into the training data for the further language model training, and
(d) performing steps (a)-(c) repeatedly until a predetermined level of accuracy of the annotations has been reached or a predetermined number of iterations have been performed.
19. The computer-readable medium of claim 18 , wherein identifying the annotation error from the knowledge graph representation and updating the respective annotation comprises receiving an annotation correction from at least one user via a user interface.
20. The computer-readable medium of claim 18 , wherein updating the respective annotation comprises overriding values derived from the knowledge graph representation, and wherein the overriding of the values is performed at least in part based on values from predetermined information in structured reference data.
21. The computer-readable medium of claim 18 , wherein the stored instructions, when executed by the one or more processors, further cause the computer to record the update to the respective annotation in a change log prior to back-propagating the corrected data.
22. The computer-readable medium of claim 18 , wherein the message store is configured to provide for at least one of data insertion and data querying for at least one of entities, concepts, relationships, and metadata associated with the one or more messages.
23. The computer-readable medium of claim 18 , wherein identifying the annotation error comprises identifying a categorization error from a named entity recognition (NER) model.