Generic contextual named entity recognition
Improved systems and methods for named entity recognition (NER) are disclosed and can include attaching domain-specific context to extracted data. In a particular example implementation, the techniques can include an artificial intelligence (AI) based entity extraction and labeling process using unstructured data as input. The generated labels can include automatically determined entity types. The techniques can further include a domain-aware entity resolution process. First, applying a reverse question-and-answer (Q&A) technique to the output of the entity extraction and labeling process can generate a set of predicted entity keys (e.g., predicted metadata identifiers, such as likely database column references) for the extracted entities and entity types. Second, entity alignment operations can enable determining domain-specific entity keys for the predicted entity keys. In some implementations, the techniques can be utilized to identify named entities in an electronic conversation, such as a chat session.
1. A computer-implemented method for generic contextual named entity recognition (NER) using entity alignment, the method comprising:
receiving, by a computing engine communicatively coupled to a subscriber computing system, unstructured data;
applying, by the computing engine, a natural language processing technique to the unstructured data to generate a set of labeled entity tokens;
for a particular labeled entity token in the set of labeled entity tokens,
using a reverse question-and-answer model, generating a predicted entity key corresponding to the particular labeled entity token; and
performing entity alignment operations, the entity alignment operations comprising:
accessing a subscriber ontology provided by the subscriber computing system, wherein the subscriber ontology comprises a key-value pair including a subscriber entity key; and
searching the subscriber ontology using the predicted entity key to determine the subscriber entity key, wherein determining the subscriber entity key comprises calculating a similarity value between the predicted entity key and the subscriber entity key; and
using the subscriber entity key, generating and transmitting to the subscriber computing system, an electronic signal comprising two or more of: (i) the particular labeled entity token, (ii) the subscriber entity key, or (iii) the similarity value, wherein the electronic signal comprises computer-executable instructions to cause the subscriber computing system to perform an activity.
2. The computer-implemented method of claim 1 , the method comprising:
generating a set of related items by extracting at least a first portion of the unstructured data and a second portion of the unstructured data, wherein the first portion of the unstructured data is contextually relevant to the second portion of the unstructured data; and
applying, by the computing engine, the natural language processing technique to the set of related items to generate the set of labeled entity tokens; and
determining, by the computing engine, that the first portion of the unstructured data is contextually relevant to the second portion of the unstructured data by using the first portion of the unstructured data to generate a token and using the token to query a data source for a set of candidate items for the second portion of the unstructured data,
wherein the second portion of the unstructured data is identified in response to detecting a user indication, via the subscriber computing system, of an item in the set of candidate items.
3. The computer-implemented method of claim 1 , wherein the similarity value is computed by determining Levenshtein distance, determining Jaro-Winkler distance, or applying a Longest Common Subsequence technique to compare the predicted entity key and the subscriber entity key.
4. The computer-implemented method of claim 1 , the method comprising:
applying, by the computing engine, the natural language processing technique to the unstructured data to generate a set of labeled entity tokens by executing one or more AI models trained to automatically determine a data type corresponding to a portion of the unstructured data,
wherein the particular labeled entity token comprises the portion of the unstructured data and a determined data type.
5. The computer-implemented method of claim 4 , the method comprising:
generating the predicted entity key using the portion of the unstructured data and the determined data type as an input to the reverse question-and-answer model.
6. The computer-implemented method of claim 1 , the method comprising:
in response to determining that the similarity value between the predicted entity key and the subscriber entity key is at or greater than a predetermined threshold, generating and causing a transmission, to the subscriber computing system, of the electronic signal.
7. The computer-implemented method of claim 1 , the method comprising:
searching the subscriber ontology using one or more of a fuzzy matcher, a set of if-then statements, or a neural network trained to classify subscriber entity keys into categories.
8. A computer-implemented method for using generic contextual named entity recognition (NER) to contextualize items in chat sessions, the method comprising:
accessing, by a computing engine communicatively coupled to a subscriber computing system, a chat session at a graphical user interface (GUI) of the subscriber computing system; and
while the chat session is active, performing NER operations comprising:
using at least a portion of a transcript of the chat session, generating a set of labeled entity tokens;
for a particular labeled entity token in the set of labeled entity tokens,
generating a predicted entity key corresponding to the particular labeled entity token; and
performing semantic matching operations on the predicted entity key to determine a subscriber entity key and an associated confidence score;
causing the GUI of the subscriber computing system to perform at least one of: (i) visually emphasizing an item in the transcript that corresponds to the particular labeled entity token, or (ii) displaying the subscriber entity key and the associated confidence score.
9. The computer-implemented method of claim 8 , the method comprising:
generating a set of related items by extracting at least a first portion of the transcript and a second portion of the transcript, wherein the first portion is contextually relevant to the second portion; and
applying, by the computing engine, a natural language processing technique to the set of related items to generate the set of labeled entity tokens.
10. The computer-implemented method of claim 8 , the method comprising:
applying, by the computing engine, a natural language processing technique to the transcript of the chat session to generate the set of labeled entity tokens by executing one or more AI models trained to automatically determine a data type corresponding to a portion of the transcript of the chat session,
wherein the particular labeled entity token comprises the portion of the transcript of the chat session and a determined data type.
11. The computer-implemented method of claim 10 , the method comprising:
generating the predicted entity key using the portion of the transcript and the determined data type as an input to a reverse question-and-answer model.
12. The computer-implemented method of claim 8 , the method comprising:
in response to determining that the associated confidence score is at or greater than a predetermined threshold, performing at least one of: (i) visually emphasizing the item in the transcript that corresponds to the particular labeled entity token, or (ii) displaying the subscriber entity key and the associated confidence score.
13. The computer-implemented method of claim 8 , the method comprising:
searching a subscriber ontology for the subscriber entity key using fuzzy matching or a set of if-then statements.
14. The computer-implemented method of claim 8 , the method comprising:
determining the subscriber entity key by applying a neural network trained to classify subscriber entity keys into categories, wherein a particular category for the subscriber entity key corresponds to the predicted entity key.
15. One or more non-transitory, computer-readable media having instructions stored thereon, the instructions, when executed by at least one processor, configured to cause a computing engine to perform a computer-implemented method for using generic contextual named entity recognition (NER) to contextualize items in chat sessions, the method comprising:
accessing, by the computing engine communicatively coupled to a subscriber computing system, a chat session at a graphical user interface (GUI) of the subscriber computing system; and
while the chat session is active, performing NER operations comprising:
using at least a portion of a transcript of the chat session, generating a set of labeled entity tokens;
for a particular labeled entity token in the set of labeled entity tokens,
generating a predicted entity key corresponding to the particular labeled entity token; and
performing semantic matching operations on the predicted entity key to determine a subscriber entity key and an associated confidence score;
causing the GUI of the subscriber computing system to perform at least one of: (i) visually emphasizing an item in the transcript that corresponds to the particular labeled entity token, or (ii) displaying the subscriber entity key and the associated confidence score.
16. The media of claim 15 , the instructions further comprising:
generating a set of related items by extracting at least a first portion of the transcript and a second portion of the transcript, wherein the first portion is contextually relevant to the second portion; and
applying, by the computing engine, a natural language processing technique to the set of related items to generate the set of labeled entity tokens.
17. The media of claim 15 , the instructions further comprising:
applying, by the computing engine, a natural language processing technique to the transcript of the chat session to generate the set of labeled entity tokens by executing one or more AI models trained to automatically determine a data type corresponding to a portion of the transcript of the chat session,
wherein the particular labeled entity token comprises the portion of the transcript of the chat session and a determined data type.
18. The media of claim 17 , the method comprising:
generating the predicted entity key using the portion of the transcript and the determined data type as an input to a reverse question-and-answer model.
19. The media of claim 15 , the method comprising:
in response to determining that the associated confidence score is at or greater than a predetermined threshold, performing at least one of: (i) visually emphasizing the item in the transcript that corresponds to the particular labeled entity token, or (ii) displaying the subscriber entity key and the associated confidence score.
20. The media of claim 15 , the method comprising:
searching a subscriber ontology for the subscriber entity key using fuzzy matching or a set of if-then statements.