Techniques for training and deploying a named entity recognition model

1 . A computer-implemented method comprising:

accessing training data comprising examples, wherein at least one of the examples comprise a particular token that is a hierarchical entity comprised of subsets of information and the particular token is labeled with multiple labels representing multiple named entity classes associated with the subsets of information;

concatenating the multiple labels into a composite label representing the multiple named entity classes;

for each of the at least one of the examples comprising the particular token labeled with the multiple labels, associating the composite label with the particular token to obtain a modified set of training data;

training a machine learning model on the modified set of training data, wherein the training comprises:

iteratively inputting the examples from the modified set of training data into the machine learning model;

predicting a final class for a token recognized as a named entity in each of the examples, wherein, for the at least one of the examples, the recognized named entity is the hierarchical entity and the predicted final class is representative of the multiple named entity classes, and wherein predicting the class comprises:

generating a sequence embedding for each of the examples;

inputting the sequence embeddings into a first layer of the machine learning model;

predicting, by the first layer, the class for the token recognized as the named entity in each of the examples, wherein the class is not representative of the composite label;

outputting, by the first layer, logits representative of the predicted class for the token recognized as the named entity in each of the examples;

combining the logits with the sequence embeddings, respectively, to obtain new sequence outputs;

inputting the new sequence outputs into a second layer of the machine learning model;

predicting, by the second layer, the class for the particular token recognized as the hierarchical named entity in the at least one of the examples, wherein the class is representative of the composite label;

outputting, by the second layer, logits representative of the predicted class for the particular token recognized as the hierarchical named entity in the at least one of the examples; and

generating the final class for the token in each of the examples based on the logits output from the first layer and the logits output from the second layer;

comparing the final class predicted for the token in each of the examples to a label associated with the token, wherein, for the at least one of the examples, the particular token is associated with the composite label; and

modifying model parameters of the machine learning model based on the comparing to minimize a difference between the final class predicted for each token and the label associated with said each token to generate a trained machine learning model; and

providing the trained machine learning model.

2 . The computer-implemented method of claim 1 , wherein the logits are combined with the sequence embeddings by applying a linear transformation on the logits and concatenating the result of the linear transformation with the sequence embeddings, respectively.

3 . The computer-implemented method of claim 1 , wherein the predicting the final class further comprises:

inputting the new sequence outputs into a third layer of the machine learning model;

predicting, by the third layer, the class for the particular token as part of an interval or duration in the at least one of the examples, wherein the class is representative of the composite label;

outputting, by the third layer, logits representative of the predicted class for the particular token as part of the interval or duration in the at least one of the examples; and

generating the final class for the token in each of the examples based on the logits output from the first layer, the logits output from the second layer, and the logits output from the third layer.

4 . The computer-implemented method of claim 1 , wherein the composite label is an ordered sequence of the multiple labels for the multiple named entity classes.

5 . The computer-implemented method of claim 1 , further comprising:

receiving an utterance from a user;

inputting the utterance into the trained machine learning model;

predicting, by the trained machine learning model, a class for a token in the utterance, wherein the token is recognized as a hierarchical entity and the predicted class is representative of multiple named entity classes comprising at least a parent class and a child class associated with a composite label;

extracting an entity hierarchy for the predicted class based on a hierarchical template associated with the composite label, wherein the entity hierarchy comprises at least the parent class and the child class in a hierarchical structure; and

generating a response to the utterance based on the entity hierarchy.

6 . The computer-implemented method of claim 5 , wherein the extracting the entity hierarchy comprises:

accessing a matrix or database that maps hierarchical templates with composite labels, wherein the hierarchical templates are defined in a file format used to store and transmit data objects comprised of attribute-value pairs and arrays;

identifying, using the matrix or database, the hierarchical template associated with the composite label; and

disambiguating the predicted class into the entity hierarchy based on an attribute-value pair and array in the hierarchical template.

7 . The computer-implemented method of claim 5 , wherein the entity hierarchy lists out the parent and child classes in the hierarchical structure along with start and end information for a duration or interval child class; and wherein the generating the response comprises looking up information related to the token based on the entity hierarchy, generating dialogue with the user based on the entity hierarchy, or a combination thereof.

8 . A system comprising:

one or more processors; and

one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform operations comprising:

accessing training data comprising examples, wherein at least one of the examples comprise a particular token that is a hierarchical entity comprised of subsets of information and the particular token is labeled with multiple labels representing multiple named entity classes associated with the subsets of information;

concatenating the multiple labels into a composite label representing the multiple named entity classes;

for each of the at least one of the examples comprising the particular token labeled with the multiple labels, associating the composite label with the particular token to obtain a modified set of training data;

training a machine learning model on the modified set of training data, wherein the training comprises:

iteratively inputting the examples from the modified set of training data into the machine learning model;

predicting a final class for a token recognized as a named entity in each of the examples, wherein, for the at least one of the examples, the recognized named entity is the hierarchical entity and the predicted final class is representative of the multiple named entity classes, and wherein predicting the class comprises:

generating a sequence embedding for each of the examples;

inputting the sequence embeddings into a first layer of the machine learning model;

predicting, by the first layer, the class for the token recognized as the named entity in each of the examples, wherein the class is not representative of the composite label;

outputting, by the first layer, logits representative of the predicted class for the token recognized as the named entity in each of the examples;

combining the logits with the sequence embeddings, respectively, to obtain new sequence outputs;

inputting the new sequence outputs into a second layer of the machine learning model;

predicting, by the second layer, the class for the particular token recognized as the hierarchical named entity in the at least one of the examples, wherein the class is representative of the composite label;

outputting, by the second layer, logits representative of the predicted class for the particular token recognized as the hierarchical named entity in the at least one of the examples; and

generating the final class for the token in each of the examples based on the logits output from the first layer and the logits output from the second layer;

comparing the final class predicted for the token in each of the examples to a label associated with the token, wherein, for the at least one of the examples, the particular token is associated with the composite label; and

modifying model parameters of the machine learning model based on the comparing to minimize a difference between the final class predicted for each token and the label associated with said each token to generate a trained machine learning model; and

providing the trained machine learning model.

9 . The system of claim 8 , wherein the logits are combined with the sequence embeddings by applying a linear transformation on the logits and concatenating the result of the linear transformation with the sequence embeddings, respectively.

10 . The system of claim 8 , wherein the predicting the final class further comprises:

inputting the new sequence outputs into a third layer of the machine learning model;

predicting, by the third layer, the class for the particular token as part of an interval or duration in the at least one of the examples, wherein the class is representative of the composite label;

outputting, by the third layer, logits representative of the predicted class for the particular token as part of the interval or duration in the at least one of the examples; and

generating the final class for the token in each of the examples based on the logits output from the first layer, the logits output from the second layer, and the logits output from the third layer.

11 . The system of claim 8 , wherein the composite label is an ordered sequence of the multiple labels for the multiple named entity classes.

12 . The system of claim 8 , wherein the operations further comprise:

receiving an utterance from a user;

inputting the utterance into the trained machine learning model;

predicting, by the trained machine learning model, a class for a token in the utterance, wherein the token is recognized as a hierarchical entity and the predicted class is representative of multiple named entity classes comprising at least a parent class and a child class associated with a composite label;

extracting an entity hierarchy for the predicted class based on a hierarchical template associated with the composite label, wherein the entity hierarchy comprises at least the parent class and the child class in a hierarchical structure; and

generating a response to the utterance based on the entity hierarchy.

13 . The system of claim 12 , wherein the extracting the entity hierarchy comprises:

accessing a matrix or database that maps hierarchical templates with composite labels, wherein the hierarchical templates are defined in a file format used to store and transmit data objects comprised of attribute-value pairs and arrays;

identifying, using the matrix or database, the hierarchical template associated with the composite label; and

disambiguating the predicted class into the entity hierarchy based on an attribute-value pair and array in the hierarchical template.

14 . The system of claim 12 , wherein the entity hierarchy lists out the parent and child classes in the hierarchical structure along with start and end information for a duration or interval child class; and wherein the generating the response comprises looking up information related to the token based on the entity hierarchy, generating dialogue with the user based on the entity hierarchy, or a combination thereof.

15 . One or more non-transitory computer-readable media for storing instructions which, when executed by one or more processors, cause a system to perform operations comprising:

accessing training data comprising examples, wherein at least one of the examples comprise a particular token that is a hierarchical entity comprised of subsets of information and the particular token is labeled with multiple labels representing multiple named entity classes associated with the subsets of information;

concatenating the multiple labels into a composite label representing the multiple named entity classes;

for each of the at least one of the examples comprising the particular token labeled with the multiple labels, associating the composite label with the particular token to obtain a modified set of training data;

training a machine learning model on the modified set of training data, wherein the training comprises:

iteratively inputting the examples from the modified set of training data into the machine learning model;

predicting a final class for a token recognized as a named entity in each of the examples, wherein, for the at least one of the examples, the recognized named entity is the hierarchical entity and the predicted final class is representative of the multiple named entity classes, and wherein predicting the class comprises:

generating a sequence embedding for each of the examples;

inputting the sequence embeddings into a first layer of the machine learning model;

predicting, by the first layer, the class for the token recognized as the named entity in each of the examples, wherein the class is not representative of the composite label;

outputting, by the first layer, logits representative of the predicted class for the token recognized as the named entity in each of the examples;

combining the logits with the sequence embeddings, respectively, to obtain new sequence outputs;

inputting the new sequence outputs into a second layer of the machine learning model;

predicting, by the second layer, the class for the particular token recognized as the hierarchical named entity in the at least one of the examples, wherein the class is representative of the composite label;

outputting, by the second layer, logits representative of the predicted class for the particular token recognized as the hierarchical named entity in the at least one of the examples; and

generating the final class for the token in each of the examples based on the logits output from the first layer and the logits output from the second layer;

comparing the final class predicted for the token in each of the examples to a label associated with the token, wherein, for the at least one of the examples, the particular token is associated with the composite label; and

modifying model parameters of the machine learning model based on the comparing to minimize a difference between the final class predicted for each token and the label associated with said each token to generate a trained machine learning model; and

providing the trained machine learning model.

16 . The one or more non-transitory computer-readable media of claim 15 , wherein the logits are combined with the sequence embeddings by applying a linear transformation on the logits and concatenating the result of the linear transformation with the sequence embeddings, respectively.

17 . The one or more non-transitory computer-readable media of claim 15 , wherein the predicting the final class further comprises:

inputting the new sequence outputs into a third layer of the machine learning model;

predicting, by the third layer, the class for the particular token as part of an interval or duration in the at least one of the examples, wherein the class is representative of the composite label;

outputting, by the third layer, logits representative of the predicted class for the particular token as part of the interval or duration in the at least one of the examples; and

generating the final class for the token in each of the examples based on the logits output from the first layer, the logits output from the second layer, and the logits output from the third layer.

18 . The one or more non-transitory computer-readable media of claim 15 , wherein the composite label is an ordered sequence of the multiple labels for the multiple named entity classes.

19 . The one or more non-transitory computer-readable media of claim 15 , wherein the operations further comprise:

receiving an utterance from a user;

inputting the utterance into the trained machine learning model;

predicting, by the trained machine learning model, a class for a token in the utterance, wherein the token is recognized as a hierarchical entity and the predicted class is representative of multiple named entity classes comprising at least a parent class and a child class associated with a composite label;

extracting an entity hierarchy for the predicted class based on a hierarchical template associated with the composite label, wherein the entity hierarchy comprises at least the parent class and the child class in a hierarchical structure; and

generating a response to the utterance based on the entity hierarchy.

20 . The one or more non-transitory computer-readable media of claim 19 , wherein the extracting the entity hierarchy comprises:

accessing a matrix or database that maps hierarchical templates with composite labels, wherein the hierarchical templates are defined in a file format used to store and transmit data objects comprised of attribute-value pairs and arrays;

identifying, using the matrix or database, the hierarchical template associated with the composite label; and

disambiguating the predicted class into the entity hierarchy based on an attribute-value pair and array in the hierarchical template.