Learning to extract entities from conversations with neural networks
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for extracting entities from conversation transcript data. One of the methods includes obtaining a conversation transcript sequence, processing the conversation transcript sequence using a span detection neural network configured to generate a set of text token spans; and for each text token span: processing a span representation using an entity name neural network to generate an entity name probability distribution over a set of entity names, each probability in the entity name probability distribution representing a likelihood that a corresponding entity name is a name of the entity referenced by the text token span; and processing the span representation using an entity status neural network to generate an entity status probability distribution over a set of entity statuses.
1. A method performed by one or more computers, the method comprising:
obtaining a conversation transcript sequence comprising a sequence of text tokens from a conversation between two or more participants;
processing the conversation transcript sequence using a span detection neural network configured to:
process the conversation transcript sequence to generate a respective feature representation for each of the text tokens in the sequence; and
process the respective feature representations to generate a set of text token spans, each text token span comprising one or more consecutive text tokens in the sequence of text tokens that references an entity of a particular type;
for each text token span:
generating a span representation from the respective feature representations for the consecutive text tokens in the text token span;
processing the span representation using an entity name neural network to generate an entity name probability distribution over a set of entity names, each probability in the entity name probability distribution representing a likelihood that a corresponding entity name is a name of the entity referenced by the text token span, wherein the entity name neural network comprises an entity name softmax layer configured to:
generate, from the respective feature representations, a respective logit for each entity name; and
generate, from the respective logits for each of the entity names, the entity name probability distribution; and
processing the span representation using an entity status neural network to generate an entity status probability distribution over a set of entity statuses, each probability in the entity status probability distribution representing a likelihood that a corresponding entity status is a status of the entity referenced by the text token span.
2. The method of claim 1 , wherein the conversation is a clinical conversation, the two or more participants are at least a patient and a clinician and the entity of a particular type is a symptom entity,
wherein processing the span representation using an entity name neural network to generate an entity name probability distribution over a set of entity names, each probability in the entity name probability distribution representing a likelihood that a corresponding entity name is a name of the entity referenced by the text token span comprises processing the span representation using a symptom name neural network to generate a symptom name probability distribution over a set of symptom names, each probability in the symptom name probability distribution representing a likelihood that a corresponding symptom name is a name of the symptom entity referenced by the text token span, and wherein processing the span representation using an entity status neural network to generate an entity status probability distribution over a set of entity statuses, each probability in the entity status probability distribution representing a likelihood that a corresponding entity status is a status of the entity referenced by the text token span comprises processing the span representation using a symptom status neural network to generate a symptom status probability distribution over a set of symptom statuses, each probability in the symptom status probability distribution representing a likelihood that a corresponding symptom status is a status for the patient of the symptom entity referenced by the text token span.
3. The method of claim 1 , further comprising:
for each text token span:
selecting, using the entity name probability distribution, an entity name for the entity referenced by the text token span, and
selecting, using the entity status probability distribution, an entity status for the entity referenced by the text token span.
4. The method of claim 1 , wherein the span detection neural network comprises a conditional random field (CRF) layer configured to generate the set of text token spans from the respective feature representations.
5. The method of claim 1 , wherein the span detection neural network comprises:
an embedding layer configured to generate a respective embedding of each of the text tokens in the sequence; and
one or more recurrent neural network layers configured to generate a respective contextual representation of each of the text tokens in the sequence from the embeddings.
6. The method of claim 5 , wherein at least one of the recurrent neural network layers is a bi-directional recurrent neural network layer.
7. The method of claim 5 , wherein the span detection neural network further comprises:
one or more feedforward layers configured to generate the respective feature representations from the respective contextual representations.
8. The method of claim 1 , wherein the entity status neural network comprises an entity status softmax layer configured to:
generate, from the respective feature representations, a respective logit for each entity status; and
generate, from the respective logits for each of the entity statuses, the entity status probability distribution.
9. The method of claim 1 , wherein the span representation is a mean of the respective feature representations for the consecutive text tokens in the text token span.
10. The method of claim 1 , wherein the span representation is a sum of the respective feature representations for the consecutive text tokens in the text token span.
11. The method of claim 1 , wherein the span representation is a hidden state of an encoder neural network generated by processing an encoder input comprising the respective feature representations for the consecutive text tokens in the text token span through the encoder neural network.
12. The method of claim 1 , wherein the span detection neural network, the entity name neural network, and the entity status neural network have been trained jointly on labeled training data using multitask learning.
13. The method of claim 12 , wherein during training, for each training conversation transcript sequence in the labeled training data, the reference location of each entity reference in the training conversation transcript sequence from the labeled training data is used to generate the span representation with probability p, and the text token span generated by the span detection neural network is used to generate the span representation with probability 1−p.
14. The method of claim 13 , wherein p is decreased as training progresses.
15. One or more non-transitory computer-readable media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
obtaining a conversation transcript sequence comprising a sequence of text tokens from a conversation between two or more participants;
processing the conversation transcript sequence using a span detection neural network configured to:
process the conversation transcript sequence to generate a respective feature representation for each of the text tokens in the sequence; and
process the respective feature representations to generate a set of text token spans, each text token span comprising one or more consecutive text tokens in the sequence of text tokens that references an entity of a particular type;
for each text token span:
generating a span representation from the respective feature representations for the consecutive text tokens in the text token span;
processing the span representation using an entity name neural network to generate an entity name probability distribution over a set of entity names, each probability in the entity name probability distribution representing a likelihood that a corresponding entity name is a name of the entity referenced by the text token span, wherein the entity name neural network comprises an entity name softmax layer configured to:
generate, from the respective feature representations, a respective logit for each entity name; and
generate, from the respective logits for each of the entity names, the entity name probability distribution; and
processing the span representation using an entity status neural network to generate an entity status probability distribution over a set of entity statuses, each probability in the entity status probability distribution representing a likelihood that a corresponding entity status is a status of the entity referenced by the text token span.
16. A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
obtaining a conversation transcript sequence comprising a sequence of text tokens from a conversation between two or more participants;
processing the conversation transcript sequence using a span detection neural network configured to:
process the conversation transcript sequence to generate a respective feature representation for each of the text tokens in the sequence; and
process the respective feature representations to generate a set of text token spans, each text token span comprising one or more consecutive text tokens in the sequence of text tokens that references an entity of a particular type;
for each text token span:
generating a span representation from the respective feature representations for the consecutive text tokens in the text token span;
processing the span representation using an entity name neural network to generate an entity name probability distribution over a set of entity names, each probability in the entity name probability distribution representing a likelihood that a corresponding entity name is a name of the entity referenced by the text token span, wherein the entity name neural network comprises an entity name softmax layer configured to:
generate, from the respective feature representations, a respective logit for each entity name; and
generate, from the respective logits for each of the entity names, the entity name probability distribution; and
processing the span representation using an entity status neural network to generate an entity status probability distribution over a set of entity statuses, each probability in the entity status probability distribution representing a likelihood that a corresponding entity status is a status of the entity referenced by the text token span.
17. The system of claim 16 , wherein the conversation is a clinical conversation, the two or more participants are at least a patient and a clinician and the entity of a particular type is a symptom entity,
wherein processing the span representation using an entity name neural network to generate an entity name probability distribution over a set of entity names, each probability in the entity name probability distribution representing a likelihood that a corresponding entity name is a name of the entity referenced by the text token span comprises processing the span representation using a symptom name neural network to generate a symptom name probability distribution over a set of symptom names, each probability in the symptom name probability distribution representing a likelihood that a corresponding symptom name is a name of the symptom entity referenced by the text token span, and wherein processing the span representation using an entity status neural network to generate an entity status probability distribution over a set of entity statuses, each probability in the entity status probability distribution representing a likelihood that a corresponding entity status is a status of the entity referenced by the text token span comprises processing the span representation using a symptom status neural network to generate a symptom status probability distribution over a set of symptom statuses, each probability in the symptom status probability distribution representing a likelihood that a corresponding symptom status is a status for the patient of the symptom entity referenced by the text token span.
18. The system of claim 16 , the operations further comprising:
for each text token span:
selecting, using the entity name probability distribution, an entity name for the entity referenced by the text token span, and
selecting, using the entity status probability distribution, an entity status for the entity referenced by the text token span.
19. The system of claim 16 , wherein the span detection neural network comprises a conditional random field (CRF) layer configured to generate the set of text token spans from the respective feature representations.
20. The system of claim 16 , wherein the span detection neural network, the entity name neural network, and the entity status neural network have been trained jointly on labeled training data using multitask learning.