Automatically recognizing and surfacing important moments in multi-party conversations
A system and a method are disclosed for identifying a subjectively interesting moment in a transcript. In an embodiment, a device receives a transcription of a conversation, and identifies a participant of the conversation. The device accesses a machine learning model corresponding to the participant, and applies, as input to the machine learning model, the transcription. The device receives as output from the machine learning model a portion of the transcription having relevance to the participant, and generates for display, to the participant, information pertaining to the portion.
1. A non-transitory computer-readable medium comprising instructions encoded thereon to identify a moment in a transcript, the instructions when executed by at least one processor causing the at least one processor to:
receive a transcription of a conversation, the conversation being ongoing between a plurality of participants and the transcription received as the conversation continues;
identify each participant of the plurality of participants;
access a plurality of machine learning models, each machine learning model selected for a corresponding participant based on a respective profile of the corresponding participant;
apply, as input to each machine learning model of the plurality of machine learning models, the transcription on an ongoing basis as the conversation continues;
receive, as output from each respective machine learning model, a respective portion of the transcription having relevance to its respective participant; and
generate for display, to each respective participant, on an ongoing basis as the conversation from which the transcription was received continues, respective information pertaining to the respective portion, each respective information tailored to each respective participant based on the respective portion output by the respective machine learning model.
2. The non-transitory computer-readable medium of claim 1 , wherein the instructions to access the plurality of machine learning models further comprise instructions that when executed causes the at least one processor to, for each respective participant:
determine a group of which the respective participant is a part;
identify a group model trained based on preferences of the group; and
assign the group model as the machine learning model corresponding to the respective participant.
3. The non-transitory computer-readable medium of claim 2 , wherein the group model is used to surface moments to other users who are a part of the group.
4. The non-transitory computer-readable medium of claim 1 , further comprising instructions to train a given machine learning model of the plurality of machine learning models to correspond to a given participant, the instructions when executed causing the at least one processor to:
access a profile of the given participant, the profile indicating terms in historical search queries performed by the given participant, and indicating interaction by the given participant with results of the historical search queries; and
label the terms based on the indicated participant interaction.
5. The non-transitory computer-readable medium of claim 4 , wherein a strength of association between the terms and the label is updated based on:
a frequency with which the given participant uses the terms; and
how recently, relative to a present time, a term was used in the historical search queries by the given participant.
6. The non-transitory computer-readable medium of claim 1 , wherein the instructions to apply, as input to each machine learning model, the transcription further comprise instructions that when executed causes the at least one processor to:
identify one or more word embeddings corresponding to the transcription; and
apply, as additional input to each machine learning model, the one or more word embeddings.
7. The non-transitory computer-readable medium of claim 1 , wherein the instructions to receive, as output from each respective machine learning model, a respective portion of the transcription having relevance to its respective participant further comprise instructions that when executed cause the at least one processor to:
receive a plurality of scores for different portions of the transcription;
compare each score of the plurality of scores to a threshold; and
determine the portion of the transcription having relevance to the respective participant based on its corresponding score exceeding the threshold.
8. The non-transitory computer-readable medium of claim 1 , further comprising instructions to receive the transcription automatically and in real-time during the conversation.
9. A method for identifying a moment in a transcript, the method comprising:
receiving a transcription of a conversation, the conversation being ongoing between a plurality of participants and the transcription received as the conversation continues;
identifying each participant of the plurality of participants;
accessing a plurality of machine learning models, each machine learning model selected for a corresponding participant based on a respective profile of the corresponding participant;
applying, as input to each machine learning model of the plurality of machine learning models, the transcription on an ongoing basis as the conversation continues;
receiving, as output from each respective machine learning model, a respective portion of the transcription having relevance to its respective participant; and
generating for display, to each respective participant, on an ongoing basis as the conversation from which the transcription was received continues, respective information pertaining to the respective portion, each respective information tailored to each respective participant based on the respective portion output by the respective machine learning model.
10. The method of claim 9 , wherein accessing the plurality of machine learning models further comprises, for each respective participant:
determining a group of which the respective participant is a part;
identifying a group model trained based on preferences of the group; and
assigning the group model as the machine learning model corresponding to the respective participant.
11. The method of claim 10 , wherein the group model is used to surface moments to other users who are a part of the group.
12. The method of claim 9 , further comprising training a given machine learning model of the plurality of machine learning models to correspond to a given participant by:
accessing a profile of the given participant, the profile indicating terms in historical search queries performed by the given participant, and indicating interaction by the given participant with results of the historical search queries; and
labeling the terms based on the indicated participant interaction.
13. The method of claim 12 , wherein a strength of association between the terms and the label is updated based on:
a frequency with which the given participant uses the terms; and
how recently, relative to a present time, a term was used in the historical search queries by the given participant.
14. The method of claim 9 , applying, as input to each machine learning model, the transcription further comprises:
identifying one or more word embeddings corresponding to the transcription; and
applying, as additional input to each machine learning model, the one or more word embeddings.
15. The method of claim 9 , wherein receiving, as output from each respective machine learning model, a respective portion of the transcription having relevance to its respective participant further comprises:
receiving a plurality of scores for different portions of the transcription;
comparing each score of the plurality of scores to a threshold; and
determining the portion of the transcription having relevance to the respective participant based on its corresponding score exceeding the threshold.
16. The method of claim 9 , further comprising receiving the transcription automatically and in real-time during the conversation.
17. A system for identifying a moment in a transcript, the system comprising:
memory with instructions encoded thereon; and
one or more processors that, when executing the instructions, are caused to perform operations comprising:
receiving a transcription of a conversation, the conversation being ongoing between a plurality of participants and the transcription received as the conversation continues;
identifying each participant of the plurality of participants;
accessing a plurality of machine learning models, each machine learning model selected for a corresponding participant based on a respective profile of the corresponding participant;
applying, as input to each machine learning model of the plurality of machine learning models, the transcription on an ongoing basis as the conversation continues;
receiving, as output from each respective machine learning model, a respective portion of the transcription having relevance to its respective participant; and
generating for display, to each respective participant, on an ongoing basis as the conversation from which the transcription was received continues, respective information pertaining to the respective portion, each respective information tailored to each respective participant based on the respective portion output by the respective machine learning model.
18. The system of claim 17 , wherein accessing the plurality of machine learning models further comprises, for each respective participant:
determining a group of which the respective participant is a part;
identifying a group model trained based on preferences of the group; and
assigning the group model as the machine learning model corresponding to the respective participant.
19. The system of claim 18 , wherein the group model is used to surface moments to other users who are a part of the group.
20. The system of claim 17 , the operations further comprising training a given machine learning model of the plurality of machine learning models to correspond to a given participant by:
accessing a profile of the given participant, the profile indicating terms in historical search queries performed by the given participant, and indicating interaction by the given participant with results of the historical search queries; and
labeling the terms based on the indicated participant interaction.