Granted Patent B2

US 12,406,684 · App. 18/140,981 · Granted Sep 2, 2025

Inventors: Amro Younes (Redwood City, CA); Winfred James (Campbell, CA); Tao Xing (San Jose, CA); Cheng Yuan (San Jose, CA); Yun Fu (Cupertino, CA); Simon Lau (San Jose, CA); Robert Firebaugh (Tiburon, CA); Sam Liang (Palo Alto, CA)

Assignee: Otter.ai, Inc.

G10L21/10G06F40/169G06Q10/1095G10L17/02G10L17/14H04L65/403

Abstract

Claims (79)

1. A computer-implemented method for joining a virtual participant in a conversation, the method comprising:

identifying a first conversation scheduled to be participated by a first group of actual participants by at least obtaining schedule information associated with the first conversation from a synchronized calendar;

automatically joining a first virtual participant into the first conversation at least before a start time indicated by the schedule information by:

automatically opening a meeting application for the first virtual participant;

automatically entering credential information associated with the first conversation and user information associated with the first virtual participant; and

automatically joining the first virtual participant into the first conversation;

obtaining, via the first virtual participant configured to capture audio data, a first set of audio data associated with the first conversation while the first conversation occurs;

transcribing, via the first virtual participant, the first set of audio data into a first set of text data while the first conversation occurs; and

presenting the first set of text data to the first group of actual participants while the first conversation occurs.

2. The computer-implemented method of claim 1 , wherein the automatically opening a meeting application includes:

determining, before the start time, that the first conversation is about to start; and

in response to determining that the first conversation is about to start, automatically opening the meeting application before the start time.

3. The computer-implemented method of claim 1 , wherein:

the automatically joining a first virtual participant into the first conversation at least before a start time indicated by the schedule information further includes automatically joining the first virtual participant in response to identifying the first virtual participant in association with a user in a participant list presented to one or more active participants during the first conversation.

4. The computer-implemented method of claim 1 , further comprising:

automatically joining a second virtual participant into the first conversation by at least:

automatically opening the meeting application for the second virtual participant;

automatically entering the credential information associated with the first conversation and user information associated with the second virtual participant; and

automatically joining the second virtual participant into the first conversation;

obtaining, via the second virtual participant, a second set of audio data associated with the first conversation while the first conversation occurs;

transcribing, via the second virtual participant, the second set of audio data into a second set of text data while the first conversation occurs;

generating a master transcription based at least in part upon the first set of text data and the second set of text data; and

presenting the master transcription to the first group of actual participants while the first conversation occurs.

5. The computer-implemented method of claim 4 , wherein:

the transcribing, via the first virtual participant, the first set of audio data includes:

obtaining a first set of voiceprints associated with a first subgroup of actual participants of the first group of actual participants; and

transcribing the first set of audio data into the first set of text data based at least in part upon the first set of voiceprints; and

the transcribing, via the second virtual participant, the second set of audio data includes:

obtaining a second set of voiceprints associated with a second subgroup of actual participants of the first group of actual participants; and

transcribing the second set of audio data into the second set of text data based at least in part upon the second set of voiceprints.

6. The computer-implemented method of claim 5 , wherein each voiceprint of the first set of voiceprints and the second set of voiceprints includes an acoustic model and a language model associated with an actual participant of the first group of actual participants.

7. The computer-implemented method of claim 1 , further comprising:

obtaining, via the first virtual participant, a set of visual data associated with the first conversation while the first conversation occurs;

transforming, via the first virtual participant, the set of visual data into a set of annotated visual data while the first conversation occurs; and

presenting the set of annotated visual data to the first group of actual participants while the first conversation occurs.

8. The computer-implemented method of claim 7 , wherein the obtaining, via the first virtual participant, a set of visual data includes:

obtaining an audio capture command or a textual capture command from an actual participant of the first group of actual participants; and

obtaining first visual data upon obtaining the audio capture command or the textual capture command.

9. The computer-implemented method of claim 1 , further comprising:

identifying a second conversation scheduled to be participated by a second group of actual participants;

joining the first virtual participant into the second conversation;

obtaining, via the first virtual participant, a third set of audio data associated with the second conversation while the second conversation occurs;

transcribing, via the first virtual participant, the third set of audio data into a third set of text data while the second conversation occurs; and

presenting the third set of text data to the second group of actual participants while the second conversation occurs.

10. The computer-implemented method of claim 9 , wherein at least a part of the second conversation occurs simultaneously as the first conversation occurs.

11. The computer-implemented method of claim 1 , wherein:

the presenting the first set of text data includes presenting the first set of text data to be searchable, navigable, annotatable, or modifiable, while the first conversation occurs.

12. The computer-implemented method of claim 11 , further comprising:

obtaining, from one or more actual participants of the first group of actual participants, one or more modifications and/or one or more annotations;

updating the first set of text data according to the one or more modifications and/or the one or more annotations; and

presenting the updated first set of text data to the first group of actual participants.

13. The computer-implemented method of claim 1 , further comprising:

presenting the first set of text data to a group of non-participants;

wherein the first group of actual participants is a source of the first set of audio data and the group of non-participants is not.

14. The computer-implemented method of claim 1 , further comprising:

generating a complete transcription upon conclusion of the first conversation; and

transmitting the complete transcription to the first group of actual participants.

15. The computer-implemented method of claim 14 , wherein the generating a complete transcription includes synchronizing the complete transcription with a complete audio data associated with the first conversation.

16. The computer-implemented method of claim 14 , wherein the generating a complete transcription includes automatically segmenting the complete transcription into a plurality of segments when a speaker change occurs or a natural pause occurs.

17. A computing system for joining a virtual participant in a conversation, the computing system comprising:

one or more processors; and

a memory storing instructions that, upon execution by the one or more processors, cause the computing system to perform one or more processes including:

identifying a conversation scheduled to be participated by a group of actual participants by at least obtaining schedule information associated with the conversation from a synchronized calendar;

automatically joining a virtual participant into the conversation at least before a start time indicated by the schedule information by:

automatically opening a meeting application for the virtual participant;

automatically entering credential information associated with the conversation and user information associated with the virtual participant; and

automatically joining the virtual participant into the conversation;

obtaining, via the virtual participant configured to capture audio data, a set of audio data associated with the conversation while the conversation occurs;

transcribing, via the virtual participant, the set of audio data into a set of text data while the conversation occurs; and

presenting the set of text data to the group of actual participants while the conversation occurs.

18. A non-transitory computer-readable medium storing instructions for joining a virtual participant in a conversation, the instructions upon execution by one or more processors of a computing system, cause the computing system to perform one or more processes including:

identifying a conversation scheduled to be participated by a group of actual participants by at least obtaining schedule information associated with the conversation from a synchronized calendar;

automatically joining a virtual participant into the conversation at least before a start time indicated by the schedule information by:

automatically opening a meeting application for the virtual participant;

automatically entering credential information associated with the conversation and user information associated with the virtual participant; and

automatically joining the virtual participant into the conversation;

obtaining, via the virtual participant configured to capture audio data, a set of audio data associated with the conversation while the conversation occurs;

transcribing, via the virtual participant, the set of audio data into a set of text data while the conversation occurs; and

presenting the set of text data to the group of actual participants while the conversation occurs.

Assignments (1)

Continuity (3)

References Cited (190)