Granted Patent B2

US 12,406,672 · App. 17/863,837 · Granted Sep 2, 2025

Inventors: Yun Fu (Cupertino, CA); Tao Xing (San Jose, CA); Kaisuke Nakajima (Sunnyvale, CA); Brian Francis Williams (San Carlos, CA); James Mason Altreuter (Belmont, CA); Xiaoke Huang (Foster City, CA); Simon Lau (San Jose, CA); Sam Song Liang (Palo Alto, CA); Kean Kheong Chin (Santa Clara, CA); Wen Sun (San Francisco, CA); Julius Cheng (Cupertino, CA); Hitesh Anand Gupta (Santa Clara, CA)

Assignee: Otter.ai, Inc.

G10L17/02G10L15/04G10L15/183G10L15/26G10L17/00H04H20/95H04L12/18H04L12/1822H04L12/1831

Abstract

Claims (57)

1. A computer-implemented method for processing and broadcasting one or more moment-associating elements, the method comprising:

connecting with one or more calendar systems containing event information associated with an event;

receiving the event information from the one or more calendar systems, the event information including one or more speaker names associated with one or more speakers, one or more speech titles, one or more starting times, one or more end times, a custom vocabulary, location information, and attendee information associated with one or more attendees;

receiving one or more voiceprints corresponding to one or more voice-generating sources respectively;

granting subscription permission to one or more subscribers;

receiving the one or more moment-associating elements, the one or more moment-associating elements including one or more voice elements of the one or more voice-generating sources;

transforming the one or more moment-associating elements into one or more pieces of moment-associating information based at least in part on the one or more voiceprints and the event information; and

transmitting at least one piece of the one or more pieces of moment-associating information to the one or more subscribers;

wherein the transforming the one or more moment-associating elements into one or more pieces of moment-associating information based at least in part on the one or more voiceprints and the event information includes:

segmenting the one or more moment-associating elements into a plurality of moment-associating segments based at least in part on the one or more voiceprints;

assigning a segment speaker for each segment of the plurality of moment-associating segments based at least in part on the one or more voiceprints;

creating a custom language model based at least in part on the event information;

transcribing the plurality of moment-associating segments into a plurality of transcribed segments based at least in part on the custom language model and the one or more voiceprints; and

generating the one or more pieces of moment-associating information based at least in part on the plurality of transcribed segments and the segment speaker assigned for each segment of the plurality of moment-associating segments;

wherein the creating a custom language model based at least in part on the event information includes:

determining the language model based at least in part on one or more points of interest corresponding to the location information;

building the custom language model based at least in part on social information corresponding to the one or more speakers and the one or more attendees; and

integrating the custom vocabulary into the custom language model.

2. The computer-implemented method of claim 1 wherein the social information includes company information, industry information, and school information.

3. The computer-implemented method of claim 1 wherein the creating a custom language model based at least in part on the event information includes:

receiving the social information via one or more social networks.

4. The computer-implemented method of claim 3 wherein the receiving the social information via one or more social networks includes:

receiving the social information from an event page or a profile page.

5. The computer-implemented method of claim 1 wherein the receiving the one or more moment-associating elements includes assigning a timestamp associated with each element of the one or more moment-associating elements.

6. The computer-implemented method of claim 1 wherein the receiving the one or more moment-associating elements includes at least one selected from a group consisting of receiving one or more ambient sound elements, receiving one or more visual elements, and receiving one or more environmental elements.

7. The computer-implemented method of claim 6 wherein the receiving one or more visual elements includes at least one selected from a group consisting of receiving one or more pictures, receiving one or more images, receiving one or more screenshots, receiving one or more video frames, receiving one or more projections, and receiving one or more holograms.

8. The computer-implemented method of claim 6 wherein the receiving one or more environmental elements includes at least one selected from a group consisting of receiving one or more global positions, receiving one or more location types, and receiving one or more moment conditions.

9. The computer-implemented method of claim 8 wherein each global position of the one or more global positions is represented by at least one selected from a group consisting of a longitude, a latitude, an altitude, a country, a city, and a street.

10. The computer-implemented method of claim 8 wherein each location type of the one or more location types is represented by at least one selected from a group consisting of home, office, school, coffee shop, indoor, and outdoor.

11. The computer-implemented method of claim 8 wherein each moment condition of the one or more moment conditions is represented by at least one selected from a group consisting of a temperature, a humidity, a movement, a velocity, a direction, an ambient noise level, and an echo property.

12. The computer-implemented method of claim 1 wherein the transcribing the plurality of moment-associating segments into a plurality of transcribed segments based at least in part on the custom language model and the one or more voiceprints includes:

transcribing a first segment of the plurality of moment-associating segments into a first transcribed segment of the plurality of transcribed segments;

transcribing a second segment of the plurality of moment-associating segments into a second transcribed segment of the plurality of transcribed segments; and

correcting the first transcribed segment based at least in part on the second transcribed segment.

13. The computer-implemented method of claim 1 wherein the segmenting the one or more moment-associating elements into a plurality of moment-associating segments based at least in part on the one or more voiceprints includes:

determining one or more speaker-change timestamps, each timestamp of the one or more speaker-change timestamps corresponding to a timestamp when a speaker change occurs;

determining one or more sentence-change timestamps, each timestamp of the one or more sentence-change timestamps corresponding to a timestamp when a sentence change occurs; and

determining one or more topic-change timestamps, each timestamp of the one or more topic-change timestamps corresponding to a timestamp when a topic change occurs.

14. The computer-implemented method of claim 13 wherein the segmenting the one or more moment-associating elements into a plurality of moment-associating segments is performed based on at least one selected from a group consisting of the one or more speaker-change timestamps, the one or more sentence-change timestamps, and the one or more topic-change timestamps.

15. A non-transitory computer-readable medium with instructions stored thereon, that when executed by a processor, perform the processes comprising:

connecting with one or more calendar systems containing event information associated with an event;

receiving the event information from the one or more calendar systems, the event information including one or more speaker names associated with one or more speakers, one or more speech titles, one or more starting times, one or more end times, a custom vocabulary, location information, and attendee information associated with one or more attendees;

receiving one or more voiceprints corresponding to one or more voice-generating sources respectively;

granting subscription permission to one or more subscribers;

receiving the one or more moment-associating elements, the one or more moment-associating elements including one or more voice elements of the one or more voice-generating sources;

transforming the one or more moment-associating elements into one or more pieces of moment-associating information based at least in part on the one or more voiceprints and the event information; and

transmitting at least one piece of the one or more pieces of moment-associating information to the one or more subscribers;

wherein the transforming the one or more moment-associating elements into one or more pieces of moment-associating information based at least in part on the one or more voiceprints and the event information includes:

segmenting the one or more moment-associating elements into a plurality of moment-associating segments based at least in part on the one or more voiceprints;

assigning a segment speaker for each segment of the plurality of moment-associating segments based at least in part on the one or more voiceprints;

creating a custom language model based at least in part on the event information;

transcribing the plurality of moment-associating segments into a plurality of transcribed segments based at least in part on the custom language model and the one or more voiceprints; and

generating the one or more pieces of moment-associating information based at least in part on the plurality of transcribed segments and the segment speaker assigned for each segment of the plurality of moment-associating segments;

wherein the creating a custom language model based at least in part on the event information includes:

determining the language model based at least in part on one or more points of interest corresponding to the location information;

building the custom language model based at least in part on social information corresponding to the one or more speakers and the one or more attendees; and

integrating the custom vocabulary into the custom language model.

Assignments (3)

Continuity (3)

References Cited (197)