Granted Patent B2

US 12,400,661 · App. 18/514,666 · Granted Aug 26, 2025

Inventors: Yun Fu (Cupertino, CA); Simon Lau (San Jose, CA); Kaisuke Nakajima (Sunnyvale, CA); Julius Cheng (Cupertino, CA); Sam Song Liang (Palo Alto, CA); James Mason Altreuter (Belmont, CA); Kean Kheong Chin (Santa Clara, CA); Zhenhao Ge (Sunnyvale, CA); Hitesh Anand Gupta (Santa Clara, CA); Xiaoke Huang (Foster City, CA); James Francis McAteer (San Francisco, CA); Brian Francis Williams (San Carlos, CA); Tao Xing (San Jose, CA)

Assignee: Otter.ai, Inc.

G10L15/26G06F16/906G10L15/04G10L17/00

Abstract

Claims (50)

1. A computer-implemented method for processing a plurality of moment-associating elements, the method comprising:

segmenting a plurality of audio elements from the plurality of moment-associating elements into a plurality of audio segments;

transcribing, by an automated speech recognition (ASR) system using one or more acoustic models and one or more language models that are trained on data including one or more human speech audios, a first element of the plurality of audio segments into a first text element of a plurality of text elements, each acoustic model of the one or more acoustic model configured to recognize a specific type of sound;

transcribing, by the ASR system using the one or more language models, a second element of the plurality of audio segments into a second text element of the plurality of text elements;

identifying, by the ASR system using the one or more acoustic models and the one or more language models, one or more incorrectly transcribed words in the first transcribed text element;

updating the first transcribed text element by correcting the one or more incorrectly transcribed words in the first transcribed text element using one or more transcribed words in the second transcribed text element; and

generating one or more pieces of moment-associating information based on at least the plurality of audio segments and the plurality of text elements.

2. The computer-implemented method of claim 1 , further comprising assigning a timestamp associated with each element of the plurality of moment-associating elements.

3. The computer-implemented method of claim 1 , wherein the plurality of audio elements includes one or more voice elements of one or more voice-generating sources.

4. The computer-implemented method of claim 1 , wherein the plurality of moment-associating elements further includes one or more visual elements or one or more environmental elements.

5. The computer-implemented method of claim 4 , wherein the one or more visual elements includes at least one selected from a group consisting of pictures, images, screenshots, video frames, one or more projections, and holograms.

6. The computer-implemented method of claim 4 , wherein the one or more environmental elements includes at least one selected from a group consisting of global positions, location types, and conditions associated with the one or more environmental element.

7. The computer-implemented method of claim 4 , wherein the one or more environmental elements includes at least one selected from a group consisting of a longitude, a latitude, an altitude, a country, a city, a street, a location type, a temperature, a humidity, a movement, a velocity of a movement, a direction of a movement, an ambient noise level, and one or more echo properties.

8. The computer-implemented method of claim 1 , further comprising:

receiving one or more voice elements of one or more voice-generating sources; and

receiving one or more voiceprints corresponding to the one or more voice-generating sources respectively.

9. The computer-implemented method of claim 8 , further comprising:

segmenting the plurality of audio elements into a plurality of audio segments includes segmenting the plurality of audio elements into the plurality of audio segments based on at least the one or more voiceprints;

wherein the transcribing the plurality of audio segments into a plurality of text elements includes transcribing the plurality of audio segments into the plurality of text elements based on at least the one or more voiceprints.

10. The computer-implemented method of claim 8 , wherein the receiving one or more voiceprints corresponding to the one or more voice-generating sources respectively includes receiving one or more acoustic models and/or language models corresponding to the one or more voice-generating sources respectively.

11. The computer-implemented method of claim 1 , further comprising:

establishing one or more anchor points based on at least the plurality of moment-associating elements;

wherein:

the one or more anchor points correspond to one or more timestamps respectively; and

each anchor point of the one or more anchor points is navigable, searchable, or both navigable and searchable.

12. The computer-implemented method of claim 11 , further comprising using the one or more anchor points to navigate the one or more pieces of moment-associating information based on at least the one or more timestamps.

13. The computer-implemented method of claim 11 , wherein the one or more anchor points include at least one selected from a group consisting of a word, a phrase, a photo, and a screenshot.

14. The computer-implemented method of claim 1 , further comprising:

obtaining one or more moment-associating photos, the one or more moment-associating photos being one or more parts of the plurality of moment-associating elements; and

transforming the one or more moment-associating photos into one or more anchor photos, wherein the one or more anchor photos correspond to one or more timestamps respectively.

15. The computer-implemented method of claim 14 , wherein each anchor photo of the one or more anchor photos is navigable, searchable, or both navigable and searchable.

16. The computer-implemented method of claim 15 , further comprising:

using the one or more anchor photos to navigate the one or more pieces of moment-associating information based on at least the one or more timestamps.

17. A system for processing a plurality of moment-associating elements, the system comprising:

a transforming module comprising at least one selected from a group consisting of one or more software modules or one or more processors, the transforming module configured to transform a plurality of moment-associating elements into one or more pieces of moment-associating information by:

segmenting a plurality of audio elements from the plurality of moment-associating elements into a plurality of audio segments;

transcribing, by an automated speech recognition (ASR) system using one or more acoustic models and one or more language models that are trained on data including one or more human speech audios, a first element of the plurality of audio segments into a first text element of a plurality of text elements;

transcribing, by the ASR system using the one or more language models, a second element of the plurality of audio segments into a second text element of the plurality of text elements;

identifying, by the ASR system using the one or more acoustic models and the one or more language models, one or more incorrectly transcribed words in the first transcribed text element;

updating the first transcribed text element by correcting the one or more incorrectly transcribed words in the first transcribed text element using one or more transcribed words in the second transcribed text element; and

generating one or more pieces of moment-associating information based on at least the plurality of audio segments and the plurality of text elements.

18. A computer-implemented method for receiving and processing a plurality of moment-associating elements, the method comprising:

receiving the plurality of moment-associating elements, the plurality of moment-associating elements including a plurality of audio elements;

establishing, by an automated speech recognition (ASR) system using one or more acoustic models and one or more language models that are trained on data including one or more human speech audios, one or more anchor points based on at least the plurality of moment-associating elements, the one or more anchor points corresponding to one or more timestamps respectively, and at least one of the one or more anchor points including an image that is searchable via a user-inputted image, each acoustic model of the one or more acoustic model configured to recognize a specific type of sound;

transforming, by the ASR system using the one or more language models, the plurality of moment-associating elements and the one or more anchor points into one or more pieces of moment-associating information; and

transmitting at least one piece of the one or more pieces of moment-associating information.

19. The method of claim 18 , further comprising:

determining that the user-inputted image matches the image included in the at least one of the one or more anchor points by at least a predetermined threshold; and

navigating to at least one of the one or more pieces of moment-associating information based at least in part on the at least one of the one or more anchor points.

20. The method of claim 19 , further comprising: transcribing the plurality of audio segments into a plurality of text elements, wherein the image that is searchable is presented in-line with the plurality of text elements.

Assignments (2)

Continuity (10)

References Cited (188)