IP Library Granted Patent US 12,080,299
Granted Patent B2
US 12,080,299 · App. 17/863,881 · Granted Sep 3, 2024

Systems and methods for team cooperation with real-time recording and transcription of conversations and/or speeches

Inventors: Simon Lau (San Jose, CA); Yun Fu (Cupertino, CA); James Mason Altreuter (Belmont, CA); Brian Francis Williams (San Carlos, CA); Xiaoke Huang (Foster City, CA); Tao Xing (San Jose, CA); Wen Sun (San Francisco, CA); Tao Lu (Hayward, CA); Kaisuke Nakajima (Sunnyvale, CA); Kean Kheong Chin (Santa Clara, CA); Hitesh Anand Gupta (Santa Clara, CA); Julius Cheng (Cupertino, CA); Jing Pan (Mountain View, CA); Sam Song Liang (Palo Alto, CA)
Assignee: Otter.ai, Inc.
G10L17/02G10L15/04G10L15/183G10L15/26G10L17/00H04H20/95H04L12/18H04L12/1822H04L12/1831
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,080,299
App. No.
17/863,881
Granted
Sep 3, 2024
Kind
B2
Abstract

Methods and systems for team cooperation with real-time recording of one or more moment-associating elements. For example, a method includes: delivering, in response to an instruction, an invitation to each member of one or more members associated with a workspace; granting, in response to acceptance of the invitation by one or more subscribers of the one or more members, subscription permission to the one or more subscribers; receiving the one or more moment-associating elements; transforming the one or more moment-associating elements into one or more pieces of moment-associating information; and transmitting at least one piece of the one or more pieces of moment-associating information to the one or more subscribers.

Claims (85)

1. A computer-implemented method for team cooperation with real-time recording of one or more moment-associating elements, the method comprising:

delivering, in response to an instruction, an invitation to a plurality of members associated with a workspace and one or more external users;

granting subscription permission to a plurality of subscribers who have accepted the invitation, the plurality of subscribers including at least one member of the plurality of members and at least one external user of the one or more external users;

receiving the one or more moment-associating elements;

transforming the one or more moment-associating elements into one or more pieces of moment-associating information by at least:

segmenting the one or more moment-associating elements into a plurality of moment-associating segments,

assigning a segment speaker for each segment of the plurality of moment-associating segments,

transcribing the plurality of moment-associating segments into a plurality of transcribed segments, and

generating the one or more pieces of moment-associating information based at least in part on the plurality of transcribed segments and the segment speaker assigned for each segment of the plurality of moment-associating segments;

receiving, from at least two of the plurality of members, one or more edits to the one or more pieces of moment-associating information;

updating the one or more pieces of moment-associating information based at least in part on the one or more edits; and

transmitting at least one piece of the one or more pieces of updated moment-associating information to the plurality of subscribers, wherein at least one of the plurality of subscribers is not any member of the plurality of members.

2. The computer-implemented method of claim 1 , further comprising receiving event information associated with an event, the event information includes at least one of:

one or more speaker names;

one or more speech titles;

one or more starting times;

one or more end times;

a custom vocabulary;

location information; and

attendee information;

wherein the transforming the one or more moment-associating elements into one or more pieces of moment-associating information includes transforming the one or more moment-associating elements into one or more pieces of moment-associating information based at least in part on the event information.

3. The computer-implemented method of claim 2 , further comprising:

connecting with one or more calendar systems containing the event information; and

receiving the event information from the one or more calendar systems.

4. The computer-implemented method of claim 2 , wherein the transforming the one or more moment-associating elements into one or more pieces of moment-associating information based at least in part on the event information includes:

creating a custom language model based at least in part on the event information; and

transcribing the plurality of moment-associating segments into a plurality of transcribed segments based at least in part on the custom language model.

5. The computer-implemented method of claim 1 , wherein the receiving the one or more moment-associating elements includes assigning a timestamp associated with each element of the one or more moment-associating elements.

6. The computer-implemented method of claim 1 , wherein the one or more moment-associating elements includes at least one selected from a group consisting of one or more audio elements, one or more visual elements, and one or more environmental elements.

7. The computer-implemented method of claim 6 , wherein the one or more audio elements includes one or more voice elements of one or more voice-generating sources or one or more ambient sound elements.

8. The computer-implemented method of claim 6 , wherein the one or more visual elements includes at least one selected from a group consisting of one or more pictures, one or more images, one or more screenshots, one or more video frames, one or more projections, and one or more holograms.

9. The computer-implemented method of claim 6 , wherein the one or more environmental elements includes at least one selected from a group consisting of one or more global positions, one or more location types, and one or more moment conditions.

10. The computer-implemented method of claim 6 , wherein the one or more environmental elements includes at least one selected from a group consisting of a longitude, a latitude, an altitude, a country, a city, a street, a location type, a temperature, a humidity, a movement, a velocity of a movement, a direction of a movement, an ambient noise level, and one or more echo properties.

11. The computer-implemented method of claim 6 , wherein the transforming the one or more moment-associating elements into one or more pieces of moment-associating information includes:

segmenting the one or more audio elements into a plurality of audio segments;

assigning a segment speaker for each segment of the plurality of audio segments;

transcribing the plurality of audio segments into a plurality of text segments; and

generating the one or more pieces of moment-associating information based at least in part on the plurality of text segments and the segment speaker assigned for each segment of the plurality of audio segments.

12. The computer-implemented method of claim 11 , wherein the transcribing the plurality of audio segments into a plurality of text segments includes transcribing two or more segments of the plurality of audio segments in conjunction with each other.

13. The computer-implemented method of claim 1 , and further comprising:

receiving one or more voice elements of one or more voice-generating sources; and

receiving one or more voiceprints corresponding to the one or more voice-generating sources respectively.

14. The computer-implemented method of claim 13 , wherein the transforming the one or more moment-associating elements into one or more pieces of moment-associating information further includes:

segmenting the one or more moment-associating elements into the plurality of moment-associating segments based at least in part on the one or more voiceprints;

assigning a segment speaker for each segment of the plurality of moment-associating segments based at least in part on the one or more voiceprints; and

transcribing the plurality of moment-associating segments into the plurality of transcribed segments based at least in part on the one or more voiceprints.

15. The computer-implemented method of claim 13 , wherein the receiving one or more voiceprints corresponding to the one or more voice-generating sources respectively includes at least one of:

receiving one or more acoustic models corresponding to the one or more voice-generating sources respectively; or

receiving one or more language models corresponding to the one or more voice-generating sources respectively.

16. The computer-implemented method of claim 1 , wherein the transcribing the plurality of moment-associating segments into a plurality of transcribed segments includes:

transcribing a first segment of the plurality of moment-associating segments into a first transcribed segment of the plurality of transcribed segments;

transcribing a second segment of the plurality of moment-associating segments into a second transcribed segment of the plurality of transcribed segments; and

correcting the first transcribed segment based at least in part on the second transcribed segment.

17. The computer-implemented method of claim 1 , wherein the segmenting the one or more moment-associating elements into a plurality of moment-associating segments includes:

determining one or more speaker-change timestamps, each timestamp of the one or more speaker-change timestamps corresponding to a timestamp when a speaker change occurs;

determining one or more sentence-change timestamps, each timestamp of the one or more sentence-change timestamps corresponding to a timestamp when a sentence change occurs; and

determining one or more topic-change timestamps, each timestamp of the one or more topic-change timestamps corresponding to a timestamp when a topic change occurs.

18. The computer-implemented method of claim 17 , wherein the segmenting the one or more moment-associating elements into a plurality of moment-associating segments is performed based at least in part on one of:

the one or more speaker-change timestamps;

the one or more sentence-change timestamps; and

the one or more topic-change timestamps.

19. The computer-implemented method of claim 1 , wherein the subscription permission to the plurality of subscribers is granted in response to receiving an authentication token from the plurality of subscribers.

20. A system for team cooperation with real-time recording of one or more moment-associating elements, the system comprising:

an invitation delivering module configured to deliver, in response to an instruction, an invitation to a plurality of members associated with a workspace and one or more external users;

a permission module configured to grant subscription permission to a plurality of subscribers who have accepted the invitation, the plurality of subscribers including at least one member of the plurality of members and at least one external user of the one or more external users;

a receiving module configured to receive the one or more moment-associating elements;

a transforming module configured to transform the one or more moment-associating elements into one or more pieces of moment-associating information by at least:

segmenting the one or more moment-associating elements into a plurality of moment-associating segments,

assigning a segment speaker for each segment of the plurality of moment-associating segments,

transcribing the plurality of moment-associating segments into a plurality of transcribed segments, and

generating the one or more pieces of moment-associating information based at least in part on the plurality of transcribed segments and the segment speaker assigned for each segment of the plurality of moment-associating segments;

an editing module configured to receive one or more edits to the one or more pieces of moment-associating information from at least two of the plurality of members and update the one or more pieces of moment-associating information based at least in part on the one or more edits; and

a transmitting module configured to transmit at least one piece of the one or more pieces of updated moment-associating information to the plurality of subscribers, wherein at least one of the plurality of subscribers is not any member of the plurality of members.

21. A non-transitory computer-readable medium with instructions stored thereon, that when executed by a processor, perform the processes comprising:

delivering, in response to an instruction, an invitation to a plurality of members associated with a workspace and one or more external users;

granting subscription permission to a plurality of subscribers who have accepted the invitation, the plurality of subscribers including at least one member of the plurality of members and at least one external user of the one or more external users;

receiving the one or more moment-associating elements;

transforming the one or more moment-associating elements into one or more pieces of moment-associating information by at least:

segmenting the one or more moment-associating elements into a plurality of moment-associating segments,

assigning a segment speaker for each segment of the plurality of moment-associating segments,

transcribing the plurality of moment-associating segments into a plurality of transcribed segments, and

generating the one or more pieces of moment-associating information based at least in part on the plurality of transcribed segments and the segment speaker assigned for each segment of the plurality of moment-associating segments;

receiving, from at least two of the plurality of members, one or more edits to the one or more pieces of moment-associating information;

updating the one or more pieces of moment-associating information based at least in part on the one or more edits; and

transmitting at least one piece of the one or more pieces of updated moment-associating information to the plurality of subscribers, wherein at least one of the plurality of subscribers is not any member of the plurality of members.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE NAME OF THE ASSIGNEE ON THE COVERSHEET PREVIOUSLY RECORDED ON REEL 060821 FRAME 0462. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Sep 7, 2022
From: LAU, SIMON; FU, YUN; ALTREUTER, JAMES MASON; WILLIAMS, BRIAN FRANCIS; HUANG, XIAOKE; XING, TAO; SUN, WEN; LU, TAO; NAKAJIMA, KAISUKE; CHIN, KEAN KHEONG; GUPTA, HITESH ANAND; CHENG, JULIUS; PAN, JING; LIANG, SAM SONG
To: AISENSE, INC.
Reel/Frame 061389/0958 →
CHANGE OF NAME Recorded Aug 19, 2022
From: AISENSE INC.
To: OTTER.AI, INC.
Reel/Frame 061289/0954 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 16, 2022
From: LAU, SIMON; FU, YUN; ALTREUTER, JAMES MASON; WILLIAMS, BRIAN FRANCIS; HUANG, XIAOKE; XING, TAO; SUN, WEN; LU, TAO; NAKAJIMA, KAISUKE; CHIN, KEAN KHEONG; GUPTA, HITESH ANAND; CHENG, JULIUS; PAN, JING; LIANG, SAM SONG
To: OTTER.AI, INC.
Reel/Frame 060821/0462 →
Continuity (5)
Continuation 16780630 · Feb 3, 2020
Continuation In Part 16598820 · Oct 10, 2019
Provisional Application 62802098 · Feb 6, 2019
Provisional Application 62747001 · Oct 17, 2018
Related Publication 20220353102A1 · Nov 3, 2022
Cited By (8)
US 12,400,661 US 12,406,672 US 12,406,684 US 12,456,465 US 12,462,808 US 12,494,929 US 12,518,748 US 12,555,581