IP Library Granted Patent US 12,388,888
Granted Patent B2
US 12,388,888 · App. 18/069,372 · Granted Aug 12, 2025

Packet replacement during poor network connectivity or network congestion

Inventor: Nick Swerdlow (Santa Clara, CA)
Assignee: Zoom Communications, Inc.
H04L65/75G10L15/26H04L65/403
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,388,888
App. No.
18/069,372
Granted
Aug 12, 2025
Kind
B2
Abstract

A server generates a continuous audio stream during periods of poor network connectivity or network congestion. The server obtains a first audio stream from a user device connected to a real-time communication session and detects one or more missing packets in the first audio stream. The server obtains recorded portions of audio that correspond to the missing packets and combines the recorded portions of the audio with the first audio stream to generate the continuous audio stream without missing packets. The server transmits the continuous audio stream to other user devices connected to the real-time communication session.

Claims (61)

1. A method comprising:

obtaining a first audio stream from a user device connected to a real-time communication session via a unified communications as a service (UCaaS) platform;

determining a context of a phrase in the first audio stream;

determining that a syntax of the phrase in the first audio stream is indeterminable;

generating, using a deep neural network (DNN) trained on speech cadence data associated with a speed at which a user speaks to convert text data to a wave form, synthesized words based on the determination that the syntax of the phrase in the first audio stream is indeterminable, wherein the synthesized words are generated based on the context of the phrase in the first audio stream using a speech profile that includes a geographic region;

combining the synthesized words with the first audio stream to generate a second audio stream; and

transmitting the second audio stream to one or more other user devices connected to the real-time communication session.

2. The method of claim 1 , wherein the synthesized words are generated using a machine learning model.

3. The method of claim 1 , further comprising:

detecting speech data in the first audio stream;

converting the speech data to text data; and

transmitting the text data to one or more real-time communication session participant devices.

4. The method of claim 1 , further comprising:

detecting speech data in the first audio stream;

converting the speech data to text data; and

transmitting the text data to one or more real-time communication session participant devices, wherein the text data is embedded in the second audio stream.

5. The method of claim 1 , wherein the first audio stream is stored in a buffer for a threshold duration to combine the synthesized words with the first audio stream.

6. The method of claim 1 , wherein the synthesized words correspond to missing packets.

7. The method of claim 1 , wherein the synthesized words correspond to respective missing packets, and wherein combining the synthesized words with the first audio stream comprises:

correlating a timestamp of the synthesized words to a sequence number of the respective missing packets.

8. A system comprising:

a first user device connected to a real-time communication session via a unified communications as a service (UCaaS) platform;

a second user device connected to the real-time communication session via the UCaaS platform; and

a server configured to:

obtain a first audio stream from the first user device;

determine a context of a phrase in the first audio stream;

determine that a syntax of the phrase in the first audio stream is indeterminable;

generate, via a deep neural network (DNN) trained on speech cadence data associated with a speed at which a user speaks to convert text data to a wave form, synthesized words based on the determination that the syntax of the phrase in the first audio stream is indeterminable, wherein the synthesized words are generated based on the context of the phrase in the first audio stream using a speech profile that includes a geographic region;

combine the synthesized words with the first audio stream to generate a second audio stream; and

transmit the second audio stream to the second user device.

9. The system of claim 8 , wherein the server is further configured to:

transmit an acknowledgement to the first user device responsive to the first audio stream.

10. The system of claim 8 , wherein the server is further configured to:

detect speech data in the first audio stream;

convert the speech data to text data; and

simultaneously transmit the text data and the second audio stream to the second user device.

11. The system of claim 8 , wherein the server is further configured to:

detect speech data in the first audio stream;

convert the speech data to text data; and

embed the text data in the second audio stream.

12. The system of claim 8 , wherein the server is further configured to:

store the first audio stream in a buffer for a threshold duration.

13. The system of claim 8 , wherein the synthesized words correspond to one or more missing packets.

14. The system of claim 8 , wherein the server is further configured to:

generate a list of predicted words based on the syntax of the phrase in the first audio stream.

15. A non-transitory computer-readable medium comprising instructions stored in a memory, that when executed by a processor, cause the processor to perform operations comprising:

obtaining a first audio stream from a user device connected to a real-time communication session via a unified communications as a service (UCaaS) platform;

determining a context of a phrase in the first audio stream;

determining that a syntax of the phrase in the first audio stream is indeterminable;

generating, using a deep neural network (DNN) trained on speech cadence data associated with a speed at which a user speaks to convert text data to a wave form, synthesized words based on the determination that the syntax of the phrase in the first audio stream is indeterminable, wherein the synthesized words are generated based on the context of the phrase in the first audio stream using a speech profile that includes a geographic region;

combining the synthesized words with the first audio stream to generate a second audio stream; and

transmitting the second audio stream to one or more other user devices connected to the real-time communication session.

16. The non-transitory computer-readable medium of claim 15 , wherein the synthesized words are generated using a machine learning model.

17. The non-transitory computer-readable medium of claim 15 , the operations further comprising:

detecting speech data in the first audio stream;

converting the speech data to text data; and

transmitting the text data to one or more real-time communication session participants.

18. The non-transitory computer-readable medium of claim 15 , wherein the first audio stream is stored in a buffer to combine the synthesized words with the first audio stream.

19. The non-transitory computer-readable medium of claim 15 , wherein the synthesized words correspond to respective missing packets.

20. The non-transitory computer-readable medium of claim 15 , wherein combining the synthesized words with the first audio stream comprises:

inserting the synthesized words into respective timeslots corresponding to respective missing packets.

Assignments (2)
CHANGE OF NAME Recorded Jan 7, 2025
From: ZOOM VIDEO COMMUNICATIONS, INC.
To: ZOOM COMMUNICATIONS, INC.
Reel/Frame 069839/0593 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 21, 2022
From: SWERDLOW, NICK
To: ZOOM VIDEO COMMUNICATIONS, INC.
Reel/Frame 062169/0334 →