IP Library Granted Patent US 12,652,326
Granted Patent B2
US 12,652,326 · App. 17/813,340 · Granted Jun 9, 2026

Indicator for avoiding speech confliction in a communications session when network latency is high

Inventors: Zongpeng Qiao (Nanjing City, CN); Dan Hu (Nanjing City, CN); Ke Xu (Nanjing City, CN); Jia Yin (Nanjing City, CN)
H04L65/4038G08B5/36G10L25/78G10L25/87H04L43/0852H04L65/1069
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,652,326
App. No.
17/813,340
Filed
Jul 19, 2022
Granted
Jun 9, 2026
Kind
B2
Art Unit
2693
USPC
704/231
Abstract

A computing system includes first and second client computing devices accessing a communications network to establish a communications session. The first client computing device operates an audio analysis agent to determine network latency within the communications session based on communications with an audio analysis agent in the second client computing device. In response to the network latency exceeding a latency threshold, audio input from a user of the first client computing device is analyzed to determine a speaking status of the user. The audio analysis agent generates an indicator command message for the second client computing device based on the determined speaking status of the user. The second client computing device displays an indicator based on the indicator command message indicating when a user of the second client computing device can speak to avoid speech confliction with the user of said first client computing device.

Claims (66)

1 . A computing system comprising:

first and second client computing devices configured to access a communications network to establish a communications session, each client computing device comprising an audio analysis agent;

said first client computing device is configured to operate the audio analysis agent to perform the following:

determine network latency within the communications session based on communications with the audio analysis agent in said second client computing device;

in response to the network latency exceeding a latency threshold, analyze audio input from a user of said first client computing device to determine a speaking status of the user;

and generate an indicator command message for said second client computing device based on the determined speaking status of the user; and

said second client computing device configured to perform the following:

display an indicator based on the indicator command message indicating when a user of said second client computing device can speak to avoid speech confliction with the user of said first client computing device;

wherein determining the network latency is further based on the following:

the audio analysis agent in said first client computing device transmits a monitoring message with a start timestamp;

the audio analysis agent in said second client computing device applies an end timestamp upon receipt of the monitoring message;

determine, by said first client device or said second client device, a time gap between the start and end timestamps; and

compare the time gap to the latency threshold.

2 . The computing system according to claim 1 further comprising additional client computing devices participating in the communications session, wherein the audio analysis agent in said first client computing device is further configured to perform the following:

communicate with the audio analysis agents in said additional client computing devices to determine the network latency for each additional client computing device within the communications session; and

select a worst network latency from among all of the determined network latencies to be used to compare to the latency threshold.

3 . The computing system according to claim 1 wherein in response to the network latency exceeding the latency threshold, the audio analysis agent in said first client computing device further communicates with the audio analysis agent in said second client computing device to enable the indicator for said second client computing device.

4 . The computing system according to claim 3 wherein the network latency is periodically determined, and in response to the network latency being less than the latency threshold, the indicator is disabled by said first client computing device.

5 . The computing system according to claim 1 wherein the indicator displays at least one of colors, text or graphics to the user of said second client computing device to indicate the speaking status of the user of said first client computing device.

6 . The computing system according to claim 5 wherein the colors include:

a red color indicating that the user is talking;

a yellow color indicating that there is a pause in the user speaking; and

a green color indicating that the user has stopped talking based on the pause being extended.

7 . The computing system according to claim 1 wherein said first and second client computing devices each comprise an audio buffer, and wherein the indicator command message includes the following:

a size of the audio buffer in said first client computing device representing audio data that has not been sent yet to said second computing device; and

an indicator command for the indicator in said second client computing device to display a speaking status of the user of said first client computing device.

8 . The computing system according to claim 7 wherein in response to there being a pause in the user of said first client computing device speaking, the audio analysis agent in said second client computing device displays the indicator based on the indicator command after the audio data has been received by the audio buffer in said second client computing device.

9 . The computing system according to claim 1 wherein the audio analysis agent in said first client computing device analyzes the audio input from the user based on the following:

divide the audio input into a plurality of overlapping windows;

calculate signal energy of the audio input in each window; and

compare the calculated signal energy in each window to an audio threshold, with the calculated signal energy below the audio threshold representing user silence, and with the calculated signal energy above the audio threshold representing that the user is speaking.

10 . The computing system according to claim 1 wherein said first and second client computing devices are each configured to launch a collaboration app for the users to participate in a collaboration session, with the communications session operating within in the collaboration session.

11 . The computing system according to claim 10 wherein the collaboration app is accessed via a virtual computing session, and wherein the indicator is configured as a plug-in to the collaboration app.

12 . A client computing device comprising:

a display;

a processor coupled to said display and configured to perform the following:

access a communications network to establish a communications session with at least one other client computing device, and execute an audio analysis agent to perform the following:

determine network latency within the communications session based on communications with an audio analysis agent in the at least one other client computing device;

in response to the network latency exceeding a latency threshold, analyze audio input from a user of the client computing device to determine a speaking status of the user; and

generate an indicator command message for the at least one other client computing device based on the determined speaking status of the user, with the indicator command message causing display of an indicator by the at least one other client computing device indicating when a user of the at least one other client computing device can speak to avoid speech confliction with the user of the client computing device;

wherein determining the network latency is further based on the following:

the audio analysis agent in said first client computing device transmits a monitoring message with a start timestamp;

the audio analysis agent in said second client computing device applies an end timestamp upon receipt of the monitoring message;

determine, by said first client device or said second client device, a time gap between the start and end timestamps; and

compare the time gap to the latency threshold.

13 . The client computing device according to claim 12 wherein in response to the network latency exceeding the latency threshold, the audio analysis agent further communicates with the audio analysis agent in the at least one other client computing device to enable the indicator for the at least one other client computing device.

14 . The client computing device according to claim 13 wherein the network latency is periodically determined, and in response to the network latency being less than the latency threshold, the audio analysis agent further communicates with the audio analysis agent in the at least one other client computing device to disable the indicator for the at least one other client computing device.

15 . The client computing device according to claim 12 further comprising an audio buffer, and wherein the indicator command message includes the following:

a size of the audio buffer representing audio data that has not been sent yet to the at least one other computing device; and

an indicator command for the indicator in the at least one other client computing device to display a speaking status of the user of the first client computing device.

16 . The client computing device according to claim 12 wherein the audio analysis agent analyzes the audio input from the user based on the following:

divide the audio input into a plurality of overlapping windows;

calculate signal energy of the audio input in each window; and

compare the calculated signal energy in each window to an audio threshold, with the calculated signal energy below the audio threshold representing user silence, and with the calculated signal energy above the audio threshold representing that the user is speaking.

17 . The client computing device according to claim 12 wherein said processor is further configured to launch a collaboration app for the users to participate in a collaboration session, with the communications session operating within in the collaboration session.

18 . A method comprising:

operating a client computing device to access a communications network to establish a communications session with at least one other client computing device; and

executing an audio analysis agent within the client computing device to perform the following:

determine network latency within the communications session based on communications with an audio analysis agent in the at least one other client computing device;

in response to the network latency exceeding a latency threshold, analyze audio input from a user of the client computing device to determine a speaking status of the user; and

generate an indicator command message for the at least one other client computing device based on the determined speaking status of the user, with the indicator command message causing display of an indicator by the at least one other client computing device indicating when a user of the at least one other client computing device can speak to avoid speech confliction with the user of the client computing device;

wherein determining the network latency is further based on the following:

the audio analysis agent in said first client computing device transmits a monitoring message with a start timestamp;

the audio analysis agent in said second client computing device applies an end timestamp upon receipt of the monitoring message;

determine, by said first client device or said second client device, a time gap between the start and end timestamps; and

compare the time gap to the latency threshold.