IP Library Granted Patent US 12,212,790
Granted Patent B2
US 12,212,790 · App. 18/373,253 · Granted Jan 28, 2025

Methods and systems for efficient streaming of audio from contact center cloud platform to third-party servers

Inventors: Camiel Coppelmans (Menlo Park, CA); Felix Immanuel Wyss (Indianapolis, IN); Matt Jahns (Indianapolis, IN)
H04N21/233H04N21/2181
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,212,790
App. No.
18/373,253
Granted
Jan 28, 2025
Kind
B2
Abstract

A method for providing streaming audio over a network from a client to a server associated with a third-party service. The method includes receiving configuration data from a tenant regarding the server. The method may further include establishing a connection with the server via a connection process. The connection process includes sending a connection request that includes: a bidirectional communication protocol configured to transmit data from the client to the server and from the server to the client by reusing an established connection channel; a tenant-Id parameter; and a session-Id parameter. The method may further include implementing an open transaction for initiating the streaming of the audio of the ongoing conversation to the server. The open transaction includes transmitting participant parameters identifying the customer and at least one choice regarding an audio format for the audio streaming.

Claims (66)

1. A computer-implemented method for providing streaming audio over a network from a client, which comprises a contact center cloud platform, to a server associated with a third-party service, wherein the streaming audio comprises audio derived from an ongoing conversation occurring via the contact center cloud platform between an agent and a customer, the agent comprising a representative of a tenant of the contact center cloud platform and the customer being a customer of the tenant, the method comprising the steps of:

receiving, by the contact center cloud platform, configuration data from the tenant regarding establishing a connection with the server of the third-party service, the configuration data including at least a network address of the server;

in relation to streaming the audio of the ongoing conversation, establishing, by the contact center cloud platform, a connection with the server of the third-party service via a connection process given the configuration data provided by the tenant, wherein the connection process includes sending a connection request that comprises:

a bidirectional communication protocol configured to transmit data from the client to the server and from the server to the client by reusing an established connection channel;

a tenant-Id parameter that identifies the tenant; and

a session-Id parameter that identifies an audio streaming session to which the connection request pertains;

in response to establishing the connection with the server of the third-party service, implementing, by the contact center cloud platform, an open transaction for initiating the streaming of the audio of the ongoing conversation to the server, wherein the open transaction comprises:

transmitting an open message to the server that includes:

participant parameters identifying the customer; and

at least one choice regarding an audio format for the audio streaming;

receiving an opened message from the server that indicates:

a readiness to receive the streaming audio of the ongoing conversation; and

a selection in regard to the at least one choice regarding the audio format;

streaming, by the contact center cloud platform, the audio of the ongoing conversation pursuant to a streaming protocol, wherein the streaming protocol comprises sending successive messages per the bidirectional communication protocol that each includes:

raw audio data consistent with the selection regarding the audio format;

a position property that indicates a current position in the audio stream from a perspective of the client when the client sent the message.

2. The method of claim 1 , wherein the third-party service comprises an unaffiliated entity in relation to the contact center cloud platform, the third-party service providing analytics for the tenant for which the streaming audio is required.

3. The method of claim 1 , wherein the step of establishing the connection with the server of the third-party service is performed in relation to detecting an occurrence of a triggering event;

further comprising the step of monitoring, by the contact center cloud platform, conversations between agents representing the tenant and customers of the tenant and, in regard to the ongoing conversation between the agent and the customer, detecting, by the contact center cloud platform, a triggering event signaling a need to stream the audio of the on-going conversation to the server.

4. The method of claim 3 , wherein the trigging event comprises one of: an initiation of the ongoing conversation; and the ongoing conversation being transferred to a predetermined queue.

5. The method of claim 2 , wherein the bidirectional communication protocol comprises a Web Sockets protocol.

6. The method of claim 5 , wherein the connection request comprises an HTTP GET request with upgrade header.

7. The method of claim 6 , wherein the connection request further comprises a signature, the client employing a client secret as a key to sign the connection request with the signature.

8. The method of claim 1 , wherein the at least one choice regarding the audio format comprises both a first choice regarding a codec for streaming the audio, and a second choice regarding a sampling rate.

9. The method of claim 1 , wherein the open message further comprises a choice of channels, the choice of channels providing alternatives to the server of receiving:

streaming audio of an internal channel that provides audio of what the customer is hearing;

streaming audio of an external channel that provides audio of what the customer is saying;

streaming audio of both the internal and external channels;

wherein the opened message received from the server further includes a selection in regard to the choice of channels.

10. The method of claim 1 , wherein the streaming protocol further comprises the client maintaining a history buffer of at least 20 seconds of the audio of the ongoing conversation.

11. The method of claim 10 , wherein the streaming protocol further comprises the client at regular intervals:

sending a ping message to the server;

receiving a pong message from the server in response to the ping message; and

calculating a round-trip time given the ping message and the pong message reply.

12. A system for providing streaming audio over a network from a client, which comprises a contact center cloud platform, to a server associated with a third-party service, wherein the streaming audio comprises audio derived from an ongoing conversation occurring via the contact center cloud platform between an agent and a customer, the agent comprising a representative of a tenant of the contact center cloud platform and the customer being a customer of the tenant, the method comprising the steps of:

a processor; and

a memory storing instructions which, when executed by the processor, cause the processor to perform a process that includes the steps of:

receiving, by the contact center cloud platform, configuration data from the tenant regarding establishing a connection with the server of the third-party service, the configuration data including at least a network address of the server;

in relation to streaming the audio of the ongoing conversation, establishing, by the contact center cloud platform, a connection with the server of the third-party service via a connection process given the configuration data provided by the tenant, wherein the connection process includes sending a connection request that comprises:

a bidirectional communication protocol configured to transmit data from the client to the server and from the server to the client by reusing an established connection channel;

a tenant-Id parameter that identifies the tenant; and

a session-Id parameter that identifies an audio streaming session to which the connection request pertains;

in response to establishing the connection with the server of the third-party service, implementing, by the contact center cloud platform, an open transaction for initiating the streaming of the audio of the ongoing conversation to the server, wherein the open transaction comprises:

transmitting an open message to the server that includes:

participant parameters identifying the customer; and

at least one choice regarding an audio format for the audio streaming;

receiving an opened message from the server that indicates:

a readiness to receive the streaming audio of the ongoing conversation; and

a selection in regard to the at least one choice regarding the audio format;

streaming, by the contact center cloud platform, the audio of the ongoing conversation pursuant to a streaming protocol, wherein the streaming protocol comprises sending successive messages per the bidirectional communication protocol that each includes:

raw audio data consistent with the selection regarding the audio format;

a position property that indicates a current position in the audio stream from a perspective of the client when the client sent the message.

13. The system of claim 12 , wherein the third-party service comprises an unaffiliated entity in relation to the contact center cloud platform, the third-party service providing analytics for the tenant for which the streaming audio is required.

14. The system of claim 12 , wherein the step of establishing the connection with the server of the third-party service is performed in relation to detecting an occurrence of a triggering event;

wherein the instructions, when executed by the processor, further cause the processor to perform the steps off:

monitoring, by the contact center cloud platform, conversations between agents representing the tenant and customers of the tenant and, in regard to the ongoing conversation between the agent and the customer, detecting, by the contact center cloud platform, a triggering event signaling a need to stream the audio of the on-going conversation to the server.

15. The system of claim 14 , wherein the trigging event comprises one of: an initiation of the ongoing conversation; and the ongoing conversation being transferred to a predetermined queue.

16. The system of claim 13 , wherein the bidirectional communication protocol comprises a Web Sockets protocol.

17. The system of claim 16 , wherein the connection request comprises an HTTP GET request with upgrade header.

18. The system of claim 17 , wherein the connection request further comprises a signature, the client employing a client secret as a key to sign the connection request with the signature.

19. The system of claim 12 , wherein the at least one choice regarding the audio format comprises both a first choice regarding a codec for streaming the audio, and a second choice regarding a sampling rate.

20. The system of claim 12 , wherein the open message further comprises a choice of channels, the choice of channels providing alternatives to the server of receiving:

streaming audio of an internal channel that provides audio of what the customer is hearing;

streaming audio of an external channel that provides audio of what the customer is saying;

streaming audio of both the internal and external channels;

wherein the opened message received from the server further includes a selection in regard to the choice of channels.

Assignments (3)
NOTICE OF SUCCESSION OF SECURITY INTERESTS AT REEL/FRAME 067718/0823 Recorded Feb 4, 2025
From: BANK OF AMERICA, N.A., AS RESIGNING AGENT
To: GOLDMAN SACHS BANK USA, AS SUCCESSOR AGENT
Reel/Frame 070098/0300 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 5, 2024
From: COPPELMANS, CAMIEL; WYSS, FELIX IMMANUEL; JAHNS, MATTHEW
To: GENESYS CLOUD SERVICES, INC.
Reel/Frame 069142/0335 →
SECURITY AGREEMENT Recorded Jun 11, 2024
From: GENESYS CLOUD SERVICES, INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 067718/0823 →
Continuity (2)
Provisional Application 63409844 · Sep 26, 2022
Related Publication 20240107083A1 · Mar 28, 2024
References Cited (4)
US 11528302B2 · Gutta · 2022 [cited by examiner]
US 11637929B2 · Balasaygun · 2023 [cited by examiner]
US 20180176281A1 · Arunachalam · 2018 [cited by examiner]
US 20210321002A1 · DiMaria · 2021 [cited by examiner]