Granted Patent B2

US 11,210,461 · App. 16/027,202 · Granted Dec 28, 2021

Inventors: David Thomson (Madison, NJ); Ethan Selfridge (Jersey City, NJ)

Assignee: Interactions LLC

G06F40/205G06F40/20G10L15/063G10L15/1815G10L15/22G10L25/51H04L63/0407H04L63/0414H04L65/1023H04M3/42008H04M3/5175H04M3/51H04M2203/6009

Abstract

Claims (79)

1. A computer-implemented method, performed by a computer system, for masking sensitive personal information (SPI) of a caller from an agent, the computer system configured to receive a stream of media in real time from the caller and to provide the stream of media to the agent, the method comprising:

receiving a first portion of the stream of media in real time from the caller;

prior to receiving a next portion of the stream of media, determining a likelihood that the next portion of the stream of media will include SPI by inputting the first portion of the stream of media received from the caller into a natural language processing (NLP) model that is trained on features of SPI to recognize phrases indicating forthcoming SPI;

generating a predicted length of time that the next portion of the stream of media will include SPI by inputting the first portion of the stream of media received from the caller into a machine learning model that is trained to predict a length of time for a redaction;

responsive to the determined likelihood exceeding a threshold value:

receiving the next portion of the stream of media in real time from the caller;

and

masking, for the predicted length of time, the next portion of the stream of media as it is received in real time such that the agent does not receive the next portion of the stream of media containing the identified SPI.

2. The computer-implemented method of claim 1 , wherein the media stream is an audio stream comprising a conversation between the caller and the agent and wherein determining the likelihood that the next portion of the stream of media will include SPI comprises generating a text version of the received first portion of the stream of media.

3. The computer-implemented method of claim 1 , further comprising:

receiving a prompting media stream sent to the caller and originating from the agent;

providing information from the prompting media stream originating from the agent as input to a natural language processor; and

receiving, from the natural language processor, information about contents of the prompting media stream originating from the agent;

wherein determining the likelihood that the next portion of the stream of media received from the caller will include SPI is further based on the information received from the natural language processor.

4. The computer-implemented method of claim 1 , further comprising:

determining a likelihood that the first portion of the stream of media from the caller includes SPI; and

responsive to the determined likelihood that the first portion of the stream of media from the caller includes SPI exceeding a threshold value:

collecting the SPI from the first portion of the stream of media; and

masking the first portion of the stream of media such that the agent does not receive the first portion of the stream of media.

5. The computer-implemented method of claim 1 , further comprising:

training the NLP model to recognize SPI prompts, wherein SPI prompts include a second agent asking for a value from a user, or a user announcing an upcoming value.

6. The computer-implemented method of claim 1 , wherein the media stream from the caller is delayed before being provided to the agent, the method further comprising:

playing a delayed portion of the stream of media from the caller to the agent at an increased speed.

7. The computer-implemented method of claim 1 , further comprising:

analyzing subsequent portions of the stream of media from the caller to determine whether the subsequent portions of the stream of media include SPI; and

responsive to determining that a subsequent portion of the stream of media from the caller does not include SPI:

resuming transmission of the portions of the stream of media from the caller to the agent at an increased speed.

8. A non-transitory computer-readable storage medium storing computer program instructions executable by one or more processors of a system, to perform steps for masking sensitive personal information (SPI) of a caller from an agent, the system configured to receive a stream of media in real time from the caller and to provide the stream of media to the agent, the steps comprising:

receiving a first portion of the stream of media in real time from the caller;

prior to receiving a next portion of the stream of media, determining a likelihood that the next portion of the stream of media will include SPI by inputting the first portion of the stream of media received from the caller into a natural language processing (NLP) model that is trained on features of SPI to recognize phrases indicating forthcoming SPI;

generating a predicted length of time that the next portion of the stream of media will include SPI by inputting the first portion of the stream of media received from the caller into a machine learning model that is trained to predict a length of time for a redaction;

responsive to the determined likelihood exceeding a threshold value:

receiving the next portion of the stream of media in real time from the caller;

and

masking, for the predicted length of time, the next portion of the stream of media as it is received in real time such that the agent does not receive the next portion of the stream of media containing the identified SPI.

9. The non-transitory computer-readable storage medium of claim 8 wherein the media stream is an audio stream comprising a conversation between the caller and the agent and wherein determining the likelihood that the next portion of the stream of media will include SPI comprises generating a text version of the received first portion of the stream of media.

10. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising:

receiving a prompting media stream sent to the caller and originating from the agent;

providing information from the prompting media stream originating from the agent as input to a natural language processor; and

receiving, from the natural language processor, information about contents of the prompting media stream originating from the agent;

wherein determining the likelihood that the next portion of the stream of media received from the caller will include SPI is further based on the information received from the natural language processor.

11. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising:

determining a likelihood that the first portion of the stream of media from the caller includes SPI; and

responsive to the determined likelihood that the first portion of the stream of media from the caller includes SPI exceeding a threshold value:

collecting the SPI from the first portion of the stream of media; and

masking the first portion of the stream of media such that the agent does not receive the first portion of the stream of media.

12. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising:

training the NLP model to recognize SPI prompts, wherein SPI prompts include a second agent asking for a value from a user or a user announcing an upcoming value.

13. The non-transitory computer-readable storage medium of claim 8 wherein the media stream received from the caller is delayed before being provided to the agent, the method further comprising:

playing a delayed portion of the stream of media received from the caller to the agent at an increased speed.

14. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising:

analyzing subsequent portions of the stream of media from the caller to determine whether the subsequent portions of the stream of media include SPI; and

responsive to determining that a subsequent portion of the stream of media from the caller does not include SPI:

resuming transmission of the portions of the stream of media from the caller to the agent at an increased speed.

15. A computer system comprising:

one or more computer processors for executing computer program instructions; and

a non-transitory computer-readable storage medium storing instructions for masking sensitive personal information (SPI) of a caller from an agent, the system configured to receive a stream of media in real time from the caller and to provide the stream of media to the agent, the instructions executable by the one or more computer processors to perform steps comprising:

receiving a first portion of the stream of media in real time from the caller;

prior to receiving a next portion of the stream of media, determining a likelihood that the next portion of the stream of media will include SPI by inputting the first portion of the stream of media received from the caller into a natural language processing (NLP) model that is trained on features of SPI to recognize phrases indicating forthcoming SPI;

generating a predicted length of time that the next portion of the stream of media will include SPI by inputting the first portion of the stream of media received from the caller into a machine learning model that is trained to predict a length of time for a redaction;

responsive to the determined likelihood exceeding a threshold value:

receiving the next portion of the stream of media in real time from the caller;

and

masking, for the predicted length of time, the next portion of the stream of media as it is received in real time such that the agent does not receive the next portion of the stream of media containing the identified SPI.

16. The computer system of claim 15 wherein the media stream is an audio stream comprising a conversation between the caller and the agent and wherein determining the likelihood that the next portion of the stream of media will include SPI comprises generating a text version of the received first portion of the stream of media.

17. The computer system of claim 15 , the steps further comprising:

receiving a prompting media stream sent to the caller and originating from the agent;

providing information from the prompting media stream originating from the agent as input to a natural language processor; and

receiving, from the natural language processor, information about contents of the prompting media stream originating from the agent;

wherein determining the likelihood that the next portion of the stream of media received from the caller will include SPI is further based on the information received from the natural language processor.

18. The computer system of claim 15 , the steps further comprising:

determining a likelihood that the first portion of the stream of media from the caller includes SPI; and

responsive to the determined likelihood that the first portion of the stream of media from the caller includes SPI exceeding a threshold value:

collecting the SPI from the first portion of the stream of media; and

masking the first portion of the stream of media such that the agent does not receive the first portion of the stream of media.

19. The computer system of claim 15 , the steps further comprising:

training the NLP model to recognize SPI prompts, wherein SPI prompts include a second agent asking for a value from a user or a user announcing an upcoming value.

20. The computer system of claim 15 wherein the media stream from the caller is delayed before being provided to the agent, the method further comprising:

playing a delayed portion of the stream of media from the caller to the agent at an increased speed.

Assignments (8)

Cited By (2)

US 12,586,598 US 12,597,434