IP Library Granted Patent US 11,798,528
Granted Patent B2
US 11,798,528 · App. 17/497,225 · Granted Oct 24, 2023

Systems and methods for providing notifications within a media asset without breaking immersion

Inventors: Vikram Makam Gupta (Karnataka, IN); Prateek Varshney (Karnataka, IN); Madhusudhan Seetharam (Karnataka, IN); Ashish Kumar Srivastava (Karnataka, IN); Harshith Kumar Gejjegondanahally Sreekanth (Karnataka, IN)
Assignee: Rovi Guides, Inc.
G10L13/08G06F40/205G06F40/279G10L13/00G10L13/033H04M1/72433H04W68/005H04M1/72442H04M2201/39
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,798,528
App. No.
17/497,225
Granted
Oct 24, 2023
Kind
B2
Abstract

Systems and methods for providing notifications without breaking media immersion. A notification delivery application receives notification data while a media device provides a media asset. In response to receiving the notification data while the media device provides the media asset, the notification delivery application generates a voice model based on a voice detected in the media asset. The notification delivery application converts the notification data to synthesized speech using the voice model and generates, by the media device, the synthesized speech for output at an appropriate point in the media asset based on contextual features of the media asset.

Claims (101)

1. A method comprising:

receiving notification data while a media device provides a media asset;

in response to receiving the notification data while the media device provides the media asset:

determining whether the media asset comprises a voice;

converting the notification data to synthesized speech using a text-to-voice model generated based on characteristics of the voice;

determining a position in the media asset for pausing the media asset and outputting the synthesized speech, based on contextual features of the media asset or the notification data;

pausing the media asset; and

generating, for output at the position in the media asset by the media device, the synthesized speech.

2. The method of claim 1 , wherein determining whether the media asset comprises the voice comprises:

extracting frequency and temporal characteristics from the media asset;

retrieving, from memory, vocal characteristics that comprise frequency and temporal information of speech;

comparing the frequency and temporal characteristics from the media asset with the vocal characteristics; and

in response to determining that the frequency and temporal characteristics correspond to the vocal characteristics, determining that the media asset comprises the voice.

3. The method of claim 1 , further comprising:

in response to determining that the media asset comprises the voice, generating the text-to-voice model based on characteristics of the voice; and

wherein converting the notification data to the synthesized speech using the text-to-voice model generated based on characteristics of the voice further comprises:

identifying textual information in the notification data; and

generating the synthesized speech based on the textual information, wherein the synthesized speech is an audio clip comprising a recitation, made by the text-to-voice model, of the textual information.

4. The method of claim 1 , wherein determining the position in the media asset for pausing the media asset and outputting the synthesized speech further comprises:

parsing the notification data into textual information;

identifying a keyword from the textual information;

retrieving, from memory, a plurality of priority keywords, wherein each priority keyword of the plurality of priority keywords is associated with a respective priority level;

comparing the keyword from the textual information to each priority keyword of the plurality of priority keywords; and

in response to determining that the keyword from the textual information matches a first priority keyword that is associated with a first priority level, determining the position in the media asset for pausing the media asset and outputting the synthesized speech, based on both the first priority level and the contextual features of the media asset or the notification data.

5. The method of claim 1 , wherein determining the position in the media asset for pausing the media asset and outputting the synthesized speech comprises:

retrieving notification access data from memory, wherein the notification access data is indicative of receipt times and access times for a plurality of notification types;

identifying a notification type associated with the notification data;

determining, based on the notification access data, an access delay for the notification type, wherein the access delay represents a time difference between when a notification of the notification type was received and when the notification of the notification type was accessed;

identifying a current play position of the media asset; and

determining that the position is a sum of the current play position and the access delay.

6. The method of claim 1 , wherein determining the position in the media asset for pausing the media asset and outputting the synthesized speech comprises:

determining the contextual features of the media asset, wherein the contextual features comprise silence periods, by:

retrieving metadata of the media asset; and

identifying, based on the metadata, a plurality of silence periods in the media asset, wherein a silence period of the plurality of silence periods is indicative of a time period in the media asset in which no voices are detected;

determining a candidate position in the media asset that is within the silence period;

identifying the candidate position as the position in the media asset for pausing the media asset and outputting the synthesized speech.

7. The method of claim 1 , wherein the contextual features comprise keywords and wherein determining the position in the media asset for pausing the media asset and outputting the synthesized speech comprises:

retrieving a keyword from memory;

retrieving metadata of the media asset;

identifying, based on the metadata, a time position in the media asset at which the keyword is recited;

identifying a silence period in the media asset that subsequently follows the time position at which the keyword is recited;

determining a candidate position in the media asset that is within the silence period;

identifying the candidate position as the position in the media asset for pausing the media asset and outputting the synthesized speech.

8. The method of claim 1 , wherein determining the position in the media asset for pausing the media asset and outputting the synthesized speech comprises:

detecting that a different voice is being outputted in the media asset;

determining a second position in the media asset when the different voice ceases output; and

identifying the second position as the position in the media asset for pausing the media asset and outputting the synthesized speech.

9. The method of claim 1 , wherein generating, for output at the position in the media asset by the media device, the synthesized speech comprises:

unpausing the media asset in response to completing output of the synthesized speech.

10. The method of claim 1 , wherein generating, for output at the position in the media asset by the media device, the synthesized speech, comprises outputting the synthesized speech at a higher frequency than a normal frequency of the voice.

11. A system comprising:

audio output circuitry;

control circuitry configured to:

receive notification data while a media device provides a media asset;

in response to receiving the notification data while the media device provides the media asset:

determine whether the media asset comprises a voice;

convert the notification data to synthesized speech using a text-to-voice model generated based on characteristics of the voice;

determine a position in the media asset for pausing the media asset and outputting the synthesized speech, based on contextual features of the media asset or the notification data;

pause the media asset; and

generate, via the audio output circuitry, the synthesized speech for output at the position in the media asset by the media device.

12. The system of claim 11 , wherein the control circuitry, when determining whether the media asset comprises the voice, is further configured to:

extract frequency and temporal characteristics from the media asset;

retrieve, from memory, vocal characteristics that comprise frequency and temporal information of speech;

compare the frequency and temporal characteristics from the media asset with the vocal characteristics; and

in response to determining that the frequency and temporal characteristics correspond to the vocal characteristics, determine that the media asset comprises the voice.

13. The system of claim 11 , wherein the control circuitry is further configured to, in response to determining that the media asset comprises the voice, generate the text-to-voice model based on characteristics of the voice; and

wherein the control circuitry, when converting the notification data to the synthesized speech using the text-to-voice model generated based on characteristics of the voice, is further configured to:

identify textual information in the notification data; and

generate, via the audio output circuitry, the synthesized speech based on the textual information, wherein the synthesized speech is an audio clip comprising a recitation, made by the text-to-voice model, of the textual information.

14. The system of claim 11 , wherein the control circuitry, when determining the position in the media asset for pausing the media asset and outputting the synthesized speech, is further configured to:

parse the notification data into textual information;

identify a keyword from the textual information;

retrieve, from memory, a plurality of priority keywords, wherein each priority keyword of the plurality of priority keywords is associated with a respective priority level;

compare the keyword from the textual information to each priority keyword of the plurality of priority keywords; and

in response to determining that the keyword from the textual information matches a first priority keyword that is associated with a first priority level, determine the position in the media asset for pausing the media asset and outputting the synthesized speech, based on both the first priority level and the contextual features of the media asset or the notification data.

15. The system of claim 11 , wherein the control circuitry, when determining the position in the media asset for pausing the media asset and outputting the synthesized speech, is further configured to:

retrieve notification access data from memory, wherein the notification access data is indicative of receipt times and access times for a plurality of notification types;

identify a notification type associated with the notification data;

determine, based on the notification access data, an access delay for the notification type, wherein the access delay represents a time difference between when a notification of the notification type was received and when the notification of the notification type was accessed;

identify a current play position of the media asset; and

determine that the position is a sum of the current play position and the access delay.

16. The system of claim 11 , wherein the control circuitry, when determining the position in the media asset for pausing the media asset and outputting the synthesized speech, is further configured to:

determine the contextual features of the media asset, wherein the contextual features comprise silence periods, by:

retrieving metadata of the media asset; and

identifying, based on the metadata, a plurality of silence periods in the media asset, wherein a silence period of the plurality of silence periods is indicative of a time period in the media asset in which no voices are detected;

determine a candidate position in the media asset that is within the silence period;

identify the candidate position as the position in the media asset for pausing the media asset and outputting the synthesized speech.

17. The system of claim 11 , wherein the contextual features comprise keywords and wherein the control circuitry, when determining the position in the media asset for pausing the media asset and outputting the synthesized speech, is further configured to:

retrieve a keyword from memory;

retrieve metadata of the media asset;

identify, based on the metadata, a time position in the media asset at which the keyword is recited;

identify a silence period in the media asset that subsequently follows the time position at which the keyword is recited;

determine a candidate position in the media asset that is within the silence period;

identify the candidate position as the position in the media asset for pausing the media asset and outputting the synthesized speech.

18. The system of claim 11 , wherein the control circuitry, when determining the position in the media asset for pausing the media asset and outputting the synthesized speech, is further configured to:

detect that a different voice is being outputted in the media asset;

determine a second position in the media asset when the different voice ceases output; and

identify the second position as the position in the media asset for pausing the media asset and outputting the synthesized speech.

19. The system of claim 11 , wherein the control circuitry, when generating, via the audio output circuitry, the synthesized speech for output at the position in the media asset by the media device, is further configured to

unpause the media asset in response to completing output of the synthesized speech.

20. The system of claim 11 , wherein the control circuitry, when generating, via the audio output circuitry, the synthesized speech for output at the position in the media asset by the media device, is further configured to output the synthesized speech at a higher frequency than a normal frequency of the voice.

Assignments (3)
CHANGE OF NAME Recorded Oct 3, 2024
From: ROVI GUIDES, INC.
To: ADEIA GUIDES INC.
Reel/Frame 069106/0164 →
SECURITY INTEREST Recorded May 19, 2023
From: ADEIA GUIDES INC.; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063707/0884 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 8, 2021
From: GUPTA, VIKRAM MAKAM; VARSHNEY, PRATEEK; SEETHARAM, MADHUSUDHAN; SRIVASTAVA, ASHISH KUMAR; SREEKANTH, HARSHITH KUMAR GEJJEGONDANAHALLY
To: ROVI GUIDES, INC.
Reel/Frame 057742/0398 →
Continuity (2)
Continuation 16144395 · Sep 27, 2018
Related Publication 20220044669A1 · Feb 10, 2022