Systems and methods for providing notifications within a media asset without breaking immersion
Systems and methods for providing notifications without breaking media immersion. A notification delivery application receives notification data while a media device provides a media asset. In response to receiving the notification data while the media device provides the media asset, the notification delivery application generates a voice model based on a voice detected in the media asset. The notification delivery application converts the notification data to synthesized speech using the voice model and generates, by the media device, the synthesized speech for output at an appropriate point in the media asset based on contextual features of the media asset.
1. A method comprising:
receiving notification data during a display of a media asset by a media device, wherein the notification data is unrelated to the media asset;
in response to receiving the notification data during the display of the media asset on the media device:
determining that the media asset comprises a voice;
determining that the notification data comprises non-textual visual information;
converting the non-textual visual information to text;
converting the text to synthesized speech using a text-to-voice model generated based on characteristics of the voice; and
generating, for output by the media device, the synthesized speech by:
determining a position in the media asset for outputting the synthesized speech, based on one or more of contextual features of the media asset and the notification data; and
generating, for output at the position in the media asset by the media device, the synthesized speech.
2. The method of claim 1 , wherein the determining that the notification data comprises the non-textual visual information comprises:
determining that the notification data consists of a payload comprising an image.
3. The method of claim 2 , wherein the image is part of a social media post.
4. The method of claim 1 , wherein the non-textual visual information is an image, and wherein the converting the image to the text comprises:
using image processing and machine learning to identify objects in the image;
using machine learning to textually describe the identified objects in the image.
5. The method of claim 4 , wherein the image processing is edge detection and segmentation, and wherein the machine learning to identify the objects in the image comprises classification.
6. The method of claim 1 , wherein the determining the position in the media asset for outputting the synthesized speech and generating for output the synthesized speech further comprises:
detecting that a different voice is being outputted in the media asset;
determining a second position in the media asset when the different voice ceases output; and
identifying the second position as the position in the media asset for outputting the synthesized speech.
7. The method of claim 1 , wherein determining the position in the media asset for outputting the synthesized speech and generating for output the synthesized speech further comprises:
determining the contextual features of the media asset, wherein the contextual features comprise silence periods, by:
retrieving metadata of the media asset; and
identifying, based on the metadata, a plurality of silence periods in the media asset, wherein a silence period of the plurality of silence periods is indicative of a time period in the media asset in which no voices are detected;
determining a candidate position in the media asset that is within the silence period;
identifying the candidate position as the position in the media asset for outputting the synthesized speech.
8. The method of claim 1 , wherein the contextual features comprise keywords and determining the position in the media asset for outputting the synthesized speech and generating for output the synthesized speech further comprises:
retrieving a keyword from memory;
retrieving metadata of the media asset;
identifying, based on the metadata, a time position in the media asset at which the keyword is recited;
identifying a silence period in the media asset that subsequently follows the time position at which the keyword is recited;
determining a candidate position in the media asset that is within the silence period;
identifying the candidate position as the position in the media asset for outputting the synthesized speech.
9. The method of claim 1 , wherein the determining the position in the media asset for outputting the synthesized speech and generating for output the synthesized speech further comprises:
pausing the media asset prior to generating for output the synthesized speech; and
unpausing the media asset in response to completing output of the synthesized speech.
10. A system comprising:
audio generating circuitry;
control circuitry configured to:
receive notification data during a display of a media asset by a media device, wherein the notification data is unrelated to the media asset;
in response to receiving the notification data during the display of the media asset on the media device:
determine that the media asset comprises a voice;
determine that the notification data comprises non-textual visual information;
convert the non-textual visual information to text;
convert the text to synthesized speech using a text-to-voice model generated based on characteristics of the voice;
generate, via the audio generating circuitry, for output by the media device, the synthesized speech by:
determining a position in the media asset for outputting the synthesized speech, based on one or more of contextual features of the media asset and the notification data; and
generating, via the audio generating circuitry, for output at the position in the media asset by the media device, the synthesized speech.
11. The system of claim 10 , wherein the control circuitry, when determining that the notification data comprises the non-textual visual information is further configured to:
determine that the notification data consists of a payload comprising an image.
12. The system of claim 11 , wherein the image is part of a social media post.
13. The system of claim 10 , wherein the non-textual visual information is an image, and wherein the control circuitry is further configured to convert the image to the text by:
using image processing and machine learning to identify objects in the image;
using machine learning to textually describe the identified objects in the image.
14. The system of claim 13 , wherein the image processing is edge detection and segmentation, and wherein the machine learning to identify the objects in the image comprises classification.
15. The system of claim 10 , wherein the control circuitry is further configured to determine the position in the media asset for outputting the synthesized speech and generate, via the audio generating circuitry, for output the synthesized speech by:
detecting that a different voice is being outputted in the media asset;
determining a second position in the media asset when the different voice ceases output; and
identifying the second position as the position in the media asset for outputting the synthesized speech.
16. The system of claim 10 , wherein the control circuitry is further configured to determine the position in the media asset for outputting the synthesized speech and generate, via the audio generating circuitry, for output the synthesized speech by:
determining the contextual features of the media asset, wherein the contextual features comprise silence periods, by:
retrieving metadata of the media asset; and
identifying, based on the metadata, a plurality of silence periods in the media asset, wherein a silence period of the plurality of silence periods is indicative of a time period in the media asset in which no voices are detected;
determining a candidate position in the media asset that is within the silence period;
identifying the candidate position as the position in the media asset for outputting the synthesized speech.
17. The system of claim 10 , wherein the contextual features comprise keywords and wherein the control circuitry is configured to determine the position in the media asset for outputting the synthesized speech and generate, via the audio generating circuitry, for output the synthesized speech by:
retrieving a keyword from memory;
retrieving metadata of the media asset;
identifying, based on the metadata, a time position in the media asset at which the keyword is recited;
identifying a silence period in the media asset that subsequently follows the time position at which the keyword is recited;
determining a candidate position in the media asset that is within the silence period;
identifying the candidate position as the position in the media asset for outputting the synthesized speech.
18. The system of claim 10 , wherein the control circuitry is configured to determine the position in the media asset for outputting the synthesized speech and generate, via the audio generating circuitry, for output the synthesized speech by:
pausing the media asset prior to generating for output the synthesized speech; and
unpausing the media asset in response to completing output of the synthesized speech.