Automatic in-game subtitles and closed captions
An approach is provided for a gaming overlay application to provide automatic in-game subtitles and/or closed captions for video game applications. The overlay application accesses an audio stream and a video stream generated by an executing game application. The overlay application processes the audio stream through a text conversion engine to generate at least one subtitle. The overlay application determines a display position to associate with the at least one subtitle. The overlay application generates a subtitle overlay comprising the at least one subtitle located at the associated display position. The overlay application causes a portion of the video stream to be displayed with the subtitle overlay.
1. A method comprising:
accessing a multichannel or positional audio stream and a video stream generated by one or more executing applications;
generating, at a text conversion engine circuitry, at least one subtitle based on the multichannel or positional audio stream;
analyzing the multichannel or positional audio stream to identify a speaker associated with the at least one subtitle;
locating an object associated with the speaker by triangulation from the multichannel or positional audio stream;
determining a display position proximate to the object associated with the speaker;
generating a subtitle overlay comprising the at least one subtitle located at the display position; and
causing a portion of the video stream to be displayed with the subtitle overlay.
2. The method of claim 1 , further comprising loading the multichannel or positional audio stream into size limited buffers for real-time or near real-time processing.
3. The method of claim 1 , wherein determining the display position includes analyzing the video stream for exclusion areas that contain user interface elements of the one or more executing applications.
4. The method of claim 1 , wherein the speaker is an in-game speaker.
5. The method of claim 1 , wherein analyzing the multichannel or positional audio stream to identify the speaker includes matching at least one trait of the speaker to an associated classification in a voice profile database.
6. The method of claim 5 , wherein the at least one trait comprises age, gender, and dialect.
7. The method of claim 1 , wherein determining the display position includes processing the video stream with computer vision to identify the speaker.
8. The method of claim 1 , wherein the object is an in-game object.
9. The method of claim 1 , wherein generating the subtitle overlay includes configuring one or more visual characteristics of the at least one subtitle.
10. The method of claim 9 , wherein the one or more visual characteristics include at least one of: font attribute, font color, font size, and speech bubble type.
11. The method of claim 9 , wherein the one or more visual characteristics are determined based on at least one of: stored user preferences, readability when the video stream is displayed with the subtitle overlay, and speaker sentiment analyzed from the multichannel or positional audio stream.
12. The method of claim 1 , wherein the one or more executing applications is a multiplayer game, and wherein the multichannel or positional audio stream includes voice chat from participants in the multiplayer game.
13. The method of claim 1 , wherein determining the display position includes accessing stored user preferences for subtitle positioning.
14. A system comprising:
one or more processors configured to:
access a multichannel or positional audio stream and a video stream generated by one or more executing applications;
process the multichannel or positional audio stream through a text conversion engine to generate at least one subtitle;
analyze the multichannel or positional audio stream to identify a speaker associated with the at least one subtitle;
locate an object associated with the speaker by triangulation from the multichannel or positional audio stream;
determine a display position proximate to the object associated with the speaker;
generate a subtitle overlay comprising the at least one subtitle located at the display position; and
cause a portion of the video stream to be displayed with the subtitle overlay.
15. The system of claim 14 , wherein the one or more processors are configured to load the multichannel or positional audio stream into size limited buffers for real-time or near real-time processing.
16. The system of claim 14 , wherein the speaker is an in-game speaker.
17. The system of claim 14 , wherein the object is an in-game object.
18. One or more non-transitory computer readable media comprising instructions executable by one or more processors, which cause the one or more processors to:
access a multichannel or positional audio stream and a video stream generated by one or more executing applications;
process the multichannel or positional audio stream through a text conversion engine to generate at least one subtitle;
analyze the multichannel or positional audio stream to identify a speaker associated with the at least one subtitle;
locate an object associated with the speaker by triangulation from the multichannel or positional audio stream;
determine a display position proximate to the object associated with the speaker;
generate a subtitle overlay comprising the at least one subtitle located at the display position; and
cause a portion of the video stream to be displayed with the subtitle overlay.
19. The one or more non-transitory computer readable media of claim 18 , wherein the instructions, when executed by the one or more processors, further cause loading the multichannel or positional audio stream into size limited buffers for real-time or near real-time processing.
20. The one or more non-transitory computer readable media of claim 18 , wherein the speaker is an in-game speaker.