Automatic speech recognition for live video comments
In one embodiment, a system includes a microphone, memory, and a processor. The processor is operable to access environmental sounds captured by the microphone while the user is watching a live video. The processor is further operable to determine, by analyzing the captured sounds, a response by the user to the live video and generate a comment or a reaction element based on the determined response by the user to the live video. The processor is further operable to post the generated comment or reaction element for display along with the live video.
1. A method comprising:
by one or more computing devices, receiving an indication from a client device that a user has selected an option to view a live video;
by the one or more computing devices, providing the live video for display in an interface on the client device;
by the one or more computing devices, receiving captured sounds from the client device, the captured sounds comprising environmental sounds captured by a microphone of the client device while the user is watching the live video;
by the one or more computing devices, determining, by utilizing speech recognition on the captured sounds, a spoken comment of the user about the live video and another spoken comment of the user not relevant to the live video;
by the one or more computing devices, generating an electronic comment based on the determined spoken comment and filtering the determined another spoken comment so as to prevent generation of an electronic comment based on the determined another spoken comment of the user not relevant to the live video;
by the one or more computing devices, determining, from the captured sounds, a sentiment of the spoken comment of the user about the live video;
by the one or more computing devices, formatting the generated electronic comment to have a different visual appearance based on the determined sentiment; and
by the one or more computing devices, posting the formatted electronic comment for viewing in a comment area of the interface in which the live video is displayed.
2. The method of claim 1 , wherein formatting the generated electronic comment comprises changing one or more of:
a background color;
a font;
a size of text;
a text color;
a text emphasis comprising one or more of bold text, underlined text, and italic text; and
a text capitalization.
3. The method of claim 1 , further comprising:
by the one or more computing devices, determining, from the captured sounds, a reaction of the user to the live video; and
by the one or more computing devices, posting a reaction element to the live video based on the determined reaction of the user.
4. The method of claim 3 , wherein the reaction element comprises one of:
a reaction element that indicates the user likes the live video;
a reaction element that indicates the user loves the live video;
a reaction element that indicates the user thinks the live video is funny;
a reaction element that indicates the user is wowed by the live video;
a reaction element that indicates the user is saddened by the live video; and
a reaction element that indicates the user is angered by the live video.
5. The method of claim 1 , further comprising:
by the one or more computing devices, determining, from an image or video captured by the client device, a gesture of the user about the live video; and
by the one or more computing devices, posting a reaction element to the live video based on the determined gesture of the user, the reaction element comprising one of:
a reaction element that indicates the user likes the live video;
a reaction element that indicates the user loves the live video;
a reaction element that indicates the user thinks the live video is funny;
a reaction element that indicates the user is wowed by the live video;
a reaction element that indicates the user is saddened by the live video; and
a reaction element that indicates the user is angered by the live video.
6. The method of claim 1 , further comprising:
by the one or more computing devices, determining, from an image or video captured by the client device, a gesture of the user about the live video; and
by the one or more computing devices, formatting the generated electronic comment based on the determined gesture of the user, the formatting of the generated electronic comment comprising changing one or more of:
a background color;
a font;
a size of text;
a text color;
a text emphasis comprising one or more of bold text, underlined text, and italic text; and
a text capitalization.
7. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
access sounds captured by a client device of a user, the captured sounds comprising environmental sounds captured by a microphone of the client device while the user is watching a live video;
determine, by utilizing speech recognition on the captured sounds, a spoken comment of the user about the live video and another spoken comment of the user not relevant to the live video;
generate an electronic comment based on the determined spoken comment;
filter the determined another spoken comment so as to prevent generation of an electronic comment based on the determined another spoken comment of the user not relevant to the live video;
determine, from the captured sounds, a sentiment of the spoken comment of the user about the live video;
format the generated electronic comment to have a different visual appearance based on the determined sentiment; and
post the formatted electronic comment for viewing in a comment area of an interface in which the live video is displayed.
8. The media of claim 7 , wherein formatting the generated electronic comment comprises changing one or more of:
a background color;
a font;
a size of text;
a text color;
a text emphasis comprising one or more of bold text, underlined text, and italic text; and
a text capitalization.
9. The media of claim 7 , the software further operable when executed to:
determine, from the captured sounds, a reaction of the user to the live video; and
post a reaction element to the live video based on the determined reaction of the user.
10. The media of claim 9 , wherein the reaction element comprises one of:
a reaction element that indicates the user likes the live video;
a reaction element that indicates the user loves the live video;
a reaction element that indicates the user thinks the live video is funny;
a reaction element that indicates the user is wowed by the live video;
a reaction element that indicates the user is saddened by the live video; and
a reaction element that indicates the user is angered by the live video.
11. The media of claim 7 , the software further operable when executed to:
determine, from an image or video captured by the client device, a gesture of the user about the live video; and
post a reaction element to the live video based on the determined gesture of the user, the reaction element comprising one of:
a reaction element that indicates the user likes the live video;
a reaction element that indicates the user loves the live video;
a reaction element that indicates the user thinks the live video is funny;
a reaction element that indicates the user is wowed by the live video;
a reaction element that indicates the user is saddened by the live video; and
a reaction element that indicates the user is angered by the live video.
12. The media of claim 7 , the software further operable when executed to:
determine, from an image or video captured by the client device, a gesture of the user about the live video; and
format the generated electronic comment based on the determined gesture of the user, the formatting of the generated electronic comment comprising changing one or more of:
a background color;
a font;
a size of text;
a text color;
a text emphasis comprising one or more of bold text, underlined text, and italic text; and
a text capitalization.
13. A system comprising:
a microphone;
one or more processors; and
a memory coupled to the processors, the memory comprising instructions executable by the one or more processors, the one or more processors being operable when executing the instructions to:
access environmental sounds captured by the microphone while the user is watching a live video;
determine, by analyzing the captured sounds, a spoken comment of the user about the live video and another spoken comment of the user not relevant to the live video;
generate a comment or a reaction element based on the determined spoken comment of the user about the live video;
filter the determined another spoken comment so as to prevent generation of an electronic comment based on the determined another spoken comment of the user not relevant to the live video;
determine, from the captured sounds, a sentiment of the spoken comment of the user about the live video;
format the generated comment or reaction element to have a different visual appearance based on the determined sentiment; and
post the generated comment or reaction element for display along with the live video.
14. The system of claim 13 , wherein formatting the generated comment or reaction element comprises changing one or more of:
a background color of the generated comment;
a font of text within the generated comment;
a size of text within the generated comment;
a text color of the generated comment;
a text emphasis comprising one or more of bold text, underlined text, and italic text of the generated comment;
a text capitalization of the generated comment;
a size of the reaction element; and
a color or shading of the reaction element.
15. The system of claim 13 , wherein the reaction element comprises one of:
a reaction element that indicates the user likes the live video;
a reaction element that indicates the user loves the live video;
a reaction element that indicates the user thinks the live video is funny;
a reaction element that indicates the user is wowed by the live video;
a reaction element that indicates the user is saddened by the live video; and
a reaction element that indicates the user is angered by the live video.
16. The system of claim 13 , the one or more processors further operable when executing the instructions to:
determine, from an image or video captured by the client device, a gesture of the user about the live video, wherein the generated comment or reaction element is further based on the determined gesture.
17. The system of claim 16 , wherein the gesture of the user comprises one or more of:
a facial expression; and
a hand gesture.
18. The method of claim 1 ,
wherein generating the electronic comment based on the determined spoken comment and determining, from the captured sounds, the sentiment of the spoken comment of the user about the live video are performed in a predetermined order, and
wherein determining, from the captured sounds, the sentiment of the spoken comment of the user about the live video and formatting the generated electronic comment to have the different visual appearance based on the determined sentiment are performed in another predetermined order.
19. The media of claim 7 , the software further operable when executed to:
generate the electronic comment based on the determined spoken comment and determine from the captured sounds, the sentiment of the spoken comment of the user about the live video in a predetermined order; and
determine, from the captured sounds, the sentiment of the spoken comment of the user about the live video and format the generated electronic comment to have the different visual appearance based on the determined sentiment in another predetermined order.
20. The system of claim 13 , the one or more processors being operable when executing instructions to:
generate the comment based on the determined spoken comment and determine, from the captured sounds, the sentiment of the spoken comment of the user about the live video in a predetermined order; and
determine from the captured sounds, the sentiment of the spoken comment of the user about the live video and format the comment to have the different visual appearance based on the determined sentiment in another predetermined order.