Methods and systems for providing audio information based on gaze information are described. In an example, a computing device may be configured to receive gaze information indicative of a wearer-view associated with the computing device. Based on the gaze information, the computing device may be configured to determine content displayed in the wearer-view. Further, the computing device may be configured to determine audio information relating to the content and cause an audio stream associated with the audio information to be played at the computing device.
1. A method, comprising:
receiving, at a wearable computing device, data comprising: (i) image data indicative of a field of view associated with the wearable computing device, and (ii) view data indicating a view direction in the field of view;
detecting, at a location in the image data that corresponds to the view direction, a physical advertisement space;
determining, based on the image data, that video from a media advertisement is playing in the physical advertisement space, wherein the media advertisement comprises the video and corresponding audio;
in response to determining that the video from the media advertisement is playing in the physical advertisement space, determining a portion of the audio that corresponds to the video from the media advertisement that is being played in the physical advertisement space; and
causing the determined audio portion to be played at the wearable computing device such that the determined audio portion is synchronized to the video of the media advertisement.
2. The method of claim 1 , further comprising:
determining, based on the view data, an amount of time during which the location corresponds to the view direction, wherein causing the determined audio portion to be played at the wearable computing device comprises causing the determined audio portion to be played at the wearable computing device in response to determining that the amount of time exceeds a predetermined threshold time period.
3. The method of claim 1 , further comprising:
determining, based on the view data, a change in the view direction, wherein the change is indicative that the video that is playing in the physical advertisement space is disregarded;
causing the determined audio portion to stop playing; and
causing the determined audio portion to resume playing in response to determining that view direction has changed back to correspond to the physical advertisement space at which the video from the media advertisement is playing.
4. The method of claim 1 , wherein determining the portion of the audio comprises:
generating a search query based on a portion of the image data that corresponds to the view direction;
providing the search query to a server; and
receiving an identification of the media advertisement.
5. The method of claim 4 , wherein generating the search query is further based on an amount of time during which the location corresponds to the view direction.
6. The method of claim 1 , wherein the image data comprises one or more images of the video from the media advertisement, wherein the one or more images are captured from a point-of-view (POV) of the wearable computing device, and wherein determining the portion of the audio comprises identifying the video based on the one or more images.
7. The method of claim 1 , wherein the media advertisement comprises multiple video items that are playing in the physical advertisement space, the method further comprising:
identifying, based on the data, a respective video item of the multiple video items; and
causing a respective audio portion relating to the respective video item to be played at the wearable computing device.
8. The method of claim 7 , further comprising:
determining, based on the view data, a change in the view direction, wherein the change is indicative that the respective video item is disregarded;
causing the respective audio portion to stop playing;
identifying, based on the view data, a given video item gazed at in the changed view direction; and
causing a given audio portion relating to the given video item to be played at the wearable computing device.
9. A non-transitory computer readable medium having stored thereon instructions that, when executed a wearable computing device, cause the wearable computing device to perform functions comprising:
receiving data comprising: (i) image data indicative of a field of view associated with the wearable computing device, and (ii) view data indicating a view direction in the field of view;
detecting, at a location in the image data that corresponds to the view direction, a physical advertisement space;
determining, based on the image data, that video from a media advertisement is playing in the physical advertisement space, wherein the media advertisement comprises the video and corresponding audio;
in response to determining that the video from the media advertisement is playing in the physical advertisement space, determining a portion of the audio that corresponds to the video from the media advertisement that is being played in the physical advertisement space; and
causing the determined audio portion to be played at the wearable computing device such that the determined audio portion is synchronized to the video of the media advertisement.
10. The non-transitory computer readable medium of claim 9 , wherein the functions further comprise:
determining, based on the view data, a focus metric that is based on an amount of time during which the location corresponds to the view direction, wherein the function of causing the determined audio portion to be played comprises causing the determined audio portion to be played based on a comparison of the focus metric to a predetermined focus threshold.
11. The non-transitory computer readable medium of claim 9 , wherein the functions further comprise:
determining, based on the view data, a change in the view direction, wherein the change is indicative that the video that is playing in the physical advertisement space is disregarded;
causing the determined audio portion to stop playing; and
causing the determined audio portion to resume playing in response to determining that view direction has changed back to correspond to the physical advertisement space at which the media advertisement is playing.
12. The non-transitory computer readable medium of claim 9 , wherein the function of determining the portion of the audio comprises:
generating a search query based on (i) a portion of the image data that corresponds to the view direction, and (ii) an amount of time during which the location corresponds to the view direction;
providing the search query to a server; and
receiving an identification of the media advertisement.
13. The non-transitory computer readable medium of claim 9 , wherein the media advertisement comprises multiple video items that are playing in the physical advertisement space, and wherein the functions further comprise:
identifying, based on the data, a respective video item of the multiple video items; and
causing a respective audio portion relating to the respective video item to be played at the computing device.
14. The non-transitory computer readable medium of claim 13 , wherein the functions further comprise:
determining, based on the view data, a change in the view direction, wherein the change is indicative that the respective video item is disregarded;
causing the respective audio portion to stop playing;
identifying, based on the gaze information, a given video item gazed at in the changed view direction; and
causing a given audio portion relating to the given video item to be played at the computing device.
15. A system, comprising:
a head-mountable display (HMD);
an eye-sensing device coupled to the HMD;
an image-capture device coupled to the HMD, wherein the image-capture device is configured to capture a field of view of the HMD;
at least one processor in communication with the HMD, the eye-sensing device, and the image-capture device; and
data storage comprising instructions that, when executed by the at least one processor, cause the system to perform functions comprising:
receiving, from the image-capture device, image data indicative of the field of view of the HMD,
receiving, from the eye-sensing device, view data indicating a view direction in the field of view;
detecting, at a location in the image data that corresponds to the view direction, a physical advertisement space;
determining that video from a media advertisement is playing in the physical advertisement space, wherein the media advertisement comprises the video and corresponding audio;
in response to determining that the video from the media advertisement is playing in the physical advertisement space, determining a portion of the audio that corresponds to the video from the media advertisement that is being played in the physical advertisement space; and
causing the determined audio portion to be played at the HMD such that the determined audio portion is synchronized to the video of the media advertisement.
16. The system of claim 15 , wherein the functions further comprise:
determining, based on the view data, an amount of time during which the location corresponds to the view direction, wherein the function of causing the determined audio portion to be played at the HMD comprises causing the determined audio portion to be played at the HMD in response to determining that the amount of time exceeds a predetermined threshold time period.
17. The system of claim 15 , wherein the functions further comprise:
determining, based on the view data, a change in the view direction, wherein the change is indicative that the video that is playing in the physical advertisement space is disregarded;
causing the determined audio portion to stop playing; and
causing the determined audio portion to resume playing in response to determining that view direction has changed back to correspond to the physical advertisement space at which the media advertisement is playing.
18. The system of claim 15 , wherein the function of determining the portion of the audio comprises:
generating a search query based on (i) a portion of the image data that corresponds to the view direction, and (ii) an amount of time during which the location corresponds to the view direction;
providing the search query to a server; and
receiving an identification of the media advertisement.
19. The system of claim 15 , wherein the image data comprises one or more images of the video, wherein the one or more images are captured from a point-of-view of the HMD, and wherein the function of determining the portion of the audio comprises identifying the video based on the one or more images.
20. The system of claim 15 , wherein the media advertisement comprises multiple video items that are playing in the physical advertisement space, and wherein the functions further comprise:
identifying a first video item of the multiple video items;
causing a respective audio portion relating to the first video item to be played;
determining, based on the view data, a change in the view direction, wherein the change is indicative that the first video item is disregarded;
causing the respective audio portion to stop playing;
identifying a second video item gazed at in the changed view direction; and
causing a respective audio portion relating to the second video item to be played.