Overlay non-video content on a mobile device
Methods, systems, and devices are described for presenting non-video content through a mobile device that uses a video camera to track a video on another screen. In one embodiment, a system includes a video display, such as a TV, that displays video content. A mobile device with an integrated video camera captures video data from the TV and allows a user to select an area in the video in order to hear/feel/smell what is at that location in the video.
1. A mobile device comprising:
a video camera;
a display;
a processor; and
a memory communicatively coupled with the processor and storing computer-readable instructions that, upon execution by the processor, cause the mobile device to:
capture, by at least using the video camera, video content displayed on a video display, the video content having a marker that includes time code, the time code identifying a temporal position corresponding to a portion of time in the video content;
track the video content based on the marker;
receive a user selection of a subportion of an image in the video content;
access non-video content associated with the subportion of the image and synchronized with the temporal position of the video content using the time code; and
present, on the display, the non-video content associated with the subportion of the image at substantially the same time as the video content is captured using the video camera.
2. The mobile device of claim 1 , wherein the time code of the marker is presented with a pattern in the video content, wherein the pattern changes over time, wherein a presented change to the pattern corresponds to the time code, identifies a progression of the video content based on the time code, and does not identify the non-video content, and wherein the progression indicates the temporal position of the video content.
3. The mobile device of claim 1 , wherein the video content is provided from a video game console to the video display, and wherein the computer-readable instructions further cause the mobile device to transmit data associated with the non-video content to a game controller communicatively coupled with the video game console.
4. The mobile device of claim 1 , wherein tracking the video content comprises determining, based on the marker, a position and an orientation of the video content relative to the mobile device and a size of the video content.
5. The mobile device of claim 4 , wherein the non-video content is accessed based on at least the position or orientation of the video content.
6. The mobile device of claim 1 , wherein the marker comprises an on-screen marker that is presented at the video display, and wherein the presented on-screen marker comprises a binary pattern that identifies the video content and the time code.
7. The mobile device of claim 1 , wherein the computer-readable instructions further cause the mobile device to store a low-resolution version of the video content, and wherein the presentation of the non-video content is time synchronized with the presentation of the video content by at least performing image matching based on a hash of the low-resolution version of the video content.
8. The mobile device of claim 1 , wherein the marker is presented the video display at a frequency not perceptible to the human eye, and wherein the computer-readable instructions further cause the mobile device to identify the marker based on an application of a Fourier transform to the video content.
9. The mobile device of claim 1 , wherein the marker further includes an audio marker, and wherein the temporal position is identified based on the audio marker.
10. The mobile device of claim 1 , wherein the video content is provided from a video game console to the video display, and wherein accessing the non-video content comprises receiving the non-video content from the video game console based on the marker.
11. The mobile device of claim 1 , wherein non-video content is pre-loaded onto the mobile device.
12. The mobile device of claim 1 , wherein the non-video content is accessed and presented in response to the user selection of the subportion of the image.
13. A computer-implemented method comprising:
capturing, by a video camera of a mobile device, video content displayed on a video display, the video content having a marker that includes time code, the time code identifying a temporal position corresponding to a portion of time in the video content;
tracking, by the mobile device, the video content based on the marker;
receiving, by the mobile device, a user selection of a subportion of an image in the video content;
accessing, by the mobile device, non-video content associated with the subportion of the image and synchronized with the temporal position of the video content using the time code; and
presenting, by the mobile device on a display of the mobile device, the non-video content associated with the subportion of the image at substantially the same time as the video content is captured using the video camera.
14. The computer-implemented method of claim 13 , wherein the time code of the marker is presented in the video content with a pattern that changes over time, wherein a presented change to the pattern corresponds to the time code, identifies a progression of the video content based on the time code, and does not identify the non-video content, and wherein the progression indicates the temporal position of the video content.
15. The computer-implemented method of claim 13 , wherein the marker includes one or more markers each having a different value.
16. The computer-implemented method of claim 13 , wherein the non-video content includes at least one of text, animations, graphic enhancements, advertisements, and hyperlinks.
17. A non-transitory computer-readable storage medium comprising instructions that, upon execution on a mobile device, cause the mobile device to perform operations comprising:
capturing, by a video camera of the mobile device, video content displayed on a video display, the video content having a marker that includes time code, the time code identifying a temporal position corresponding to a portion of time in the video content;
tracking the video content based on the marker;
receiving a user selection of a subportion of an image in the video content;
accessing non-video content associated with the subportion of the image and synchronized with the temporal position of the video content using the time code; and
presenting, on a display of the mobile device, the non-video content associated with the subportion of the image at substantially the same time as the video content is captured using the video camera.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the time code of the marker is presented in the video content with a pattern that changes over time, wherein a presented change to the pattern corresponds to the time code, identifies a progression of the video content based on the time code, and does not identify the non-video content, and wherein the progression indicates the temporal position of the video content.
19. The non-transitory computer-readable storage medium of claim 17 , wherein the marker comprises an on-screen marker that is presented at the video display, and wherein the presented on-screen marker comprises a binary pattern that identifies the video content and the time code.
20. The non-transitory computer-readable storage medium of claim 17 , wherein the marker further includes an audio marker, and wherein the temporal position is identified based on the audio marker.