Overlay non-video content on a mobile device
Methods, systems, and devices are described for presenting non-video content through a mobile device that uses a video camera to track a video on another screen. In one embodiment, a system includes a video display, such as a TV, that displays video content. A mobile device with an integrated video camera captures video data from the TV and allows a user to select an area in the video in order to hear/feel/smell what is at that location in the video.
1. A system for augmenting a video, the system comprising:
a video source configured to provide video content to a video display; and
a mobile device with a video camera and display, the mobile device configured to:
capture video data including the video content from the video display using the video camera, the video content from the video display having at least one marker identifying a temporal position within the video content, the temporal position corresponding to at least a portion of time in the video content;
track the video content in the captured video data;
receive a user selection of a subportion of an image in the captured video data;
access non-video content associated with the subportion of the image and responsive to the temporal position of the video content, the temporal position identified by the at least one marker; and
present the accessed non-video content associated with the subportion of the image to a user at substantially the same time as the video content is captured using the video camera.
2. The system of claim 1 , wherein the video camera is integrated with the mobile device.
3. The system of claim 1 , wherein the video source comprises a device selected from the group consisting of a video disc player and a video game console.
4. The system of claim 1 , wherein a physical marker off the video display includes a marker formatted to allow the mobile device to track the video content in the captured video data.
5. The system of claim 1 , wherein the non-video content is audio content associated with the selected subportion of the image in the captured video data.
6. The system of claim 1 , wherein the non-video content is tactile content associated with the selected subportion of the image in the captured video data.
7. The system of claim 1 , wherein the non-video content is olfactory content associated with the selected subportion of the image in the captured video data.
8. The system of claim 7 , wherein presenting the mobile device is configured to present the olfactory content using chemical synthesis.
9. The system of claim 7 , wherein presenting the mobile device is configured to present the olfactory content using essential oils.
10. The system of claim 1 , wherein the mobile device is configured to track the video content by identifying a position and orientation of the video content relative to the mobile device.
11. The system of claim 1 , further comprising:
a non-video content datastore, remote from the video display and mobile device and in communication with the mobile device, configured to store the non-video content, wherein the mobile device accesses the non-video content from the content datastore.
12. The system of claim 1 , wherein the video source is configured to store the non-video content, and the mobile device accesses the non-video content from the video source.
13. The system of claim 1 , wherein, the at least one marker is a pattern of dots.
14. A computer-implemented method for augmenting video, the method comprising:
capturing video data including video content from a video display, the video data captured with a video camera operatively coupled to a mobile device, the video content from the video display having at least one marker identifying a temporal position within the video content, the temporal position corresponding to at least a portion of time in the video content;
tracking the video content in the captured video data;
receiving a user selection of a subportion of an image in the captured video data;
accessing non-video content associated with the subportion of the image and responsive to the temporal position of the video content, the temporal position identified by the at least one marker; and
presenting the accessed non-video content associated with the subportion of the image to a user at substantially the same time as the video content is captured with the video camera.
15. The computer-implemented method of claim 14 , wherein,
the video content displayed on the video display includes a marker formatted to allow the video content to be tracked and the video content to be identified;
the marker changes over time to allow a temporal position in the identified video content to be identified; and
the non-video content includes at least one of audio, tactile, or olfactory information.
16. The computer-implemented method of claim 14 , wherein, the at least one marker is an invisible watermark.
17. The computer-implemented method of claim 16 , wherein, the invisible watermark is in select, predictable areas of the screen.
18. The computer-implemented method of claim 14 , wherein, the at least one marker is embedded in subtitles of the video.
19. A computer-implemented method for augmenting video, the method comprising:
receiving video data captured with a video camera operatively coupled with a mobile device, the video data including video content captured from a video display;
identifying the video content captured from the video display;
identifying a temporal position within the video content, the temporal position corresponding to at least a portion of time in the video content; and
accessing non-video content to be presented to a user substantially at the same time as the video content, the non-video content selected based at least in part on the identified video content.
20. The computer-implemented method of claim 19 , further comprising:
tracking the video content in the captured video data, wherein the tracking comprises identifying a relative position and orientation of the video content.