System and method of controlling equipment based on data transferred in-band in video via optically encoded images
View Patent ↗Data is encoded into one or more optically encoded images. The optically encoded images are then inserted as image data into a video sequence—i.e., in video frames. Data are transmitted in-band within the video, via any conceivable video distribution channel or format. The video may be trans-coded as required—because the data are optically encoded, any video processing that even crudely preserves the frame images will preserve the optically encoded data. This scheme of in-band data transfer in video is very robust. A video receiving apparatus receives the video, inspects the image data from video frames in memory, detects optically encoded images in the image data, and decodes the optically encoded images to recover the data. The frames carrying optically encoded images are typically discarded and not rendered to a display. The receiver controls connected equipment, other than a display (e.g., a musical instrument), based on the extracted data.
1. An apparatus for controlling equipment, comprising:
a video receiver configured to non-optically receive a video sequence without the use of a camera or optical scanner;
an optically encoded image decoder configured to,
detect one or more images of optically encoded data in an image portion of one or more video frames; and
decode each detected image of optically encoded data to extract the data;
whereby the apparatus is configured to control equipment, other than a display, connected to the apparatus based on the extracted data.
2. The apparatus of claim 1 wherein the receiver is configured to receive a digital representation of the video sequence via a wired or wireless data connection.
3. The apparatus of claim 1 wherein the receiver is configured to receive an analog representation of the video sequence, and further comprising:
a digital video encoder configured to convert the analog representation of the video sequence to a digital representation of the video sequence.
4. The apparatus of claim 1 wherein the optically encoded image decoder is further configured to concatenate data extracted from a plurality of images of optically encoded data.
5. The apparatus of claim 1 further comprising a video player configured to render at least part of the received video sequence to a display.
6. The apparatus of claim 5 wherein the video player is configured to suppress, from the video sequence rendered to the display, frames containing an optically encoded image.
7. The apparatus of claim 6 wherein the video player is further configured to synchronize control of the connected equipment and the video sequence rendered to the display.
8. The apparatus of claim 7 wherein
the connected equipment is, or is configured to play, a musical instrument; and
the extracted data conforms to the Musical Instrument Digital Interface (MIDI) protocol.
9. The apparatus of claim 5 further comprising or connected to an asset management system, and wherein data extracted from the optically encoded images comprise metadata related to the video sequence in which they are transmitted, and wherein the apparatus is configured to output at least the metadata to the asset management system.
10. The apparatus of claim 1 wherein an image of optically encoded data comprises alteration of visual aspects of video content images in a plurality of video frames according to an encoded image pattern, wherein the alterations are imperceptible to a human viewing a rendering of the video on a display.
11. The apparatus of claim 1 wherein the equipment connected to the apparatus based comprises one of a game console and a computer, and wherein the optically encoded data comprises instructions for a game.
12. A method of controlling equipment, comprising:
non-optically receiving a video sequence without the use of a camera or optical scanner;
detecting one or more images of optically encoded data in an image portion of one or more video frames of the video sequence;
decoding each detected image of optically encoded data to extract the data; and
controlling equipment, other than a display, connected to the apparatus based on the extracted data.
13. The method of claim 12 wherein the video sequence is in an analog representation, and further comprising converting the analog representation of the video sequence to a digital representation of the video sequence.
14. The method of claim 12 wherein the connected equipment is, or is configured to play, a musical instrument, and wherein the extracted data conforms to the Musical Instrument Digital Interface (MIDI) protocol.
15. The method of claim 12 wherein the connected equipment comprises an asset management system, and wherein data extracted from the optically encoded images comprise metadata related to the video sequence in which they are transmitted, and further comprising outputting at least the metadata to the asset management system.