Efficient application of video marking technologies
View Patent ↗Systems and methods are described for rendering information to be embedded in media content at a first location and for embedding the rendered information into the media content at a second location. In many embodiments, the embedding process is less processor intensive than the rendering process and can be performed on a consumer electronics device such as a set top box, using existing processing mechanisms. One embodiment of the invention includes rendering the information into an image at a first location and embedding the image in the media at a second location in order to achieve efficient marking of the media content.
1. A method of embedding information in media content during a process for decoding the media content, comprising:
delivering media content to a decode device configured to decode the media content and output frames of video for display on an output device, where the decode device includes an alpha blending function configured to combine an on screen display and media content during the decoding of the media content to produce a combined frame in which both the on screen display and media content are present and the decode device is configured to output the combined frame for display on the output device;
decoding a frame of the media content;
rendering the information as an overlay image and alpha values according to perceptual characteristics of the media content so that the modifications to the decoded frame are imperceptible when the overlay image is combined with the decoded frame of the media content in accordance with the alpha values using the alpha blending function of the decode device;
at the point during the process of decoding the media content at which an on screen display, when present, is overlaid on the decoded frame using the alpha blending function of the decode device, modifying the decoded frame by using the alpha blending function of the decode device to combine the entire rendered overlay image with the decoded frame of the media content in accordance with the alpha values, and thereby imperceptibly producing a marked frame in which the rendered overlay image is present and imperceptible; and
outputting the marked frame from the decode device to the output device for display.
2. The method of claim 1 wherein the information embedded in media content is transformed from a frequency domain to render the image.
3. The method of claim 1 wherein:
the media content is delivered as a compressed file; and
the perceptual characteristics of the media content are determined from the compressed electronic file.
4. The method of claim 1 wherein the information comprises metadata related to the media content.
5. The method of claim 1 wherein the information identifies a time and location of media content playback.
6. The method of claim 1 wherein the information identifies a copyright owner or recipient of the media content.
7. The method of claim 1 wherein the information represents a database index.
8. The method of claim 1 wherein the rendered image is stored for repeated or subsequent use.
9. A device for decoding media content and embedding information in the media content during decoding, comprising:
decode circuitry configured to decode the media content for display on a display device;
alpha blending circuitry configured to overlay an on screen display onto media content during the decoding of the media content to produce a combined frame in which both the on screen display and media content are present for display on an output device;
rendering circuitry configured to render the information as an overlay image and alpha values according to perceptual characteristics of the media content so that the modifications to the decoded frame are imperceptible when the overlay image is combined with the decoded frame of the media content in accordance with the alpha values using the alpha blending function of the decode device; and
wherein the device is configured to imperceptibly embed the information in the media content at the point during the process of decoding the media content at which an on screen display, when present, is overlaid on the decoded frame using the alpha blending function of the decode device, the embedding being achieved by overlaying the entire overlay image onto the decoded frame using the alpha blending circuitry in accordance with the alpha values, thereby imperceptibly producing a marked frame in which the rendered overlay image is present and imperceptible.
10. The device of claim 9 , wherein the device is a digital set top box.
11. The device of claim 9 , wherein the device is configured to transform the information that is to be embedded in the media content from the frequency domain to create the rendered image.
12. The device of claim 9 , wherein
the device is configured to determine the perceptual characteristics of the media content.
13. The device of claim 12 , wherein:
the media content is stored as a compressed electronic file;
the device is configured to determine the perceptual characteristics of the media from the compressed electronic file.
14. The method of claim 1 , wherein the device renders the image from the information.
15. The method of claim 1 , wherein the device determines the perceptual characteristics of the media content.
16. The method of claim 1 , further comprising determining the perceptual characteristics of at least one frame of the video sequence and rendered the overlay image and alpha values in accordance with the perceptual characteristics of the at least one frame of the video sequence.
17. The method of claim 1 , wherein the media content is delivered over a network.
18. The method of claim 1 , wherein the media content is delivered via a fixed media.
19. The device of claim 9 , further comprising a microprocessor connected to a memory, wherein the microprocessor is configured to render the information as an overlay image and store the overlay image in memory.
20. The device of claim 19 , wherein the microprocessor and memory is the circuitry configured to decode the media content for display on a display device and the circuitry configured to overlay an on screen display onto decoded media content for display on an output device.
21. The method of claim 1 , wherein the alpha blending function is implemented as follows:
S x,y,f =I x,y,f * α x,y +O x,y,f * (1−α x,y )
where S x,y,f is the pixel at position x,y in frame number f of the marked video;
I x,y,f is the pixel at position x,y of the overlay image;
O x,y,f is the pixel at position x,y in frame number f of the original video; and
α x,y is the alpha value indicating the strength of the overlay image at position x,y.
22. The system of claim 9 , wherein the alpha blending circuitry is configured to perform the following operation:
S x,y,f =I x,y,f * α x,y +O x,y,f * (1 −α x,y )
where S x,y,f is the pixel at position x,y in frame number f of the marked video;
I x,y,f is the pixel at position x,y of the overlay image;
O x,y,f is the pixel at position x,y in frame number f of the original video; and
α x,y is the alpha value indicating the strength of the overlay image at position x,y.
23. The method of claim 1 , wherein the overlay image and the alpha values are generated by the decode device.
24. The method of claim 1 , wherein the overlay image and the alpha values are received by the decode device.
25. A method of embedding information in media content during a process for decoding the media content, comprising:
delivering media content to a decode device configured to decode the media content and output frames of video for display on an output device, where the decode device includes an alpha blending function configured to combine an on screen display and media content during the decoding of the media content to produce a combined frame in which both the on screen display and media content are present and the decode device is configured to output the combined frame for display on the output device;
decoding a frame of the media content;
rendering the information as an overlay image frame and alpha values so that the modifications to the decoded frame are imperceptible when the overlay image frame is combined with the decoded frame of the media content in accordance with the alpha values using the alpha blending function of the decode device;
at the point during the process of decoding the media content at which an on screen display, when present, is overlaid on the decoded frame using the alpha blending function of the decode device, modifying the decoded frame by using the alpha blending function of the decode device to combine the entire rendered overlay image frame with the decoded frame of the media content in accordance with the alpha values, and thereby imperceptibly producing a marked frame in which the rendered overlay image frame is present and imperceptible; and
outputting the marked frame from the decode device to the output device for display.
26. The method of claim 25 wherein the information embedded in media content is transformed from a frequency domain to render the image.
27. The method of claim 25 wherein the information comprises metadata related to the media content.
28. The method of claim 25 wherein the rendered overlay image frame is stored for repeated or subsequent use.
29. The method of claim 25 , wherein the device renders the image from the information.
30. The method of claim 25 , wherein the media content is delivered over a network.
31. The method of claim 25 , wherein the alpha blending function is implemented as follows:
S x,y,f =I x,y,f * α x,y +O x,y,f * (1−α x,y )
where S x,y,f is the pixel at position x,y in frame number f of the marked video;
I x,y,f is the pixel at position x,y of the overlay image;
O x,y,f is the pixel at position x,y in frame number f of the original video; and
α x,y is the alpha value indicating the strength of the overlay image at position x,y.
32. The method of claim 25 , wherein the overlay image and the alpha values are generated by the decode device.
33. A device for decoding media content and embedding information in the media content during decoding, comprising:
decode circuitry configured to decode the media content for display on a display device;
alpha blending circuitry configured to overlay an on screen display onto media content during the decoding of the media content to produce a combined frame in which both the on screen display and media content are present for display on an output device;
rendering circuitry configured to render the information as an overlay image frame and alpha values so that the modifications to the decoded frame are imperceptible when the overlay image frame is combined with the decoded frame of the media content in accordance with the alpha values using the alpha blending function of the decode device; and
wherein the device is configured to imperceptibly embed the information in the media content at the point during the process of decoding the media content at which an on screen display, when present, is overlaid on the decoded frame using the alpha blending function of the decode device, the embedding being achieved by overlaying the entire overlay image frame onto the decoded frame using the alpha blending circuitry in accordance with the alpha values, thereby imperceptibly producing a marked frame in which the rendered overlay image frame is present and imperceptible.
34. The device of claim 33 , wherein the device is a digital set top box.
35. The device of claim 33 , wherein the device is configured to transform the information that is to be embedded in the media content from the frequency domain to create the rendered overlay image frame.
36. The device of claim 33 , wherein:
the device is configured to determine the perceptual characteristics of the media content; and
the device is configured to modify the overlay image frame generated from the information according to the perceptual characteristics of the media content.
37. The device of claim 36 , wherein:
the media content is stored as a compressed electronic file; and
the device is configured to determine the perceptual characteristics of the media from the compressed electronic file.
38. The device of claim 33 , further comprising a microprocessor connected to a memory, wherein the microprocessor is configured to render the information as an overlay image frame and store the overlay image frame in memory.
39. The device of claim 38 , wherein the microprocessor and memory is the circuitry configured to decode the media content for display on a display device and the circuitry configured to overlay an on screen display onto decoded media content for display on an output device.