Audio/video identification watermarking
View Patent ↗The invention is directed to marking audio/video (A/V) signals for use in tracing content to a source. An ID controller captures an A/V synchronization fingerprint from a master A/V signal, stores the fingerprint in a data store, and provides signals to an ID encoder to have encoded an identifier into a copy of the master A/V signal. The identifier is encoded to be persistent through actions such as capture, compression, and/or transcoding. In one embodiment, the identifier is encoded as a bit sequence, where each bit is encoded using an encoding technique that is hereinafter called a Mississippi encoding technique. The encoded identifier within the copied A/V signal may later be determined by extracting the fingerprint and comparing it to the stored fingerprint to determine the identifier. The identifier may then be used to trace a source of the copied A/V signal.
1. A system for identifying audio/video content using a computer, comprising:
a non-transitory computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform actions including:
receiving the audio/video content, the audio/video content including an audio component and a video component; and
determining a master audio/video synchronization fingerprint for the audio/video content, the master audio/video synchronization fingerprint identifying a plurality of variations in synchronization between the audio component and the video component;
determining an identifier, the identifier indicating a source within a distribution stream; and
modifying at least a portion of an audio/video synchronization fingerprint by adding a plurality of synchronization shifts to the audio/video content to encode the identifier into a copy of the audio/video content, each of the plurality of synchronization shifts including a change in the synchronization between the audio component and the video component, wherein the identifier is determinable by comparing the modified audio/video synchronization fingerprint with the master audio/video synchronization fingerprint; and
at least one processor.
2. The system of claim 1 , wherein encoding the identifier further comprises encoding a timestamp, and wherein the master audio/video synchronization fingerprint is determined based on an audio waveform from the audio/video content and a word-count waveform.
3. The system of claim 1 , wherein encoding the identifier further comprises encoding each bit within a bit sequence of the audio/video content using a Mississippi encoding scheme.
4. The system of claim 3 , wherein the Mississippi encoding scheme comprises:
encoding of a zero bit as a period of no synchronization change followed by a speeding up of an audio component within the audio/video content with respect to a video component for a time period, followed by a slowing down of the audio component with respect to the video component for a time period; and
encoding a one bit as a period of no synchronization change followed by a slowing down of the audio component with respect to the video component for a time period, followed by a speeding up of the audio component with respect to the video component for a time period.
5. The system of claim 1 , wherein determining an audio/video synchronization fingerprint comprises comparing a low pass filtered energy envelope of an audio component of the audio/video content with a word count from a video component of the audio/video content.
6. The system of claim 1 , wherein the actions further include:
determining if the copy of the audio/video content is authorized based in part on a license agreement;
if the copy is unauthorized, sending a notice of the unauthorized copying to an owner of the identification encoding component; and
if the copy is authorized, sending at least one of a reward, promotion, or discount to the owner of the identification encoding component.
7. A non-transitory computer readable medium that includes instructions, wherein execution of the instructions provides for identifying audio/video content by enabling actions, including:
receiving the audio/video content, the audio/video content including an audio component and a video component;
determining a master audio/video synchronization fingerprint for the audio/video content, the master audio/video synchronization fingerprint identifying a plurality of variations in synchronization between the audio component and the video component;
determining an identifier, the identifier indicating a source within a distribution stream; and
modifying at least a portion of an audio/video synchronization fingerprint by adding a plurality of synchronization shifts to the audio/video content to encode the identifier into a copy of the audio/video content, each of the plurality of synchronization shifts including a change in the synchronization between the audio component and the video component, wherein the identifier is determinable by comparing the modified audio/video synchronization fingerprint with the master audio/video synchronization fingerprint.
8. The non-transitory computer readable medium of claim 7 , wherein determining the identifier comprises:
determining a timestamp associated with a time associated with the encoding;
determining a local unique identifier configured to uniquely identify an encoding component; and
combining the local unique identifier with the timestamp to determine the identifier.
9. The non-transitory computer readable medium of claim 7 , wherein determining an audio/video synchronization fingerprint comprises:
comparing a low pass filtered energy envelope of an audio component synchronized with a word count of an associated video component.
10. The non-transitory computer readable medium of claim 7 , wherein determining the identifier comprises receiving from another computing device a base identifier useable in determining the identifier.
11. A network device for identifying audio/video content over a network, comprising:
a transceiver to send and receive data over the network; and
at least one processor that is operative to perform actions including:
receiving the audio/video content, the audio/video content including an audio component and a video component;
determining a master audio/video synchronization fingerprint for the audio/video content, the master audio/video synchronization fingerprint identifying a plurality of variations in synchronization between the audio component and the video component;
providing at least one of an instruction to determine an identifier, or providing the identifier over the network to another device, the identifier indicating a source within a distribution stream, wherein the other device is configured to modify at least a portion of an audio/video synchronization fingerprint by adding a plurality of synchronization shifts to the audio/video content to encode the unique identifier or a locally determined unique identifier, based on the instruction, into a copy of the audio/video content, each of the plurality of synchronization shifts including a change in the synchronization between the audio component and the video component, wherein the identifier is determinable by comparing the modified audio/video synchronization fingerprint with the master audio/video synchronization fingerprint.
12. The network device of claim 11 , wherein encoding the identifier comprises:
encoding into the audio/video content a zero bit as a period of no synchronization change followed by a speeding up of an audio component within the audio/video content with respect to a video component for a time period, followed by a slowing down of the audio component with respect to the video component for a time period; and
encoding into the audio/video content a one bit as a period of no synchronization change followed by a slowing down of the audio component with respect to the video component for a time period, followed by a speeding up of the audio component with respect to the video component for a time period, such that a DC component of the audio/video content is substantially unchanged.
13. The network device of claim 11 , wherein the actions further comprise:
receiving the copy of the audio/video content;
extracting a fingerprint from the copy of the audio/video content using a low pass filtered energy envelope of the audio component synchronized with a word count of the associated video component; and
extracting the identifier based on a comparison of the extracted fingerprint to a fingerprint of the received audio/video content.
14. The network device of claim 11 , wherein the plurality of synchronization shifts added to the audio/video content encode the identifier at a plurality of locations within the audio/video content.
15. A computing device that is operative to mark audio/video content for use in identifying the audio/video content, comprising:
a memory component for storing data; and
at least one processing component for executing instructions that enables actions including:
receiving the audio/video content, the audio/video content including an audio component and a video component; and
modifying at least a portion of an audio/video synchronization fingerprint of the audio/video content to encode an identifier that indicates a source within a distribution stream into a copy of the audio/video content by:
encoding into the audio/video content a zero bit as a period of no synchronization change between the audio component and the video component, followed by a speeding up of the audio component within the audio/video content with respect to the video component within the audio/video content for a time period, followed by a slowing down of the audio component with respect to the video component for a time period; and
encoding into the audio/video content a one bit as a period of no synchronization change between the audio component and the video component, followed by a slowing down of the audio component with respect to the video component for a time period, followed by a speeding up of the audio component with respect to the video component for a time period, wherein the identifier is determinable by comparing the modified audio/video synchronization fingerprint with a master audio/video synchronization fingerprint that corresponds to the received audio/video content, the master audio/video synchronization fingerprint identifying a plurality of variations in synchronization between the audio component and the video component.
16. The computing device of claim 15 , wherein the identifier further comprises a timestamp.
17. The computing device of claim 15 , wherein the encodings are configured to introduce substantially no change to an overall DC component of the audio/video content.
18. A method of using a computer to identify audio/video content over a network, comprising:
receiving the audio/video content at a first computer, the audio/video content including an audio component and a video component;
determining a master audio/video synchronization fingerprint for the audio/video content, the master audio/video synchronization fingerprint identifying a plurality of variations in synchronization between the audio component and the video component;
determining an identifier, the identifier indicating a source within a distribution stream;
modifying at least a portion of an audio/video synchronization fingerprint by adding a plurality of synchronization shifts to the audio/video content to encode the identifier into a copy of the audio/video content, each of the plurality of synchronization shifts including a change in the synchronization between the audio component and the video component, thereby creating a copy including of the audio/video content that includes the plurality of synchronization shifts;
extracting the identifier from the copy of the audio/video content based on a comparison of the modified audio/video synchronization fingerprint with the master audio/video synchronization fingerprint; and
if the identifier indicates the copied audio/video content is unauthorized, performing an action.
19. The method of claim 18 , wherein the action comprises at least one of sending a notice to an owner of a computer with which the copy was made, or inhibiting access to another audio/video content.
20. The method of claim 18 , further comprising:
if the identifier indicates that the copied audio/video content is authorized, sending to an owner of a computer with which the extracting was performed at least one of a discount, a reward, or a promotion.
21. A non-transitory computer-readable storage medium configured to include program instructions that when executed by a processor perform the method of claim 18 .