IP Library Granted Patent US 7,886,151
Granted Patent B2
US 7,886,151 · App. 10/349,322 · Granted Feb 8, 2011

Temporal synchronization of video and audio signals

Assignee: Purdue Research Foundation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,886,151
App. No.
10/349,322
Granted
Feb 8, 2011
Kind
B2
Abstract

A protocol for temporal synchronization of media signals with temporal components is used for digital watermarking and other applications. The synchronization protocol achieves initial synchronization by finding an initial synchronization key through analysis of a temporal media signal stream. It then uses features of the stream and a queue of one or more keys from previous frames to derive subsequent keys to maintain synchronization. If synchronization is lost due to channel errors or attacks, for example, the protocol uses the initial synchronization key to re-establish synchronization. In digital watermarking applications, the synchronization protocol is agnostic to the watermark embedding and reading functions.

Claims (36)

1. A method of temporal synchronization of a media signal with a temporal component, the method comprising:

using a programmed computer to perform the acts of:

establishing keys relative to temporal locations in the media signal to provide temporal synchronization;

deriving perceptual features from perceptual image or audio data in temporal locations of the media signal relative to the locations of the keys;

based on the perceptual features, deriving changes to the keys to compute derived keys for other temporal locations in the media signal than the locations corresponding to the keys, wherein the keys and the derived keys represent a synchronization protocol for the media signal, and the derived keys include at least a first derived key that is a function of another key for a different temporal location of the temporal locations and a first perceptual feature of the perceptual features.

2. The method of claim 1 wherein the synchronization protocol is used to control digital watermark embedding of an auxiliary hidden signal in the media signal as a function of the keys and the derived keys.

3. The method of claim 1 wherein a state machine takes the first perceptual feature and one or more previous keys as input, and generates output for establishing a subsequent key.

4. The method of claim 3 wherein a synchronization state in the state machine is periodic over time in the media signal.

5. The method of claim 1 wherein the synchronization protocol is used to control digital watermark reading of an auxiliary hidden signal in the media signal as a function of the keys and the derived keys.

6. A non-transitory computer readable medium on which is stored instructions, which when executed by a computer, perform a method of temporal synchronization of a media signal with a temporal component, the method comprising:

establishing keys relative to temporal locations in the media signal to provide temporal synchronization;

deriving perceptual features from perceptual image or audio data in temporal locations of the media signal relative to the locations of the keys;

based on the perceptual features, deriving changes to the keys to compute derived keys for other temporal locations in the media signal than the locations corresponding to the keys, wherein the keys and the derived keys represent a synchronization protocol for the media signal, and the derived keys include a first derived key that is a function of another key for a different temporal location of the temporal locations and a first perceptual feature of the perceptual features.

7. A temporal synchronization method for digital watermarking comprising:

using a programmed computer to perform the acts of:

computing at least one perceptual feature from an image or audio signal representing perceptual content in the time varying portions of a host signal;

using a state machine to vary a watermarking key over time as a function of the at least one perceptual feature and a key for a different temporal location; and

using the watermarking key to generate a digital watermark signal for use in embedding a digital watermark into a host signal or detecting a digital watermark from a host signal; wherein the state machine varies the watermarking key dependent on the at least one perceptual feature of the host signal.

8. The method of claim 7 wherein the state machine includes randomization.

9. The method of claim 8 wherein the randomization is derived from a random function.

10. The method of claim 8 wherein the state machine includes a cryptographic hash function.

11. The method of claim 7 including:

deriving feature values from a host signal; and

using the features values as input to the state machine to generate the watermark key.

12. The method of claim 11 wherein the features are derived from a perceptual function.

13. The method of claim 12 wherein the perceptual function comprises a perceptual hash of the host signal.

14. A temporal synchronization method for digital watermarking comprising:

using a programmed computer to perform the acts of:

computing at least one perceptual feature from an image or audio signal representing perceptual content in the time varying portions of a host signal;

using a state machine to vary a watermarking key over time as a function of the at least one perceptual feature; and

using the watermarking key to generate a digital watermark signal for use in embedding a digital watermark into a host signal or detecting a digital watermark from a host signal; wherein the state machine varies the watermarking key dependent on the at least one perceptual feature of the host signal; wherein the state machine changes the watermarking key over time by making state transitions over time, and wherein the state transitions are dependent on variations in features of the host signal.

15. The method of claim 1 wherein the first perceptual feature is a perceptual hash of an image or audio signal in the media signal.

16. A non-transitory computer readable medium on which is stored instructions, which when executed by a computer, perform a temporal synchronization method for digital watermarking comprising:

computing at least one perceptual feature from an image or audio signal representing perceptual content in the time varying portions of a host signal;

using a state machine to vary a watermarking key over time as a function of the at least one perceptual feature and a key for a different temporal location; and

using the watermarking key to generate a digital watermark signal for use in embedding a digital watermark into a host signal or detecting a digital watermark from a host signal; wherein the state machine varies the watermarking key dependent on the at least one perceptual feature of the host signal.

Assignments (5)
MERGER Recorded May 12, 2010
From: DIGIMARC CORPORATION (A DELAWARE CORPORATION)
To: DIGIMARC CORPORATION (AN OREGON CORPORATION)
Reel/Frame 024369/0582 →
CONFIRMATION OF TRANSFER OF UNITED STATES PATENT RIGHTS Recorded Nov 5, 2008
From: L-1 SECURE CREDENTIALING, INC. (FORMERLY KNOWN AS DIGIMARC CORPORATION)
To: DIGIMARC CORPORATION (FORMERLY DMRC CORPORATION)
Reel/Frame 021785/0796 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2005
From: DIGIMARC CORPORATION
To: PURDUE RESEARCH FOUNDATION
Reel/Frame 016527/0192 →
CONFIRMATION OF EXCLUSIVE LICENSE Recorded May 6, 2005
From: PURDUE RESEARCH FOUNDATION
To: DIGIMARC CORPORATION
Reel/Frame 016527/0213 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2003
From: DELP, EDWARD L; LIN, EUGENE T.
To: DIGIMARC CORPORATION
Reel/Frame 014068/0742 →
Continuity (2)
Provisional Application 60351350 · Jan 22, 2002
Related Publication 20030221106A1 · Nov 27, 2003