IP Library Granted Patent US 8,660,267
Granted Patent B2
US 8,660,267 · App. 13/220,267 · Granted Feb 25, 2014

Method and device for generating and detecting a fingerprint functioning as a trigger marker in a multimedia signal

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,660,267
App. No.
13/220,267
Granted
Feb 25, 2014
Kind
B2
Abstract

A device is configured to perform a method that detects a trigger marker for an action corresponding to a segment of a multimedia signal. A fingerprint is generated based on the segment of the multimedia signal at a trigger time point. The generated fingerprint is stored in a database and communicated to the device. During playback of the multimedia signal, fingerprints of segments of the multimedia signal are generated and matched against fingerprints in the database. When a match is found, one or more associated actions for the segment are retrieved by the device. The trigger time point may be determined as a time point near or at the segment of the multimedia signal with the matched fingerprint. In this way, trigger markers for actions may be enabled without modifying the multimedia signal.

Claims (112)

1. A method comprising:

generating a set of fingerprints based on a multimedia signal,

the set of fingerprints including a first fingerprint among the set of fingerprints,

the first fingerprint being generated based on content of a segment of the multimedia signal among a plurality of segments of the multimedia signal;

determining that the first fingerprint among the set of fingerprints matches a second fingerprint stored in a first database,

a second database storing a representation of an action as being associated with the second fingerprint and as corresponding to a trigger time point that is at least one of a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

the determining being performed by a processor of a machine based on a comparison of the first fingerprint among the set of fingerprints with the second fingerprint stored in the first database; and

retrieving the representation of the action stored as being associated with the second fingerprint and as corresponding to the trigger time point.

2. The method of claim 1 , wherein:

the second fingerprint is generated based on the content of the segment of the multimedia signal.

3. The method of claim 1 , wherein:

the set of fingerprints includes fingerprints of variable size.

4. The method of claim 1 , wherein:

the set of fingerprints includes fingerprints of fixed size.

5. The method of claim 1 , wherein:

the plurality of segments of the multimedia signal are of variable size.

6. The method of claim 1 , wherein:

the plurality of segments of the multimedia signal are of fixed size.

7. The method of claim 1 further comprising:

executing the action at the corresponding trigger time point based on the retrieved representation of the action.

8. The method of claim 7 further comprising:

receiving a user input while waiting a period of time after the executing of the action at the corresponding trigger time point.

9. The method of claim 1 further comprising:

determining that the action did not execute at the corresponding trigger time point; and

nullifying a user input in response to the determining that the action did not execute at the corresponding trigger time point.

10. The method of claim 1 further comprising:

accessing the first database that stores the second fingerprint; and wherein

the determining that the first fingerprint matches the second fingerprint is responsive to

the accessing of the first database.

11. The method of claim 10 , wherein:

the first and second databases are provided by different parties.

12. The method of claim 10 , wherein:

the accessing of the first database includes receiving the first database at a device configured to generate the set of fingerprints based on the multimedia signal.

13. The method of claim 1 further comprising:

accessing the second database that stores the representation of the action; and wherein

the retrieving of the representation of the action is from the second database.

14. The method of claim 13 , wherein:

the first and second databases are provided by different parties.

15. The method of claim 1 , wherein:

the accessing of the second database includes receiving the second database at a device configured to generate the set of fingerprints based on the multimedia signal.

16. The method of claim 1 , wherein:

the trigger time point is defined with respect to the content of the segment of the multimedia signal.

17. The method of claim 16 , wherein:

at least one of the start of the segment or the end of the segment is defined with respect to the content of the segment of the multimedia signal.

18. The method of claim 1 , wherein:

the multimedia signal includes at least one of an audio signal or a video signal.

19. The method of claim 1 , wherein:

the action includes at least one of:

retrieving and displaying additional information on a display,

retrieving and playing additional information via a speaker,

presenting biography data for an actor appearing in the content of the segment of the multimedia signal,

presenting a link to a website,

playing a local weather forecasts,

playing a local commercial,

interrupting a playback of the multimedia signal,

playing another multimedia signal instead of the multimedia signal for a period of time,

executing a control command, and

preparing a device to receive a user input.

20. The method of claim 1 , wherein:

the generating of the set of fingerprints includes generating a continuous stream of fingerprints that includes the set of fingerprints.

21. The method of claim 1 , wherein:

the generating of the set of fingerprints includes generating segments of a stream of fingerprints.

22. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

generating a set of fingerprints based on a multimedia signal,

the set of fingerprints including a first fingerprint among the set of fingerprints,

the first fingerprint being generated based on content of a segment of the multimedia signal among a plurality of segments of the multimedia signal;

determining that the first fingerprint among the set of fingerprints matches a second fingerprint stored in a first database,

a second database storing a representation of an action as being associated with the second fingerprint and as corresponding to a trigger time point that is at least one of a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

the determining being performed by the one or more processors of the machine based on a comparison of the first fingerprint among the set of fingerprints with the second fingerprint stored in the first database; and

retrieving the representation of the action stored as being associated with the second fingerprint and as corresponding to the trigger time point.

23. The non-transitory machine-readable storage medium of claim 22 , wherein the operations further comprise:

executing the action at the corresponding trigger time point based on the retrieved representation of the action.

24. A system comprising:

a processor configured by a fingerprint detector that configures the processor to generate a set of fingerprints based on a multimedia signal,

the set of fingerprints including a first fingerprint among the set of fingerprints,

the first fingerprint being generated based on content of a segment of the multimedia signal among a plurality of segments of the multimedia signal;

the fingerprint detector further configuring the processor to determine that the first fingerprint among the set of fingerprints matches a second fingerprint stored in a first database,

a second database storing a representation of an action as being associated with the second fingerprint and as corresponding to a trigger time point that is at least one of a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

the determining being performed based on a comparison of the first fingerprint among the set of fingerprints with the second fingerprint stored in the first database; and

a receiver configured to retrieve the representation of the action stored as being associated with the second fingerprint and as corresponding to the trigger time point.

25. The system of claim 24 further comprising:

a playback module configured to execute the action at the corresponding trigger time point based on the retrieved representation of the action.

26. A method comprising:

generating a first fingerprint based on content of a segment of a multimedia signal among a plurality of segments of the multimedia signal;

determining that the first fingerprint generated based on the content of the segment matches a second fingerprint that is stored in a first database and associated with a representation of an action stored in a second database,

the second database storing the representation of the action as corresponding to the content of the segment of the multimedia signal,

the second database storing the representation as corresponding to at least one of a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

the determining being performed by a processor of a machine based on a comparison of the first fingerprint generated based on the content of the segment to the second fingerprint stored in the first database;

retrieving the representation of the action stored in the second database as being associated with the second fingerprint and corresponding to the content of the segment of the multimedia signal; and

executing the action based on the retrieved representation of the action.

27. The method of claim 26 , wherein:

at least one of the start of the segment or the end of the segment is defined with respect to the content of the segment of the multimedia signal.

28. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

generating a first fingerprint based on content of a segment of a multimedia signal among a plurality of segments of the multimedia signal;

determining that the first fingerprint generated based on the content of the segment matches a second fingerprint that is stored in a first database and associated with a representation of an action stored in a second database,

the second database storing the representation of the action as corresponding to the content of the segment of the multimedia signal,

the second database storing the representation as corresponding to at least one of a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

the determining being performed by the one or more processors of the machine based on a comparison of the first fingerprint generated based on the content of the segment to the second fingerprint stored in the first database;

retrieving the representation of the action stored in the second database as being associated with the second fingerprint and corresponding to the content of the segment of the multimedia signal; and

executing the action based on the retrieved representation of the action.

29. The non-transitory machine-readable storage medium of claim 28 , wherein:

at least one of the start of the segment or the end of the segment is defined with respect to the content of the segment of the multimedia signal.

30. A system comprising:

a processor configured by a fingerprint detector that configures the processor to generate a first fingerprint based on content of a segment of a multimedia signal among a plurality of segments of the multimedia signal;

the fingerprint detector further configuring the processor to determine that the first fingerprint generated based on the content of the segment matches a second fingerprint that is stored in a first database and associated with a representation of an action stored in a second database,

the second database storing the representation of the action as corresponding to the content of the segment of the multimedia signal,

the second database storing the representation as corresponding to at least one of a predetermined distance from the start of the segment or a predetermined distance from the end of the segment,

the determining being performed by a processor of a machine based on a comparison of the first fingerprint generated based on the content of the segment to the second fingerprint stored in the first database;

a receiver configured to retrieve the representation of the action stored in the second database as being associated with the second fingerprint and corresponding to the content of the segment of the multimedia signal; and

a playback module configured to execute the action based on the retrieved representation of the action.

31. The system of claim 30 , wherein:

at least one of the start of the segment or the end of the segment is defined with respect to the content of the segment of the multimedia signal.

Assignments (15)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
RELEASE (REEL 042262 / FRAME 0601) Recorded Oct 13, 2022
From: CITIBANK, N.A.
To: GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC
Reel/Frame 061748/0001 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: GRACENOTE, INC.
To: ROKU, INC.
Reel/Frame 056103/0786 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2017
From: KONINKLIJKE PHILIPS ELECTRONICS, N.V.
To: GRACENOTE, INC.
Reel/Frame 042414/0131 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2017
From: OOSTVEEN, JOB CORNELIS; TEN KATE, WARNER RUDOLPH THEOPHILE; DENISSEN, ADRIANUS JOHANNES MARIA; ROBERTS, DAVID KEITH
To: KONINKLIJKE PHILIPS ELECTRONICS, N.V.
Reel/Frame 042414/0070 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Apr 13, 2017
From: GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE DIGITAL VENTURES, LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 042262/0601 →
RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Feb 8, 2017
From: JPMORGAN CHASE BANK, N.A.
To: GRACENOTE, INC.; CASTTV INC.; TRIBUNE MEDIA SERVICES, LLC; TRIBUNE DIGITAL VENTURES, LLC
Reel/Frame 041656/0804 →
SECURITY INTEREST Recorded Mar 19, 2014
From: GRACENOTE, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 032480/0272 →