IP Library › Granted Patent US 10,679,672
Granted Patent B2
US 10,679,672 · App. 15/946,946 · Granted Jun 9, 2020

Synthesizing a presentation from multiple media clips

Inventors: Dale T. Roberts (San Anselmo, CA); Randall E. Cook (Kensington, CA); Markus K. Cremer (Orinda, CA)
Assignee: Gracenote, Inc.
G11B27/036G11B27/031G11B27/10G11B27/28G11B27/34H04N5/262H04N7/17336H04N21/21805H04N21/2743H04N21/8549G06F16/40G06F16/958G06F40/103G06F40/106G06F40/14G06F40/174G06F40/177
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,679,672
App. No.
15/946,946
Granted
Jun 9, 2020
Kind
B2
Abstract

Examples described herein may involve synthesizing a presentation of from multiple media clips. An example implementation a plurality of media clips including a first video clip with a first audio waveform and a second video clip with a second audio waveform. The implementation matches a first acoustic fingerprint of at least a part of the first audio waveform with a second acoustic fingerprint of at least a part of the second audio waveform. The implementation determines an overlap of the first video clip with the second video clip based at least in part on the match. The implementation merges the first video clip and the second video clip into a group of overlapping video clips based on the overlap and generates a presentation that includes the group formed by merging the first video clip and the second video clip based on the overlap.

Claims (67)

1. A system comprising:

a processor-implemented media ingestion module configured to access a plurality of media clips including a first video clip with a first audio waveform and a second video clip with a second audio waveform;

a media analysis module configured to:

match a first acoustic fingerprint of at least a part of the first audio waveform of the first video clip with a second acoustic fingerprint of at least a part of the second audio waveform of the second video clip;

match a first event location of at least a part of the first video clip with a second event location of at least a part of the second video clip; and

determine an audio overlap of the first video clip with the second video clip based at least in part on the match of the first acoustic fingerprint of at least the part of the first audio waveform with the second acoustic fingerprint of at least the part of the second audio waveform and the match of the first event location of at least the part of the first video clip with the second event location of at least the part of the second video clip; and

a content creation module configured to:

merge the first video clip and the second video clip into a group of overlapping video clips based on the audio overlap of the first video clip with the second video clip; and

generate a presentation that includes the group formed by merging the first video clip and the second video clip based on the audio overlap to generate the presentation.

2. The system of claim 1 , wherein the first video clip with the first audio waveform is received from a first client device and the second video clip with the second audio waveform is received from a second client device.

3. The system of claim 1 , wherein:

the media analysis module is further configured to match a first video fingerprint of at least a part of the first video clip with a second video fingerprint of at least a part of the second video clip; and

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first video fingerprint of at least the part of the first video clip with the second video fingerprint of at least the part of the second video clip.

4. The system of claim 1 , wherein:

the media analysis module is further configured to match a first metadata of at least a part of the first video clip with a second metadata of at least a part of the second video clip; and

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first metadata of at least the part of the first video clip with the second metadata of at least the part of the second video clip.

5. The system of claim 1 , wherein:

the media analysis module is further configured to match a first event name of at least a part of the first video clip with a second event name of at least a part of the second video clip; and

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first event name of at least the part of the first video clip with the second event name of at least the part of the second video clip.

6. The system of claim 1 , wherein the match of the first event location of at least the part of the first video clip with the second event location of at least the part of the second video clip comprises a match of the first event location of at least the part of the first video clip and the second event location of at least the part of the second video clip with respective concert dates of a tour.

7. The system of claim 1 , wherein:

the first acoustic fingerprint comprises a first melody and the second acoustic fingerprint comprises a second melody;

the media analysis module is further configured to match the first melody with the second melody; and

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first melody with the second melody.

8. The system of claim 1 , wherein:

the first acoustic fingerprint comprises a first chord progression and the second acoustic fingerprint comprises a second chord progression;

the media analysis module is further configured to match the first chord progression with the second chord progression; and

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first chord progression with the second chord progression.

9. A method comprising:

accessing, by a processor, a plurality of media clips including a first video clip with a first audio waveform and a second video clip with a second audio waveform;

matching a first acoustic fingerprint of at least a part of the first audio waveform of the first video clip with a second acoustic fingerprint of at least a part of the second audio waveform of the second video clip;

matching a first event location of at least a part of the first video clip with a second event location of at least a part of the second video clip;

determining an audio overlap of the first video clip with the second video clip based at least in part on the match of the first acoustic fingerprint of at least the part of the first audio waveform with the second acoustic fingerprint of at least the part of the second audio waveform and the match of the first event location of at least the part of the first video clip with the second event location of at least the part of the second video clip;

merging the first video clip and the second video clip into a group of overlapping video clips based on the audio overlap of the first video clip with the second video clip; and

generating a presentation that includes the group formed by merging the first video clip and the second video clip based on the audio overlap.

10. The method of claim 9 , wherein the first video clip with the first audio waveform is received from a first client device and the second video clip with the second audio waveform is received from a second client device.

11. The method of claim 9 , further comprising:

matching a first video fingerprint of at least a part of the first video clip with a second video fingerprint of at least a part of the second video clip; and wherein

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first video fingerprint of at least the part of the first video clip with the second video fingerprint of at least the part of the second video clip.

12. The method of claim 9 , further comprising:

matching a first metadata of at least a part of the first video clip with a second metadata of at least a part of the second video clip; and wherein

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first metadata of at least the part of the first video clip with the second metadata of at least the part of the second video clip.

13. The method of claim 9 , further comprising:

matching a first event name of at least a part of the first video clip with a second event name of at least a part of the second video clip; and wherein

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first event name of at least the part of the first video clip with the second event name of at least the part of the second video clip.

14. The method of claim 9 , wherein the match of the first event location of at least the part of the first video clip with the second event location of at least the part of the second video clip comprises a match of the first event location of at least the part of the first video clip and the second event location of at least the part of the second video clip with respective concert dates of a tour.

15. The method of claim 9 , further comprising:

the first acoustic fingerprint comprises a first chord progression and the second acoustic fingerprint comprises a second chord progression;

matching the first chord progression with the second chord progression; and wherein

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first chord progression with the second chord progression.

16. A non-transitory machine-readable storage medium having instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

accessing a plurality of media clips including a first video clip with a first audio waveform and a second video clip with a second audio waveform;

matching a first acoustic fingerprint of at least a part of the first audio waveform of the first video clip with a second acoustic fingerprint of at least a part of the second audio waveform of the second video clip;

matching a first event location of at least a part of the first video clip with a second event location of at least a part of the second video clip;

determining an audio overlap of the first video clip with the second video clip based at least in part on the match of the first acoustic fingerprint of at least the part of the first audio waveform with the second acoustic fingerprint of at least the part of the second audio waveform and the match of the first event location of at least the part of the first video clip with the second event location of at least the part of the second video clip;

merging the first video clip and the second video clip into a group of overlapping video clips based on the audio overlap of the first video clip with the second video clip; and

generating a presentation that includes the group formed by merging the first video clip and the second video clip based on the audio overlap.

17. The non-transitory machine-readable storage medium of claim 16 , wherein the first video clip with the first audio waveform is received from a first client device and the second video clip with the second audio waveform is received from a second client device.

18. The non-transitory machine-readable storage medium of claim 16 , wherein the operations further comprise:

matching a first video fingerprint of at least a part of the first video clip with a second video fingerprint of at least a part of the second video clip; and wherein

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first video fingerprint of at least the part of the first video clip with the second video fingerprint of at least the part of the second video clip.

19. The non-transitory machine-readable storage medium of claim 16 , wherein the operations further comprise:

matching a first metadata of at least a part of the first video clip with a second metadata of at least a part of the second video clip; and wherein

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first metadata of at least the part of the first video clip with the second metadata of at least the part of the second video clip.

20. The non-transitory machine-readable storage medium of claim 16 , wherein the operations further comprise:

matching a first event name of at least a part of the first video clip with a second event name of at least a part of the second video clip; and wherein

the determining of the audio overlap of the first video clip and the second video clip is based on the match of the first event name of at least the part of the first video clip with the second event name of at least the part of the second video clip.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2018
From: ROBERTS, DALE T.; COOK, RANDALL E.; CREMER, MARKUS K.
To: GRACENOTE, INC.
Reel/Frame 045459/0513 →
Continuity (4)
Continuation 14694624 · Apr 23, 2015
Continuation 12239082 · Sep 26, 2008
Provisional Application 60976186 · Sep 28, 2007
Related Publication 20180226102A1 · Aug 9, 2018