IP Library Granted Patent US 10,714,145
Granted Patent B2
US 10,714,145 · App. 16/386,015 · Granted Jul 14, 2020

Systems and methods to associate multimedia tags with user comments and generate user modifiable snippets around a tag time for efficient storage and sharing of tagged items

Inventors: Sunil Suresh Kulkarni (Santa Cruz, CA); Oleksiy Bolgarov (San Jose, CA)
Assignee: Gracenote, Inc.
G11B27/10G11B27/031H04N21/23418H04N21/26258H04N21/44008H04N21/458H04N21/4788
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,714,145
App. No.
16/386,015
Granted
Jul 14, 2020
Kind
B2
Abstract

Example methods and apparatus to add a tagged snippet of multimedia content to a playlist are disclosed. An example apparatus comprises an automatic content recognition search service to search a fingerprint database to find a match between query fingerprints for a snippet of multimedia content captured from a multimedia program at a timestamp and reference fingerprints of matching reference multimedia content stored in the fingerprint database, a tag service to generate a tag representing the snippet of multimedia content, wherein the tag, the timestamp, meta information associated with the matching reference multimedia content, and a monitored variable for a number of viewers of the snippet of multimedia content are stored in a database storage as a tagged snippet of multimedia content, and to add the tagged snippet of multimedia content to a playlist for an identified multimedia program if the number of viewers of the tagged snippet exceeds a threshold.

Claims (41)

1. A non-transitory machine-readable storage medium having stored thereon, program instructions that when executed by a processor of a machine, cause the machine to perform operations comprising:

capturing a snippet of multimedia content from a multimedia program at a timestamp;

generating a tag representing the snippet of multimedia content, wherein the tag, the timestamp, meta information associated with matching original multimedia content, and a monitored variable for a number of viewers of the snippet of multimedia content are stored in a snippet database as a tagged snippet of multimedia content;

performing a comparison the tagged snippet of multimedia content at a selected insertion point in a playlist of tags with other snippets in the playlist of tags if the number of viewers of the tagged snippet exceeds a threshold to automatically determine whether there is an overlap between the tagged snippet and the other snippets;

determining, based on the comparison, that there is the overlap between the tagged snippet and the other snippets; and

responsive to determining that there is the overlap, automatically removing duplicate content from the tagged snippet determined to have the overlap between the tagged snippet and the other snippets to create a modified snippet with an adjusted size without the overlap for insertion in the playlist of tags.

2. The non-transitory machine-readable storage medium of claim 1 , wherein the operations further comprise: moving an initial start point to a final start point for the overlap determined at a beginning of the snippet.

3. The non-transitory machine-readable storage medium of claim 1 , wherein the operations further comprise: moving the modified snippet to a new position in the playlist of tags, to generate a modified playlist of tags.

4. The non-transitory machine-readable storage medium of claim 1 , wherein the operations further comprise: moving an initial stop point to a final stop point for the overlap determined at an end of the snippet.

5. The non-transitory machine-readable storage medium of claim 1 , wherein capturing the snippet of multimedia content from the multimedia program at the timestamp comprises selectively capturing, using a mobile device, the snippet of multimedia content from the multimedia program at the timestamp.

6. A non-transitory machine-readable storage medium having stored thereon, program instructions that when executed by a processor of a machine, cause the machine to perform operations comprising:

receiving multimedia content;

generating, according to a specified parameter, a first plurality of snippets from the multimedia content;

selecting, according to a predefined threshold, a second plurality of snippets from the first plurality of snippets;

ordering, in an ordered playlist, the second plurality of snippets according to a start time of each snippet of the second plurality of snippets in the ordered playlist; and

processing snippets from the ordered playlist that are overlapping to automatically remove duplication of multimedia content from the ordered playlist and generate a final playlist with non-overlapping snippets.

7. The non-transitory machine-readable storage medium of claim 6 , wherein the operations further comprise:

adding a new user selected snippet to the final playlist to create a modified playlist;

reordering the modified playlist according to a start time of each snippet in the reordered modified playlist; and

processing the snippets from the reordered modified playlist that are overlapping to remove duplication of multimedia content from the reordered modified playlist.

8. The non-transitory machine-readable storage medium of claim 6 , wherein the specified parameter is a classification of the multimedia content.

9. The non-transitory machine-readable storage medium of claim 6 , wherein the predefined threshold represents a specified number of viewers of the multimedia content.

10. The non-transitory machine-readable storage medium of claim 6 , wherein the predefined threshold is a fixed threshold that does not change as a number of viewers changes during repeated playing of the multimedia content.

11. The non-transitory machine-readable storage medium of claim 6 , wherein the predefined threshold is a dynamic threshold that is a percentage of a number of viewers, and

wherein the dynamic threshold changes as the number of viewers change during repeated playing of the multimedia content.

12. The non-transitory machine-readable storage medium of claim 6 , wherein the second plurality of snippets represents a quantity of snippets that were most-viewed by a plurality of viewers.

13. The non-transitory machine-readable storage medium of claim 6 , wherein the specified parameter is a time measure representing a duration of playing a snippet.

14. The non-transitory machine-readable storage medium of claim 6 , wherein the specified parameter is an indication of a user request to generate one or more snippets.

15. The non-transitory machine-readable storage medium of claim 6 , wherein the operations further comprise: identifying each snippet of the first plurality of snippets from a successful match of fingerprints based on features detected from a scale space analysis of frames in the snippet compared with original fingerprints of original multimedia content stored by the machine.

16. A system comprising:

a memory that stores instructions; and

one or more processors configured by the instructions to perform operations comprising:

capturing a snippet of multimedia content from a multimedia program at a timestamp;

generating a tag representing the snippet of multimedia content, wherein the tag, the timestamp, meta information associated with matching original multimedia content, and a monitored variable for a number of viewers of the snippet of multimedia content are stored in a snippet database as a tagged snippet of multimedia content;

performing a comparison the tagged snippet of multimedia content at a selected insertion point in a playlist of tags with other snippets in the playlist of tags if the number of viewers of the tagged snippet exceeds a threshold to automatically determine whether there is an overlap between the tagged snippet and the other snippets;

determining, based on the comparison, that there is the overlap between the tagged snippet and the other snippets; and

responsive to determining that there is the overlap, automatically removing duplicate content from the tagged snippet determined to have the overlap between the tagged snippet and the other snippets to create a modified snippet with an adjusted size without the overlap for insertion in the playlist of tags.

17. The system of claim 16 , wherein the operations further comprise: moving an initial start point to a final start point for the overlap determined at a beginning of the snippet.

18. The system of claim 16 , wherein the operations further comprise: moving the modified snippet to a new position in the playlist of tags, to generate a modified playlist of tags.

19. The system of claim 16 , wherein the operations further comprise: moving an initial stop point to a final stop point for the overlap determined at an end of the snippet.

20. The system of claim 16 , wherein capturing the snippet of multimedia content from the multimedia program at the timestamp comprises selectively capturing, using a mobile device, the snippet of multimedia content from the multimedia program at the timestamp.

Assignments (9)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 16, 2019
From: KULKARNI, SUNIL SURESH; BOLGAROV, OLEKSIY
To: ZEITERA, LLC
Reel/Frame 048901/0476 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 16, 2019
From: ZEITERA, LLC
To: GRACENOTE, INC.
Reel/Frame 048901/0493 →
Continuity (4)
Continuation 15973118 · May 7, 2018
Continuation 14255742 · Apr 17, 2014
Provisional Application 61813305 · Apr 18, 2013
Related Publication 20190348078A1 · Nov 14, 2019
Cited By (5)
US 12,301,960 US 12,425,321 US 12,556,519 US 12,666,121 US 12,701,306