Granted Patent B2

US 10,769,208 · App. 16/151,588 · Granted Sep 8, 2020

Inventors: Yale Song (New York, NY); Jordi Vallmitjana (New York, NY); Amanda Stent (New York, NY); Alejandro Jaimes (Brooklyn, NY)

Assignee: OATH INC.

G06F16/739G06F16/345G06F16/38G06F16/78

Abstract

Claims (68)

1. A method comprising:

obtaining, by a digital content summarization system server and for a first media content item having associated descriptive information, a plurality of second media content items as auxiliary data, the obtaining using the descriptive information associated with the first media content item, the first media content item comprising a plurality of units;

identifying, by the digital content summarization system server, a plurality of segments of the first media content item, each segment comprising at least one unit of the first media content item's plurality of units;

generating, by the digital content summarization system server and using the first media content item, a plurality of first feature descriptor sets, each first feature descriptor set of the plurality corresponding to a segment of the plurality of segments and comprising values determined using the segment;

identifying, by the digital content summarization system server, a first set of patterns from the plurality of first feature descriptor sets;

generating, by the digital content summarization system server, a plurality of second feature descriptor sets, each second feature descriptor set, of the plurality, corresponding to a second media content item, of the plurality, and comprising values determined using the second media content item;

identifying, by the digital content summarization system server, a second set of patterns from the plurality of second feature descriptor sets;

determining, by the digital content summarization system server and using the first and second sets of patterns, a shared dictionary comprising a number of shared patterns found in both the first and second sets of patterns;

determining, by the digital content summarization system server and using the shared dictionary, a score for each segment of the plurality of segments of the first media content item, the score determined for a segment, of the plurality of segments, is a measure of how representative the segment is of the first media content item;

identifying, by the digital content summarization system server, a number of segments of the plurality of segments of the first media content item, each segment of the number being selected using its determined score; and

generating, by the digital content summarization system server, a summary of the first media content item, the summary comprising the at least one segment, of the plurality of segments of the first media content item, identified as being more similar to the descriptive information.

2. The method of claim 1 , determining a shared dictionary further comprising:

determining a first approximation coefficient for use in approximating the shared patterns of the shared dictionary, the first approximation coefficient comprising a first approximation coefficient vector for each unit of the plurality of units of the first media content item; and

determining a second approximation coefficient that, in combination with the shared patterns of the shared dictionary, approximates the first media content item, the second approximation coefficient comprising a second approximation coefficient vector for each unit of the plurality of units of the first media content item.

3. The method of claim 2 , determining a score for a segment of the plurality of segments of the first media content item further comprising:

determining a score for each unit of the segment, the score for a unit of the segment being determined using the first approximation coefficient vector corresponding to the unit and the second approximation coefficient vector corresponding to each unit of the plurality of units; and

determining a score for the segment using the score determined for each unit of the segment.

4. The method of claim 3 , the score determined for the segment is an average determined using the score determined for each unit of the segment.

5. The method of claim 2 , further comprising:

determining a third approximation coefficient for use with the first approximation coefficient, the first media content item and the plurality of second media content items in approximating the shared patterns of the shared dictionary.

6. The method of claim 2 , further comprising:

determining a fourth approximation coefficient that, in combination with the shared patterns of the shared dictionary, approximates the plurality of second media content items.

7. The method of claim 1 , the first media content item is video content and each second media content item of the plurality of second media content items is image content.

8. The method of claim 1 , a shared pattern, of the number of shared patterns, corresponding to values of a common feature descriptor set found in both the first and second feature descriptor sets.

9. The method of claim 1 , the shared dictionary excluding any patterns not found in both the first media content item and the auxiliary data.

10. The method of claim 1 , identifying the second set of patterns from the second set of features further comprising:

extracting, from the second feature descriptor sets, a number of common feature descriptor sets, each common feature descriptor set of the number being found in each of the second feature descriptor sets; and

using the set of common feature descriptor sets in identifying the second set of patterns.

11. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device perform a method comprising:

obtaining, for a first media content item having associated descriptive information, a plurality of second media content items as auxiliary data, the obtaining using the descriptive information associated with the first media content item, the first media content item comprising a plurality of units;

identifying a plurality of segments of the first media content item, each segment comprising at least one unit of the first media content item's plurality of units;

generating, using the first media content item, a plurality of first feature descriptor sets, each first feature descriptor set of the plurality corresponding to a segment of the plurality of segments and comprising values determined using the segment;

identifying a first set of patterns from the plurality of first feature descriptor sets;

generating a plurality of second feature descriptor sets, each second feature descriptor set, of the plurality, corresponding to a second media content item, of the plurality, and comprising values determined using the second media content item;

identifying a second set of patterns from the plurality of second feature descriptor sets;

determining, using the first and second sets of patterns, a shared dictionary comprising a number of shared patterns found in both the first and second sets of patterns;

determining, using the shared dictionary, a score for each segment of the plurality of segments of the first media content item, the score determined for a segment, of the plurality of segments, is a measure of how representative the segment is of the first media content item;

identifying a number of segments of the plurality of segments of the first media content item, each segment of the number being selected using its determined score; and

generating a summary of the first media content item, the summary comprising the at least one segment, of the plurality of segments of the first media content item, identified as being more similar to the descriptive information.

12. The non-transitory computer-readable storage medium of claim 11 , determining a shared dictionary further comprising:

determining a first approximation coefficient for use in approximating the shared patterns of the shared dictionary, the first approximation coefficient comprising a first approximation coefficient vector for each unit of the plurality of units of the first media content item; and

determining a second approximation coefficient that, in combination with the shared patterns of the shared dictionary, approximates the first media content item, the second approximation coefficient comprising a second approximation coefficient vector for each unit of the plurality of units of the first media content item.

13. The non-transitory computer-readable storage medium of claim 12 , determining a score for a segment of the plurality of segments of the first media content item further comprising:

determining a score for each unit of the segment, the score for a unit of the segment being determined using the first approximation coefficient vector corresponding to the unit and the second approximation coefficient vector corresponding to each unit of the plurality of units; and

determining a score for the segment using the score determined for each unit of the segment.

14. The non-transitory computer-readable storage medium of claim 13 , the score determined for the segment is an average determined using the score determined for each unit of the segment.

15. The non-transitory computer-readable storage medium of claim 12 , further comprising:

determining a third approximation coefficient for use with the first approximation coefficient, the first media content item and the plurality of second media content items in approximating the shared patterns of the shared dictionary.

16. The non-transitory computer-readable storage medium of claim 12 , further comprising:

determining a fourth approximation coefficient that, in combination with the shared patterns of the shared dictionary, approximates the plurality of second media content items.

17. The non-transitory computer-readable storage medium of claim 11 , a shared pattern, of the number of shared patterns, corresponding to values of a common feature descriptor set found in both the first and second feature descriptor sets.

18. The non-transitory computer-readable storage medium of claim 11 , the shared dictionary excluding any patterns not found in both the first media content item and the auxiliary data.

19. The non-transitory computer-readable storage medium of claim 11 , identifying the second set of patterns from the second set of features further comprising:

extracting, from the second feature descriptor sets, a number of common feature descriptor sets, each common feature descriptor set of the number being found in each of the second feature descriptor sets; and

using the set of common feature descriptor sets in identifying the second set of patterns.

20. A computing device comprising:

a processor;

a non-transitory storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:

obtaining logic executed by the processor for obtaining, for a first media content item having associated descriptive information, a plurality of second media content items as auxiliary data, the obtaining using the descriptive information associated with the first media content item, the first media content item comprising a plurality of units;

identifying logic executed by the processor for identifying a plurality of segments of the first media content item, each segment comprising at least one unit of the first media content item's plurality of units;

generating logic executed by the processor for generating, using the first media content item, a plurality of first feature descriptor sets, each first feature descriptor set of the plurality corresponding to a segment of the plurality of segments and comprising values determined using the segment;

identifying logic executed by the processor for identifying a first set of patterns from the plurality of first feature descriptor sets;

generating logic executed by the processor for generating a plurality of second feature descriptor sets, each second feature descriptor set, of the plurality, corresponding to a second media content item, of the plurality, and comprising values determined using the second media content item;

identifying a second set of patterns from the plurality of second feature descriptor sets;

determining logic executed by the processor for determining, using the first and second sets of patterns, a shared dictionary comprising a number of shared patterns found in both the first and second sets of patterns;

determining logic executed by the processor for determining, using the shared dictionary, a score for each segment of the plurality of segments of the first media content item, the score determined for a segment, of the plurality of segments, is a measure of how representative the segment is of the first media content item;

identifying logic executed by the processor for identifying a number of segments of the plurality of segments of the first media content item, each segment of the number being selected using its determined score; and

generating logic executed by the processor for generating a summary of the first media content item, the summary comprising the at least one segment, of the plurality of segments of the first media content item, identified as being more similar to the descriptive information.

Assignments (9)

Continuity (2)