IP Library Granted Patent US 9,942,581
Granted Patent B2
US 9,942,581 · App. 14/985,692 · Granted Apr 10, 2018

Content-adaptive digital content adjustment method and system

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,942,581
App. No.
14/985,692
Granted
Apr 10, 2018
Kind
B2
Abstract

Disclosed herein is a content-adaptive digital media content adjustment system, method and architecture. Playback rate of a media content item comprising audio and video content can be increased by removing a portion of the content item's audio content and a corresponding portion of the content item's video content. An increased playback rate of a multimedia content item is accomplished by decreasing the length of the multimedia content item by removing portions of the multimedia content item's audio and video content.

Claims (61)

1. A method comprising:

modifying, by a computing device, a content item's play length by removing an amount of the content item's audio content and video content, the modifying comprising:

identifying a time allocation score for each of a plurality of audio segments in the audio content, each audio segment's time allocation score being inversely related to its tempo, each audio segment identified as a silent audio segment being assigned a given tempo and each audio segment identified as a non-silent audio segment being assigned a tempo using its audio content;

determining a desired length for each audio segment of a number of audio segments selected from the plurality, the selected audio segments having a higher time allocation score relative to non-selected audio segments of the plurality, the determining comprising using a tempo increase amount in determining the desired length;

removing, for each selected audio segment, some portion of the audio segment's content corresponding to a difference between the audio segment's original length and its desired length;

identifying, for each identified audio segment, a corresponding video segment in the content item; and

adjusting, for each adjusted audio segment, the corresponding content segment by removing a number of frames from the corresponding video segment until the adjusted length of the corresponding video segment matches the selected audio segment's desired length.

2. The method of claim 1 , the adjusting the corresponding content segment further comprising:

determining a delta score for each frame of the corresponding segment, a frame's delta score indicative of a level of difference between it and at least one other contiguous frame; and

selecting the number of frames from the corresponding video segment based on the delta score of each frame, such that a frame having a smaller delta score relative to other frames is selected.

3. The method of claim 2 , the adjusting the corresponding content segment further comprising:

increasing the delta score of a frame before and a frame after a selected frame.

4. The method of claim 1 , removing some portion of the audio segment's content further comprising:

identifying a number of perceptually insignificant portions of the audio segment's content; and

removing the perceptually insignificant portions.

5. The method of claim 1 , the content item's playback rate is increased by removing an amount of the content item's audio and video content.

6. The method of claim 1 , each audio segment's time allocation score is weighted in accordance with the content item's quality using a weighting selected based on the content item's quality, weighted time allocation scores impacting the amount of audio content and video content removed from the content item such that the weighting used with a high quality content item results in a smaller amount of audio content and video content being removed than a lower quality content item.

7. The method of claim 1 , the given tempo used with silent audio segments can be selected based on a quality of the content item.

8. The method of claim 1 , the identifying a corresponding video segment further comprising:

identifying the corresponding video segment using a start time and end time of the audio segment.

9. A system comprising:

at least one computing device, each computing device comprising a processor and a storage medium for tangibly storing thereon program logic for execution by the processor, the stored program logic comprising:

modifying logic executed by the processor for modifying a content item's play length by removing an amount of the content item's audio content and video content, the modifying logic comprising:

identifying logic executed by the processor for identifying a time allocation score for each of a plurality of audio segments in the audio content, each audio segment's time allocation score being inversely related to its tempo, each audio segment identified as a silent audio segment being assigned a given tempo and each audio segment identified as a non-silent audio segment being assigned a tempo using its audio content;

determining logic executed by the processor for determining a desired length for each audio segment of a number of audio segments selected from the plurality, the selected audio segments having a higher time allocation score relative to non-selected audio segments of the plurality, the determining comprising using a tempo increase amount in determining the desired length;

removing logic executed by the processor for removing, for each selected audio segment, some portion of the audio segment's content corresponding to a difference between the audio segment's original length and its desired length;

identifying logic executed by the processor for identifying, for each identified audio segment, a corresponding video segment in the content item; and

adjusting logic executed by the processor for adjusting, for each adjusted audio segment, the corresponding content segment by removing a number of frames from the corresponding video segment until the adjusted length of the corresponding video segment matches the selected audio segment's desired length.

10. The system of claim 9 , the adjusting logic executed by the processor for adjusting the corresponding content segment further comprising:

determining logic executed by the processor for determining a delta score for each frame of the corresponding segment, a frame's delta score indicative of a level of difference between it and at least one other contiguous frame; and

selecting logic executed by the processor for selecting the number of frames from the corresponding video segment based on the delta score of each frame, such that a frame having a smaller delta score relative to other frames is selected.

11. The system of claim 10 , the adjusting logic executed by the processor for adjusting the corresponding content segment further comprising:

increasing logic executed by the processor for increasing the delta score of a frame before and a frame after a selected frame.

12. The system of claim 9 , the removing logic executed by the processor for removing some portion of the audio segment's content further comprising:

identifying logic executed by the processor for identifying a number of perceptually insignificant portions of the audio segment's content; and

removing logic executed by the processor for removing the perceptually insignificant portions.

13. The system of claim 9 , the content item's playback rate is increased by removing an amount of the content item's audio and video content.

14. The system of claim 9 , each audio segment's time allocation score is weighted in accordance with the content item's quality using a weighting selected based on the content item's quality, weighted time allocation scores impacting the amount of audio content and video content removed from the content item such that the weighting used with a high quality content item results in a smaller amount of audio content and video content being removed than a lower quality content item.

15. The system of claim 9 , the given tempo used with silent audio segments can be selected based on a quality of the content item.

16. The system of claim 9 , the identifying logic executed by the processor for identifying a corresponding video segment further comprising:

identifying logic executed by the processor for identifying the corresponding video segment using a start time and end time of the audio segment.

17. A computer readable non-transitory storage medium for tangibly storing thereon computer readable instructions that when executed cause a processor to:

modify a content item's play length by removing an amount of the content item's audio content and video content, the instructions to modify comprising instructions that when executed cause the processor to:

identify a time allocation score for each of a plurality of audio segments in the audio content, each audio segment's time allocation score being inversely related to its tempo, each audio segment identified as a silent audio segment being assigned a given tempo and each audio segment identified as a non-silent audio segment being assigned a tempo using its audio content;

determine a desired length for each audio segment of a number of audio segments selected from the plurality, the selected audio segments having a higher time allocation score relative to non-selected audio segments of the plurality, the determining comprising using a tempo increase amount in determining the desired length;

remove, for each selected audio segment, some portion of the audio segment's content corresponding to a difference between the audio segment's original length and its desired length;

identify, for each identified audio segment, a corresponding video segment in the content item; and

adjust, for each adjusted audio segment, the corresponding content segment by removing a number of frames from the corresponding video segment until the adjusted length of the corresponding video segment matches the selected audio segment's desired length.

18. The computer readable non-transitory storage medium of claim 17 , the instructions that when executed cause the processor to adjust the corresponding content segment further comprising instructions that when executed cause the processor to:

determine a delta score for each frame of the corresponding segment, a frame's delta score indicative of a level of difference between it and at least one other contiguous frame; and

select the number of frames from the corresponding video segment based on the delta score of each frame, such that a frame having a smaller delta score relative to other frames is selected.

19. The computer readable non-transitory storage medium of claim 18 , the instructions that when executed cause the processor to adjust the corresponding content segment further comprising instructions that when executed cause the processor to:

increase the delta score of a frame before and a frame after a selected frame.

20. The computer readable non-transitory storage medium of claim 17 , instructions that when executed cause the processor to remove some portion of the audio segment's content further comprising instructions that when executed cause the processor to:

identify a number of perceptually insignificant portions of the audio segment's content; and

remove the perceptually insignificant portions.

21. The computer readable non-transitory storage medium of claim 17 , the content item's playback rate is increased by removing an amount of the content item's audio and video content.

22. The computer readable non-transitory storage medium of claim 17 , each audio segment's time allocation score is weighted in accordance with the content item's quality using a weighting selected based on the content item's quality, weighted time allocation scores impacting the amount of audio content and video content removed from the content item such that the weighting used with a high quality content item results in a smaller amount of audio content and video content being removed than a lower quality content item.

23. The computer readable non-transitory storage medium of claim 17 , the given tempo used with silent audio segments can be selected based on a quality of the content item.

24. The computer readable non-transitory storage medium of claim 17 , the instructions that when executed cause the processor to identify a corresponding video segment further comprising instructions that when executed cause the processor to:

identify the corresponding video segment using a start time and end time of the audio segment.

Assignments (6)
PATENT SECURITY AGREEMENT (FIRST LIEN) Recorded Sep 29, 2022
From: YAHOO ASSETS LLC
To: ROYAL BANK OF CANADA, AS COLLATERAL AGENT
Reel/Frame 061571/0773 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2021
From: YAHOO AD TECH LLC (FORMERLY VERIZON MEDIA INC.)
To: YAHOO ASSETS LLC
Reel/Frame 058982/0282 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: OATH INC.
To: VERIZON MEDIA INC.
Reel/Frame 054258/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2018
From: YAHOO HOLDINGS, INC.
To: OATH INC.
Reel/Frame 045240/0310 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2017
From: YAHOO! INC.
To: YAHOO HOLDINGS, INC.
Reel/Frame 042963/0211 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 31, 2015
From: JIN, HAOJIAN; WEN, ZHENG; SONG, YALE
To: YAHOO! INC.
Reel/Frame 037389/0657 →