IP Library Granted Patent US 10,262,694
Granted Patent B2
US 10,262,694 · App. 15/875,158 · Granted Apr 16, 2019

Multimedia content duration manipulation

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,262,694
App. No.
15/875,158
Granted
Apr 16, 2019
Kind
B2
Abstract

Average pixel luminosity is calculated for each frame comprising a content item. For each pair of adjacent frames, an IFD is calculated. The IFD represents the difference between a baseline pixel luminosity associated with each of the two frames. An initial set of cut frames is selected based on IFD values that are less than a minimum value IFD min , or that are greater than a maximum value IFD max . The positions of these initial cut frames are optimized using a numerical optimization technique that favors removal of frames corresponding to IFD extrema, but that also attempts to maintain a minimum time gap between cut frames. Selecting frames for removal is approached as a constraint minimization problem. Once an optimized set of cut frames is established, audio is cut and crossfaded in a temporal window surrounding cut frame positions.

Claims (135)

1. A method for manipulating a duration of a content item, the method comprising:

receiving a content item that comprises a plurality of video frames;

calculating a plurality of inter frame distance (IFD) values associated with the content item, wherein each of the IFD values represents a difference between baseline pixel luminosities associated with two adjacent video frames that comprise the content item;

selecting an initial set of frame positions at least partially based on the plurality of IFD values and a minimum time gap G separating the frame positions;

adjusting the frame positions using a numerical optimization technique; and

modifying a duration of the content item by manipulating one or more video frames at the adjusted frame positions;

wherein each of the IFD values is calculated by:

(a) determining a first luminosity of a first pixel comprising a first video frame;

(b) determining a second luminosity of a second pixel comprising a second video frame that is adjacent to the first video frame, wherein the first and second pixels correspond to a common position in the first and second video frames;

(c) determining a difference between the first and second luminosities;

(d) repeating (a), (b), and (c) for a plurality of pixels comprising the first and second video frames, thereby producing a plurality of luminosity differences; and

(e) averaging the plurality of luminosity differences.

2. The method of claim 1 , wherein the duration of the content item is modified by removing video frames at the adjusted frame positions.

3. The method of claim 1 , wherein the duration of the content item is modified by inserting a new video frame adjacent to one of the adjusted frame positions.

4. The method of claim 1 , wherein each of the frame positions in the initial set corresponds to an IFD value that is either less than a threshold value IFD min or greater than a threshold value IFD max .

5. The method of claim 1 , further comprising cross-blending a plurality of video frames adjacent to one of the adjusted frame positions.

6. A non-transitory computer readable medium encoded with instructions that, when executed by one or more processors, cause a process for reducing a duration of a multimedia content item to be carried out, the process comprising:

receiving a multimedia content item that includes a plurality of video frames and an audio track configured to be played synchronously with the video frames;

calculating a plurality of inter frame distance (IFD) values associated with the multimedia content item, wherein each of the IFD values represents a difference between baseline pixel luminosities associated with two adjacent video frames that comprise the multimedia content item;

selecting an initial subset of the video frames at least partially based on the plurality of IFD values, wherein each of the selected video frames is associated with a cut frame position, and wherein at least a minimum time gap G separates the cut frame positions from each other;

adjusting at least one of the cut frame positions using a numerical optimization technique;

removing, from the multimedia content item, a particular one of the selected video frames that was included in the initial subset; and

cutting and crossfading the audio track within a window surrounding a cut frame position associated with the removed particular video frame, thereby causing a duration of the audio track to be reduced;

wherein each of the IFD values is calculated by:

(a) determining a first luminosity of a first pixel comprising a first video frame;

(b) determining a second luminosity of a second pixel comprising a second video frame that is adjacent to the first video frame, wherein the first and second pixels correspond to a common position in the first and second video frames;

(c) determining a difference between the first and second luminosities;

(d) repeating (a), (b), and (c) for a plurality of pixels comprising the first and second video frames, thereby producing a plurality of luminosity differences; and

(e) averaging the plurality of luminosity differences.

7. The non-transitory computer readable medium of claim 6 , wherein the process further comprises cross-blending a plurality of video frames adjacent to the removed particular video frame.

8. The non-transitory computer readable medium of claim 6 , wherein the numerical optimization technique adjusts at least one of the cut frame positions by minimizing a quantity

-

λ

i

=

1

N

IFD

(

x

i

)

2

-

i

=

1

N

{

ln

[

t

(

x

i

)

-

t

(

x

i

-

1

)

-

G

]

+

ln

[

t

(

x

i

+

1

)

-

t

(

x

i

)

-

G

]

}

,

wherein:

λ is a weighting parameter;

N is a quantity of video frames in the initial subset;

x i represents an ith video frame;

IFD′(x i ) represents a modified computation of an IFD corresponding to the x i th video frame, wherein a smooth continuous correcting curve is applied for initially-calculated IFD values less than a threshold value IFD min ; and

t(x i ) corresponds to a time associated with the x i th video frame.

9. The non-transitory computer readable medium of claim 6 , wherein the minimum time gap G is between about 0.25 seconds and about 0.50 seconds.

10. The non-transitory computer readable medium of claim 6 , wherein the process further comprises:

receiving a qualitative quality control input; and

setting the minimum time gap G based on the qualitative quality control input.

11. The non-transitory computer readable medium of claim 6 , wherein IFD values are calculated for each pair of adjacent video frames comprising the multimedia content item.

12. The non-transitory computer readable medium of claim 6 , wherein cutting and crossfading the audio track further comprises:

determining a quantity of audio data segments to be extracted from the window;

identifying an audio data segment existing within the window;

determining a fundamental frequency and a corresponding sample period of the identified audio data segment; and

crossfading a plurality of samples at the cut frame position associated with the removed particular video frame, wherein said crossfading is performed using a correlation-based mixture of linear and constant power crossfade.

13. A system for manipulating a duration of a digital content item, the system comprising:

an inter frame distance (IFD) calculation module configured to calculate a plurality of IFD values associated with video frames that comprise a digital content item, wherein each of the IFD values represents a difference between baseline pixel luminosities associated with two adjacent video frames;

a frame selection module configured to identify a set of frame positions corresponding to IFD values that are less than a threshold value IFD min , wherein the frame positions are separated from each other by at least a minimum time gap G; and

a duration manipulation module configured to modify a duration of the digital content item by manipulating a particular video frame corresponding to a particular one of the frame positions;

wherein each of the IFD values is calculated by:

(a) determining a first luminosity of a first pixel comprising a first video frame;

(b) determining a second luminosity of a second pixel comprising a second video frame that is adjacent to the first video frame, wherein the first and second pixels correspond to a common position in the first and second video frames;

(c) determining a difference between the first and second luminosities;

(d) repeating (a), (b), and (c) for a plurality of pixels comprising the first and second video frames, thereby producing a plurality of luminosity differences; and

(e) averaging the plurality of luminosity differences.

14. The system of claim 13 , wherein:

the digital content item further comprises an audio track configured to be played synchronously with the video frames; and

the system further comprises an audio manipulation sub-module configured to modify a duration of the audio track by cutting and crossfading the audio track within a window surrounding one or more of the frame positions.

15. The system of claim 13 , further comprising a supplemental frame selection module configured to identify a supplemental set of frame positions corresponding to IFD values that are greater than a threshold value IFD max .

16. The system of claim 13 , further comprising a supplemental frame selection module configured to identify a supplemental set of frame positions corresponding to IFD values that are greater than a threshold value IFD max , wherein the duration manipulation module is further configured to modify the duration of the digital content item by manipulating a second particular video frame corresponding to a particular one of the frame positions in the supplemental set of frame positions.

17. The system of claim 13 , wherein the duration of the digital content item is modified by removing the particular video frame.

18. The system of claim 13 , wherein IFD values are calculated for each pair of adjacent video frames comprising the digital content item.

19. The system of claim 13 , wherein the duration of the content item is modified by inserting a new video frame adjacent to the particular video frame.

Assignments (3)
CHANGE OF NAME Recorded Apr 8, 2019
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 048867/0882 →
CHANGE OF NAME Recorded Nov 30, 2018
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 047688/0530 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 19, 2018
From: BECHERER, NICO
To: ADOBE SYSTEMS INCORPORATED
Reel/Frame 044666/0016 →