Systems and methods for motion-vector-aided video interpolation using real-time smooth video playback speed variation
View Patent ↗Systems and methods for encoding and playing back video at adjustable playback speeds by interpolating frames to achieve smooth playback in accordance with embodiments of the invention are described. One embodiment includes a source encoder that includes a processor, memory including an encoder application, where the encoder application directs the processor to: select a subset of frames from a first video sequence; generate motion vectors describing frames from the first video sequence that are not part of the selected subset of frames, where each motion vector describes movement between a frame in the subset of frames and a frame not included in the subset of frames; store the motion vectors; decimate frames not included in the subset of frames from the first video sequence to generate a second video sequence having a nominal frame rate less than the frame rate of the first video sequence; and encode the second video sequence at the nominal frame rate.
1. A non-transitory computer-readable medium containing computer-executable instructions that, when executed by a hardware processor, cause the hardware processor to perform a method for encoding video, the method comprising:
obtain a first video content comprising a plurality of frames providing a first frame rate;
generate a set of motion vectors describing movement between frames in the plurality of frames;
decimate a subset of frames from the plurality of frames of the first video content to generate a second video content having a file size less than a file size of the first video content; and
encode the second video content and associate the set of motion vectors with the second video content.
2. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises storing the set of motion vectors as encoded pixels within frames in the second video content.
3. The non-transitory computer-readable medium of claim 2 , wherein a motion vector comprises an angle component and a magnitude component, wherein the angle component is stored within a hue value of a pixel and the magnitude component is stored within a brightness value of a pixel.
4. The non-transitory computer-readable medium of claim 2 , wherein a motion vector is encoded as one or more pixels within a portion of a frame that is not displayed during playback of the frame.
5. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises encoding the second video content as a sequence of intra and inter frames.
6. The non-transitory computer-readable medium of claim 1 , wherein a motion vector describes movement of at least one pixel between a frame in the second video content and a frame in the subset of decimated frames.
7. The non-transitory computer-readable medium of claim 1 , wherein a motion vector describes movement of at least one pixel between a particular frame in the first video content and a subsequent nth frame to the particular frame.
8. The non-transitory computer-readable medium of claim 1 , wherein the set of motion vectors are stored in a separate file, and each motion vector corresponds to a particular frame in the second video content.
9. The non-transitory computer-readable medium of claim 1 , wherein the method further comprises:
computing a motion vector for each pixel in a particular frame in the first video content and a subsequent frame; and
compressing motion vectors of a subset of neighboring pixels in the particular frame to generate a new motion vector.
10. The non-transitory computer-readable medium of claim 1 , wherein a motion vector describes movement of one or more pixels across a plurality of frames of the first video content.
11. The non-transitory computer-readable medium of claim 1 , wherein a motion vector provides interpolated positions of pixels at times between a particular frame and a subsequent nth frame in the first video content.
12. The non-transitory computer-readable medium of claim 1 , wherein a motion vector provides interpolated positions of pixels at times between a particular frame and a preceding nth frame in the first video content.
13. The non-transitory computer-readable medium of claim 1 , wherein the second video content comprises the plurality of frames of the first video content without the decimated subset of frames.
14. A source encoder, comprising:
a processor;
memory including an encoder application;
where the encoder application directs the processor to:
obtain a first video content comprising a plurality of frames providing a first frame rate;
generate a set of motion vectors describing movement between frames in the plurality of frames;
decimate a subset of frames from the plurality of frames of the first video content to generate a second video content having a file size less than a file size of the first video content; and
encode the second video content and associate the set of motion vectors with the second video content.
15. The source encoder of claim 14 , wherein the encoder application directs the processor to store the set of motion vectors as encoded pixels within frames in the second video content.
16. The source encoder of claim 14 , wherein a motion vector describes movement of at least one pixel between a frame in the second video content and a frame in the subset of decimated frames.
17. The source encoder of claim 14 , wherein a motion vector describes movement of at least one pixel between a particular frame in the first video content and at least one frame selected from the group consisting of (1) a subsequent nth frame to the particular frame and (2) a preceding nth frame to the particular frame.
18. The source encoder of claim 14 , wherein a motion vector describes movement of one or more pixels across a plurality of frames of the first video content.
19. The source encoder of claim 14 , wherein the encoder application directs the processor to:
compute a motion vector for each pixel in a particular frame in the first video content and a subsequent frame; and
compress motion vectors of a subset of neighboring pixels in the particular frame to generate a new motion vector.