Encoding and Decoding Selectively Retrievable Representations of Video Content
A system and method disposed to enable encoding, decoding and manipulation of digital video with substantially less processing load than would otherwise be required. In particular, one disclosed method is directed to generating a compressed video data structure that is selectively decodable to a plurality of resolutions including the full resolution of the uncompressed stream. The desired number of data components and the content of the data components that make up the compressed video data, which determine the available video resolutions, are variable based upon the processing carried out and the resources available to decode and process the data components. During decoding, efficiency is substantially improved because only the data components necessary to generate a desired resolution are decoded. In variations, both temporal and spatial decoding are utilized to reduce frame rates, and hence, further reduce processor load. The system and method are particularly useful for real-time video editing applications.
1 . A method to temporally compress a source video, comprising:
responsive to a request to temporally compress the source video to a reduced frame rate, receiving a plurality of frames of the source video; and
for at least a first frame and a second frame of the source video:
averaging the first frame and the second frame to generate an average frame; and
generating a temporally compressed video based on the average frame, where the temporally compressed video is characterized by the reduced frame rate.
2 . The method of claim 1 , where the method further comprises:
for at least a third frame and a fourth frame of the source video:
averaging the third frame and the fourth frame to generate a second average frame; and
where the temporally compressed video is also based on the second average frame.
3 . The method of claim 2 , where the reduced frame rate of the temporally compressed video is a half frame rate of the source video.
4 . The method of claim 2 , where the method further comprises:
for at least the average frame and the second average frame:
averaging the average frame and the second average frame to generate a third average frame; and
where the temporally compressed video is also based on the third average frame.
5 . The method of claim 4 , where the reduced frame rate of the temporally compressed video is a quarter frame rate of the source video.
6 . The method of claim 1 , where the method further comprises differencing the first frame and the second frame to generate a motion difference frame.
7 . The method of claim 6 , where the method further comprises:
for at least a third frame and a fourth frame of the source video:
differencing the third frame and the fourth frame to generate a second motion difference frame; and
where the temporally compressed video is also based on the second motion difference frame.
8 . A method to efficiently compress color space data for a source video, comprising:
converting a first color space image to a second color space image comprising a first component, a second component, and a third component;
sub-sampling the second component to generate a sub-sampled second component;
sub-sampling the third component to generate a sub-sampled third component; and
generating an output data structure based on the first component, the sub-sampled second component, and the sub-sampled third component.
9 . The method of claim 8 , where:
the first color space image comprises interleaved red, green, and blue information;
the second color space image comprises brightness information and color information; and
where the method further comprises storing the brightness information separately from the color information.
10 . The method of claim 9 , where the method further comprises creating a first index pointer to the brightness information and a second index pointer to the color information.
11 . The method of claim 10 , further comprising fetching the brightness information or the color information in one continuous block.
12 . The method of claim 10 , where the method further comprises fetching and decoding the brightness information based on the first index pointer for a non-human operation.
13 . The method of claim 12 , where the non-human operation is selected from image searching, pattern recognition and biometric analysis.
14 . The method of claim 10 , where the method further comprises fetching and decoding the color information based on the second index pointer for a non-human application.
15 . An apparatus configured to efficiently access video data, comprising:
a codec;
a processor; and
a non-transitory processor readable medium comprising one or more instructions, where the one or more instructions when executed by the processor, cause the processor to:
receive a video data structure characterized by a first quality, the video data structure comprising a plurality of video components;
provide a subset of the plurality of video components to the codec to generate video data of a second quality different than the first quality; and
provide the video data to an application characterized by the second quality.
16 . The apparatus of claim 15 , where the first quality comprises a first display resolution and the second quality comprises a second display resolution.
17 . The apparatus of claim 15 , where the first quality comprises a first frame rate and the second quality comprises a second frame rate.
18 . The apparatus of claim 15 , where the first quality comprises a first color space encoding and the second quality comprises a second color space encoding.
19 . The apparatus of claim 15 , further comprising:
a video display; and
the application comprises a preview application for video editing or video playback.
20 . The apparatus of claim 15 , further comprising:
a network interface; and
the application performs an image search, pattern recognition, or biometric analysis.