Encoding system using motion vectors to represent frame-to-frame changes, wherein a decoder uses predictions of motion vectors in decoding
View Patent ↗Video data is encoded to form compressed video data wherein at least some of the frames are encoded as nonkey frames with reference to content of reference frames. In encoding a nonkey frame, segments of a reference frame can be matched to the pixels of the nonkey frame and related by a calculated motion vector for each segment, wherein a motion vector represents a difference in position of the image area represented by the segment. Motion vectors are predicted and the encoding of the nonkey frame can omit coding for calculated motion vectors that are sufficiently close to their predicted values and encoding differences between calculated and predicted motion vectors. Differences in values can be considered residue and coded as part of residue data. A decoder making similar predictions can decode the compressed video data even when the compressed video data contains no information about the predictions for motion vectors.
1. A method of encoding uncompressed video data to form compressed video data, wherein the video data comprises a sequence of a plurality of image frames, wherein at least some of the frames are encoded as nonkey frames that are encoded with reference to content of other reference frames, and wherein segmentation of the reference frames is performed, segmentation being an assignment of pixels of the reference frame to groups of pixels based on at least one of pixel color values of the pixels and location of the pixels in the reference frame, the method comprising:
encoding a first reference frame;
segmenting the first reference frame into a set of segments;
matching segments of the set of segments to pixels of a nonkey frame being encoded as a current frame;
determining, for each of a plurality of matched segments, an encoder calculated motion vector for the matched segment, wherein the encoder calculated motion vector represents a change of position between the matched segment and corresponding pixels of the current frame;
determining, for each of the plurality of matched segments, an encoder predicted motion vector for the matched segment from at least two previously encoded frames; and
encoding, for each of the plurality of matched segments, encoding a difference between the encoder calculated motion vector of the matched segment and the encoder predicted motion vector of the matched segment, while omitting encoding of a motion vector for the matched segment.
2. The method of claim 1 , wherein the encoding step further comprises, for each of the plurality of matched segments, encoding the encoder calculated motion vector of the matched segment instead of said difference when the difference between the encoder calculated motion vector and the encoder predicted motion is greater than a predetermined difference.
3. The method of claim 1 , further comprising a step of encoding unpredicted residue, wherein the unpredicted residue represents a difference between a frame represented by segments of the first reference frame modified according to the predicted motion vectors and the current frame.
4. The method of claim 1 , further comprising a decoder for decompressing the video data, the decoder performing a method comprising:
decoding a first reference frame;
segmenting the first reference frame into a second set of segments;
matching the segments of the second set of segments to pixels of a nonkey frame being decoded as the current frame;
determining, for each of the second set of segments, a decoder predicted motion vector for the matched segment from at least two previously decoded frames; and
correcting each of the decoder predicted motion vectors by the difference between the encoder calculated motion vector of the matched segment and the encoder predicted motion of the matched segment.
5. The method of claim 4 , wherein the at least two previously decoded frames includes the content of an intermediate frame between the first reference frame and the current frame.
6. The method of claim 1 , wherein the motion vectors are encoded as a hierarchy of motion vectors.
7. The method of claim 1 , wherein determining a predicted motion vector comprises comparing matched segments over a plurality of frames and predicting the predicted motion vector from an expected inertial movement of the matched segment.
8. An apparatus configured to encode uncompressed video data to form compressed video data, wherein the video data comprises a sequence of a plurality of image frames, wherein at least some of the frames are encoded as nonkey frames that are encoded with reference to content of other reference frames, and wherein segmentation of the reference frames is performed, segmentation being an assignment of pixels of the reference frame to groups of pixels, the method comprising:
a frame compressor configured to encode a first reference frame;
a segmenter configured to segment the first reference frame into a set of segments;
a motion matcher configured to match segments of the set of segments to pixels of a nonkey frame being encoded as a current frame, and further configured to determine, for each of a plurality of matched segments, an encoder calculated motion vector for the matched segment, wherein the encoder calculated motion vector represents a change of position between the matched segment and corresponding pixels of the current frame; and
a residue generator configured to determine, for each of the plurality of matched segments, an encoder predicted motion vector for the matched segment from at least two previously encoded frames, and further configured to generate, for each of the plurality of matched segments, a difference between the encoder calculated motion vector of the matched segment and the encoder predicted motion vector of the matched segment.
9. The apparatus of claim 8 , wherein the apparatus is further configured to encode, for each of the plurality of matched segments, the encoder calculated motion vector of the matched segment when the difference between the encoder calculated motion vector and the encoder predicted motion is greater than a predetermined difference.
10. The apparatus of claim 8 , wherein the apparatus is further configured to encode unpredicted residue, wherein the unpredicted residue represents a difference between a frame represented by segments of the first reference frame modified according to the predicted motion vectors and the current frame.