Bidirectionally decodable Wyner-Ziv video coding
View Patent ↗Systems and methodologies for employing bidirectionally decodable Wyner-Ziv video coding (BDWZVC) are described herein. BDWZVC can be used to generate M-frames, which have multiple reference frames at an encoder and can be forward and backward decodable. For example, optimal Lagrangian multipliers for forward and backward motion estimation can be derived and/or utilized. The optimal Lagrangian multiplier for backward motion estimation can be approximately twice as large as the optimal Lagrangian multiplier for forward motion estimation. Further, an optimal P-frame/M-frame selection scheme can be employed to enhance rate-distortion performance when video is transmitted over an error prone channel. Accordingly, a first frame in a group of pictures (GOP) can be encoded as an I-frame, a next m−1 frames can be encoded as P-frames, and a remaining n−m frames can be encoded as M-frames, where n can be a length of the GOP and m can be optimally identified.
1. A system, comprising:
a bidirectional encoder configured to encode video data using bidirectionally decodable Wyner-Ziv video coding to generate encoded video frames for transfer over a channel, wherein the encoded video frames include at least one M-frame, and the at least one M-frame is a P-frame with multiple decoder selectable reference frames at the bidirectional encoder, and wherein the bidirectional encoder includes an encoder comprising a rate determiner configured to identify a rate of an error correction code using a two-pass algorithm that simulates forward decoding and backward decoding, and the rate determiner is further configured to encode and decode frames in a current group of pictures in temporal order for a first pass and encode and decode frames in the current group of pictures in reverse temporal order for a second pass.
2. The system of claim 1 , wherein the at least one M-frame includes encoded difference information based upon comparisons with a previous frame and a subsequent frame, wherein the previous frame is one of an I-frame, a P-frame or an M-frame, whichever is closest to the at least one M-frame, and the subsequent frame is one of an I-frame, the P-frame or an M-frame, whichever is closest to the at least one M-frame.
3. The system of claim 1 , wherein the bidirectional encoder further comprises:
a transformer configured to perform an integer cosine transform (ICT) on a current frame to generate different bands of coefficients; and
a quantizer configured to quantize the different bands of coefficients into a plurality of bitplanes, wherein the encoder is further
configured to encode the plurality of bitplanes based on the error correction code.
4. The system of claim 3 , wherein the error correction code is at least one of a low density parity check (LDPC) code or a Turbo code.
5. The system of claim 1 , wherein the rate determiner is further configured to identify the rate of the error correction code by estimation of joint statistics of a current frame, a last frame prior to the current frame, and a next frame subsequent to the current frame.
6. The system of claim 1 , wherein the rate determiner is further configured to employ motion estimation for the at least one M-frame.
7. The system of claim 6 , further comprising a motion estimation optimizer configured to enable the rate determiner to utilize selected Lagrangian multipliers in connection with the motion estimation.
8. The system of claim 7 , wherein the motion estimation optimizer is configured to minimize a Lagrangian cost function to determine the selected Lagrangian multipliers.
9. The system of claim 8 , wherein the Lagrangian cost function is substantially based on J ME =D DFD +λR ME , wherein a prediction distortion D DFD (displaced frame difference) is weighted against a motion vector cost, R ME , using a Lagrangian multiplier, λ.
10. The system of claim 1 , further comprising a frame type selector configured to determine a frame type to be utilized for the encoded video frames in a group of pictures.
11. The system of claim 10 , wherein the frame type selector is configured to identify a combination of M-frames and P-frames to include in the group of pictures between adjacent I-frames based upon an expected distortion reduction.
12. The system of claim 10 , wherein the frame type selector is configured to determine that a first frame in the group of pictures is an I-frame, a next m−1 frames are P-frames, and a remaining n−m frames are M-frames, wherein n is a length of the group of pictures, and m is identified by the frame type selector and is less than or equal to n.
13. A method, comprising:
performing an integer cosine transform on a current frame to generate bands of coefficients;
quantizing the bands of coefficients into a plurality of bitplanes;
determining a rate for use with an error correction code using a two-pass algorithm;
simulating forward decoding and backward decoding as at least part of the two-pass algorithm by performing motion estimation utilizing selected Lagrangian multipliers;
encoding the plurality of bitplanes employing the error correction code with the rate; and
encoding and decoding frames in a current group of pictures in temporal order for a first pass and encoding and decoding frames in the current group of pictures in reverse temporal order for a second pass.
14. The method of claim 13 , wherein the determining includes determining the rate for use with at least one of a low density parity check (LDPC) code or a Turbo code.
15. The method of claim 13 , wherein the performing the motion estimation comprises minimizing a Lagrangian cost function including weighing predicted distortion against motion vector cost using the selected Lagrangian multipliers.
16. The method of claim 13 , further comprising determining the at least one Lagrangian multiplier for forward motion estimation and at least one Lagrangian multiplier for backward motion estimation.
17. The method of claim 16 , wherein the at least one Lagrangian multiplier for backward motion estimation is approximately twice the at least one Lagrangian multiplier for forward motion estimation.
18. A computer-readable storage non-transitory medium comprising computer-executable instructions that, in response to execution, cause a computing system to perform operations, comprising:
encoding a first frame in a group of pictures as an I-frame, wherein the group of pictures includes n frames;
encoding a next m−1 frames after the I-frame in the group of pictures as P-frames, wherein m is less than or equal to n;
encoding a remaining n−m frames in the group of pictures as M-frames, wherein the M-frames are P-frames with multiple decoder selectable reference frames at a bidirectional encoder;
identifying a rate of an error correction code using a two-pass algorithm that simulates forward decoding and backward decoding;
encoding and decoding frames in a current group of pictures in temporal order for a first pass; and
encoding and decoding frames in the current group of pictures in reverse temporal order for a second pass.
19. The computer-readable storage non-transitory medium of claim 18 , the operations further comprising determining a value of m as a function of at least: distortion reduction of using a P-frame rather than an M-frame in response to a current frame being forward decodable, distortion increase of using a P-frame in response to the P-frame not being forward decodable and the M-frame being backward decodable, and a frame loss rate.
20. The computer-readable storage non-transitory medium of claim 19 , wherein the determining the value of in includes determining a value wherein an expected distortion reduction of encoding frame in as a P-frame versus encoding frame in as an M-frame is greater than or equal to zero and wherein m+1 provides an expected distortion reduction of encoding frame m+1 as a P-frame versus encoding frame m+1 as an M-frame less than zero.