Sub-band video coding architecture for packet based transmission
Sub-band video coding architecture for packet based transmission. Video processing includes converting of a frame or picture of a video signal into a number of respective sub-bands or sub-frames (e.g., such that each respective one is a respective subsampled version of an original frame or picture) and subsequent assembly thereof into a reconstructed frame or picture. The reconstructed frame or picture then undergoes video encoding to generate an output bitstream that may subsequently undergo appropriate processing (e.g., packetization, continuous time signal generation, etc.) to generate a signal for transmission via one or more communication the links. Characteristics associated with each respective sub-band or sub-frame may be similar (e.g., similar characteristics) or different (e.g., two or more sub-bands or sub-frames have different characteristics, such as different respective number of pixels, aspect ratios, etc.). Respective sub-frames may undergo video encoding and transmission.
1. A communication device comprising:
a communication interface; and
a processing circuit, the processing circuit and the communication interface configured to:
sub-sample a frame to generate a plurality of sub-frames, wherein each of the plurality of sub-frames is a respective sub-sampled version of a full version of the frame, wherein a first sub-frame of the plurality of sub-frames is a first sub-sampled version of the full version of the frame based on a first pixel marker, and a second sub-frame of the plurality of sub-frames is a second sub-sampled version of the full version of the frame based on a second pixel marker that is offset from the first pixel marker;
assemble the plurality of sub-frames to generate a reconstructed frame having pixels differentially ordered than the frame, wherein the reconstructed frame includes the first sub-frame and the second sub-frame adjacently arranged; and
encode the reconstructed frame to generate at least a portion of an output bit stream; and
transmit the output bit stream to another communication device.
2. The communication device of claim 1 , wherein the frame and the reconstructed frame each include a common number of pixels.
3. The communication device of claim 1 , wherein the processing circuit and the communication interface are further configured to:
assemble the plurality of sub-frames with at least one padded pixel to generate the reconstructed frame, wherein the reconstructed frame includes the first sub-frame, the second sub-frame, and the at least one padded pixel adjacently arranged.
4. The communication device of claim 1 , wherein the processing circuit and the communication interface are further configured to:
packetize the output bit stream to generate a plurality of packets for transmission to the another communication device via a lossy communication channel.
5. The communication device of claim 1 , wherein:
the first sub-frame includes a first number of pixels and a first aspect ratio; and
the second sub-frame includes a second number of pixels and a second aspect ratio.
6. The communication device of claim 1 , wherein the processing circuit and the communication interface are further configured to:
sub-sample the frame to generate the plurality of sub-frames using a pixel mask applied based on the first pixel marker and the second pixel marker.
7. The communication device of claim 1 , wherein the processing circuit and the communication interface are further configured to:
perform motion estimation processing based on only a portion of the reconstructed frame corresponding to the first sub-frame or the second sub-frame to generate at least one motion vector for use in both the first sub-frame and the second sub-frame.
8. The communication device of claim 1 , wherein the processing circuit and the communication interface are further configured to:
support communications within at least one of a satellite communication system, a wireless communication system, a wired communication system, a fiber-optic communication system, or a mobile communication system.
9. The communication device of claim 1 , wherein the reconstructed frame includes the first sub-frame of the plurality of sub-frames in a first portion of the reconstructed frame and the second sub-frame of the plurality of sub-frames in a second portion of the reconstructed frame that is non-overlapping and different than the first portion of the reconstructed frame.
10. The communication device of claim 1 , wherein the processing circuit and the communication interface are further configured to:
sub-sample the frame using a first at least one parameter to generate the first sub-frame of the plurality of sub-frames; and
sub-sample the frame using a second at least one parameter to generate the second sub-frame of the plurality of sub-frames.
11. The communication device of claim 1 , wherein the processing circuit and the communication interface are further configured to:
packetize the output bit stream to generate a plurality of packets for transmission, to the another communication device, wherein each packet of the plurality of packets includes a respective header and a respective payload.
12. A method for execution by a communication device, the method comprising:
sub-sampling a frame to generate a plurality of sub-frames, wherein each of the plurality of sub-frames is a respective sub-sampled version of a full version of the frame, wherein a first sub-frame of the plurality of sub-frames is a first sub-sampled version of the full version of the frame, and a second sub-frame of the plurality of sub-frames is a second sub-sampled version of the full version of the frame;
assembling the plurality of sub-frames to generate a reconstructed frame having pixels differentially ordered than the frame, wherein the reconstructed frame includes the first sub-frame and the second sub-frame adjacently arranged; and
encoding the reconstructed frame to generate at least a portion of an output bit stream; and
transmitting, via a communication interface of the communication device, the output bit stream to another communication device.
13. The method of claim 12 further comprising:
assembling the plurality of sub-frames with at least one padded pixel to generate the reconstructed frame, wherein the reconstructed frame includes the first sub-frame, the second sub-frame, and the at least one padded pixel adjacently arranged.
14. The method of claim 12 further comprising:
packetizing the output bit stream to generate a plurality of packets; and
transmitting, via the communication interface of the communication device, the plurality of packets to the another communication device via a lossy communication channel.
15. The method of claim 12 , wherein:
the first sub-frame includes a first number of pixels and a first aspect ratio; and
the second sub-frame includes a second number of pixels and a second aspect ratio.
16. The method of claim 12 further comprising:
sub-sampling the frame to generate the plurality of sub-frames in accordance with a pixel mask applied based on a first pixel marker and a second pixel marker, wherein the first sub-frame of the plurality of sub-frames is the first sub-sampled version of the full version of the frame based on the first pixel marker, and the second sub-frame of the plurality of sub-frames is the second sub-sampled version of the full version of the frame based on the second pixel marker that is offset from the first pixel marker.
17. The method of claim 12 further comprising:
performing motion estimation processing based on only a portion of the reconstructed frame corresponding to the first sub-frame or the second sub-frame to generate at least one motion vector for use in both the first sub-frame and the second sub-frame.
18. The method of claim 12 , wherein the communication device is operative within at least one of a satellite communication system, a wireless communication system, a wired communication system, a fiber-optic communication system, or a mobile communication system.
19. The method of claim 12 , wherein the reconstructed frame includes the first sub-frame of the plurality of sub-frames in a first portion of the reconstructed frame and the second sub-frame of the plurality of sub-frames in a second portion of the reconstructed frame that is non-overlapping and different than the first portion of the reconstructed frame.
20. The method of claim 12 further comprising:
sub-sampling the frame using a first at least one parameter to generate the first sub-frame of the plurality of sub-frames; and
sub-sampling the frame using a second at least one parameter to generate the second sub-frame of the plurality of sub-frames.