Adaptive joint source channel coding
View Patent ↗Adaptive joint source channel coding associates multiple predictors with a reference data unit, such as a macroblock or frame of video data. An encoder determines a sub-codebook in which each of the selected multiple predictors decodes to the reference data unit. An identifier for the sub-codebook is transmitted through a channel to a decoder for subsequent decoding of the reference data unit. The reference data unit itself does not need to be sent. The multiple predictors are contained within a decoding region and the identifier for the sub-codebook specifies the decoding region. The decoder uses the identified sub-codebook and one of the predictors to decode the reference data unit. If none of the original predictors are correctly received, different types of error handling are employed based on the type of channel.
1. A computerized method comprising:
selecting, by an encoding device, a first set of multiple predictors for a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the predictors for the single reference unit are selected from data from a neighboring camera;
determining a sub-codebook in which each of the first set of multiple predictors decodes to the single reference unit;
transmitting an identifier for the sub-codebook to decode the single reference unit from one of the predictors in the first set of multiple predictors, wherein only one of the predictors in the first set of multiple predictors is needed to correctly decode the single reference unit;
determining that none of the predictors in the first set of multiple predictors was received; and
transmitting a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set and wherein only one of the predictors in the second set of multiple predictors is needed to correctly decode the single reference unit.
2. The computerized method of claim 1 further comprising:
transmitting the reference unit.
3. The computerized method of claim 1 further comprising:
receiving the identifier; and
decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.
4. The computerized method of claim 1 , wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.
5. A machine-readable non-transitory storage medium having instructions to cause a processor to execute a method comprising:
selecting a first set of multiple predictors for a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the predictors for the single reference unit are selected from data from a neighboring camera;
determining a sub-codebook in which each of the first set of multiple predictors decodes to the single reference unit;
transmitting an identifier for the sub-codebook to decode the single reference unit from one of the predictors in the first set of multiple predictors, wherein only one of the predictors in the first set of multiple predictors is needed to correctly decode the single reference unit;
determining that none of the predictors in the first set of multiple predictors was received; and
transmitting a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set and wherein only one of the predictors in the second set of multiple predictors is needed to correctly decode the single reference unit.
6. The machine-readable non-transitory storage medium of claim 5 having instructions to cause a processor to execute a method comprising, wherein the method further comprises:
receiving the identifier; and
decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.
7. The machine-readable non-transitory storage medium of claim 5 having instructions to cause a processor to execute a method comprising, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.
8. A machine-readable non-transitory storage medium having instructions to cause a processor to execute a method comprising:
receiving an identifier corresponding to a sub-codebook comprising a first set of multiple predictors, wherein each of the predictors in the first set of multiple predictors decodes to a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the first set of multiple predictors for the single reference unit are selected from data from a neighboring camera;
determining that none of the predictors in the first set of multiple predictors was received;
receiving a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set; and
decoding the single reference unit from one correctly received predictor from the second set of multiple predictors using the sub-codebook identified by the supplemental identifier, wherein only one of the second set of multiple predictors is needed to correctly decode the single reference unit.
9. The machine-readable non-transitory storage medium of claim 8 having instructions to cause a processor to execute a method comprising, wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.
10. A system comprising:
a processor coupled to a memory through a bus;
an encoding process executed by the processor from the memory to cause the processor to select a first set of multiple predictors for a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the first set of multiple predictors for the single reference unit are selected from data from a neighboring camera, determine a sub-codebook in which each of the first set of multiple predictors decodes to the single reference unit, and transmit an identifier for the sub-codebook to decode the single reference unit from one of the first set of multiple predictors, wherein only one of the first set of multiple predictors is needed to correctly decode the single reference unit;
a determining process executed by the processor from the memory to cause the processor to determine that none of the predictors in the first set of multiple predictors was received; and
a transmitting process executed by the processor from the memory to cause the processor to transmit a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set and wherein only one of the predictors in the second set of multiple predictors is needed to correctly decode the single reference unit.
11. The system of claim 10 , wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.
12. A system comprising:
a processor coupled to a memory through a bus; and
a first receiving process executed from the memory by the processor to cause the processor to receive an identifier corresponding to a sub-codebook comprising a first set of multiple predictors, wherein each of the first set of multiple predictors decodes to a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the first set of multiple predictors for the single reference unit are selected from data from a neighboring camera;
a determining process executed from the memory by the processor to cause the processor to determine that none of the predictors in the first set of multiple predictors was received;
a second receiving process executed from the memory by the processor to cause the processor to receive a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set; and
a decoding process executed from the memory by the processor to cause the processor to decode the single reference unit from one correctly received predictor using the sub-codebook identified by the supplemental identifier, wherein only one of the second set of multiple predictors is needed to correctly decode the single reference unit.
13. The system of claim 12 , wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.
14. An apparatus comprising:
means for selecting a first set of multiple predictors for a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the predictors for the single reference unit are selected from data from a neighboring camera;
means for determining a sub-codebook in which each of the first set of multiple predictors decodes to the single reference unit;
means for transmitting an identifier for the sub-codebook to decode the single reference unit from one of the predictors in the first set of multiple predictors, wherein only one of the predictors in the first set of multiple predictors is needed to correctly decode the single reference unit;
means for determining that none of the predictors in the first set of multiple predictors was received; and
means for transmitting a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set and wherein only one of the predictors in the second set of multiple predictors is needed to correctly decode the single reference unit.
15. The apparatus of claim 14 further comprising:
means for receiving the identifier; and
means for decoding the reference unit from a correctly received predictor using the sub-codebook identified by the identifier.
16. The apparatus of claim 14 , wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.
17. An apparatus comprising:
means for receiving an identifier corresponding to a sub-codebook comprising a first set of multiple predictors, wherein each of the predictors in the first set of multiple predictors decodes to a single reference unit of correlated data, wherein the single reference unit is selected from data from a camera and the first set of multiple predictors for the single reference unit are selected from data from a neighboring camera;
means for determining that none of the predictors in the first set of multiple predictors was received;
means for receiving a supplemental identifier for a sub-codebook in which each of a second set of multiple predictors decodes to the single reference unit, wherein the second set of multiple predictors includes the first set of multiple predictors and includes additional predictors not present in the first set; and
means for decoding the single reference unit from one correctly received predictor from the second set of multiple predictors using the sub-codebook identified by the supplemental identifier, wherein only one of the second set of multiple predictors is needed to correctly decode the single reference unit.
18. The apparatus of claim 17 , wherein the reference unit is a frame in a video stream from a camera and the predictors for the reference unit are selected from frames in a video stream from a neighboring camera.