Packetization of FGS/PFGS video bitstreams
View Patent ↗A video encoding system performs packetization of FGS/PFGS encoded video bitstreams by selecting encoded bitstream segments for packetization based on an estimated total contribution to distortion reduction associated with each encoded bitstream segment. The selected bitstream segments are then packetized according to a packet-independence packetization strategy that minimizes inter-packet dependency.
1. A method comprising:
generating a plurality of encoded bitstream segments by applying a layered coding scheme to a video stream;
calculating a contribution to distortion reduction associated with each of the plurality of encoded video bitstream segments according to a formula:
Δ D ( f,l,i )=Δ D 0 ( f,l,i )+Δ D dr ( f,l,i )
wherein:
ΔD 0 (f,l,i) represents a distortion reduction of a bitstream segment associated with an f th frame, an l th bitplane, and an i th macroblock; and
ΔD dr (f,l,i) represents a drifting suppression of the bitstream segment associated with the f th frame l th bitplane, and i th macroblock;
selecting multiple bitstream segments from the plurality of encoded bitstream segments based on a comparison of the contribution to distortion reduction associated with individual ones of the encoded bitstream segments; and
packetizing the multiple bitstream segments into data packets such that a first data packet contains bitstream segments associated with multiple bit-planes of a first macro block, and such that no other one of the data packets contains bitstream segments associated with the first macro block.
2. The method as recited in claim 1 wherein the layered coding scheme comprises a fine granularity scalable coding scheme.
3. The method as recited in claim 1 wherein the layered coding scheme comprises a progressive fine granularity scalable coding scheme.
4. The method as recited in claim 3 wherein the progressive fine granularity scalable coding scheme is frame-based.
5. The method as recited in claim 3 wherein the progressive fine granularity scalable coding scheme is macroblock based.
6. The method as recited in claim 1 wherein the selecting is further based on an available bandwidth.
7. The method as recited in claim 1 wherein the selecting comprises:
determining a bitrate associated with individual ones of the plurality of encoded bitstream segments;
determining an available bandwidth;
and selecting the multiple bitstream segments such that for each bitstream segment that is selected, the associated bitrate is supported by the available bandwidth and the associated contribution to distortion reduction is greater than or equal to a contribution to distortion reduction associated with at least one of the plurality of bitstream segments that is not selected.
8. The method as recited in claim 1 wherein the first data packet also contains bitstream segments associated with multiple bit-planes of a second macro block.
9. A method comprising:
generating a plurality of encoded bitstream segments by applying a layered coding scheme to a video stream;
selecting multiple bitstream segments from the plurality of encoded bitstream segments;
representing multiple enhancement layers associated with a frame as a binary tree wherein each node of the tree represents a particular one of the multiple bitstream segments, and wherein each child node of the binary tree is aligned with an edge of a parent node; and
generating multiple data packets such that each data packet that contains a child node also contains a parent node associated with the child node.
10. A method comprising:
identifying a plurality of encoded video bitstream segments to be packetized;
arranging the encoded video bitstream segments according to a binary tree that represents enhancement layers of a frame; and
packetizing the encoded video bitstream segments into data packets according to the binary tree such that each data packet comprises a parent node and a child node of the binary tree.
11. The method as recited in claim 10 further comprising:
pruning the binary tree to remove individual bitstream segments according to a distortion reduction contribution threshold.
12. A method comprising:
calculating a contribution to distortion reduction associated with each of a plurality of encoded video bitstream segments according to a formula:
Δ D ( f,l,i )=Δ D 0 ( f,l,i )+Δ D dr ( f,l,i )
wherein:
ΔD 0 (f,l,i) represents a distortion reduction of a bitstream segment associated with an f th frame, an l th bitplane, and an i th macroblock; and
ΔD dr (f,l,i) represents a drifting suppression of the bitstream segment associated with the f th frame, l th bitplane, and i th macroblock;
identifying for packetization, a subset of the plurality of encoded video bitstream segments based on a comparison of the contribution to distortion reduction; and
packetizing the subset of the plurality of encoded bitstream segments into data packets such that a first data packet contains bitstream segments associated with multiple bit-planes of a first macro block, and such that no other one of the data packets contains bitstream segments associated with the first macro block.
13. The method as recited in claim 12 wherein the packetizing further comprises:
raster-scan ordering the subset of the plurality of encoded bitstream segments in the first data packet according to the distortion reduction contribution of each bitstream segment associated with the first macro block.
14. A system comprising:
an encoder configured to generate encoded bitstream segments;
a bitstream selector configured to select a set of the encoded bitstream segments for packetization; and
a packet generator configured to packetize the set of the encoded bitstream segments according to a binary tree packetization strategy that minimizes dependency between packets.
15. The system as recited in claim 14 , wherein the encoder is implemented as a fine granularity scalable encoder.
16. The system as recited in claim 14 , wherein the encoder is implemented as a progressive fine granularity scalable encoder.
17. The system as recited in claim 14 , wherein the encoder is implemented as a frame-based progressive fine granularity scalable encoder.
18. The system as recited in claim 14 , wherein the encoder is implemented as a macroblock-based progressive fine granularity scalable encoder.
19. The system as recited in claim 14 , wherein the encoder is further configured to gather data describing a position, a required bitrate, and a contribution to distortion reduction associated with an encoded bitstream segment.
20. A system comprising:
means for encoding streaming video data using a layered coding scheme, resulting in a plurality of encoded bitstream segments;
means for selecting a set of the encoded bitstream segments based on a contribution to distortion reduction associated with individual ones of the plurality of encoded bitstream segments, wherein the contribution to distortion reduction is calculated according to a formula:
Δ D ( f,l,i )=Δ D 0 ( f,l,i )+Δ D dr ( f,l,i )
wherein:
ΔD 0 (f,l,i) represents a distortion reduction of a bitstream segment associated with an f th frame, an l th bitplane, and an i th macroblock; and
ΔD dr (f,l,i) represents a drifting suppression of the bitstream segment associated with the f th frame, l th bitplane, and i th macroblock; and
means for generating data packets containing the set of encoded bitstream segments such that a first data packet contains encoded bitstream segments associated with multiple bit-planes of a first macro block, and such that no other one of the data packets contains bitstream segments associated with the first macro block, so that_the packetized bitstream segments are arranged to reduce dependency between individual ones of the data packets.
21. The system as recited in claim 20 wherein the generating means further comprises:
means for organizing the set of encoded bitstream segments within the data packets based on contributions to distortion reduction associated with individual encoded bitstream segments of the set of encoded bitstream segments.
22. The system as recited in claim 20 wherein the generating means further comprises:
means for organizing the set of encoded bitstream segments within the data packets in a decreasing order based on contributions to distortion reduction associated with individual encoded bitstream segments of the set of encoded bitstream segments.
23. The system as recited in claim 20 further comprising:
means for selecting the set of encoded bitstream segments based on an available bandwidth.
24. One or more computer-readable media comprising computer executable instructions that, when executed, direct a computing system to:
calculate the contribution to distortion reduction associated with each of a plurality of encoded video bitstream segments according to a formula:
Δ D ( f,l,i )=Δ D 0 ( f,l,i )+Δ D dr ( f,l,i )
wherein:
ΔD 0 (f,l,i) represents a distortion reduction of a bitstream segment associated with an f th frame, an l th bitplane, and an i th macroblock; and
ΔD dr (f,l,i) represents a drifting suppression of the bitstream segment associated with the f th frame, l th bitplane, and i th macroblock;
determine a subset of the plurality of encoded video bitstream segments based on a comparison of the contribution to distortion reduction; and
generate a data packet containing the selected subset of video bitstream segments.
25. The one or more computer-readable media as recited in claim 24 further comprising computer executable instructions that, when executed, direct a computing system to:
calculate the drifting suppression according to a formula:
Δ
D
d
r
(
f
,
l
,
i
)
=
Δ
D
0
(
f
,
l
,
i
)
×
∑
n
=
1
N
[
∏
k
=
1
n
R
HH
(
f
+
k
)
+
∏
k
=
1
n
-
1
R
HH
(
f
+
k
)
×
R
HL
(
f
+
n
)
]
wherein:
ΔD 0 (f,l,i) represents a distortion reduction of a bitstream segment associated with an f th frame, an l th bitplane, and an i th macroblock;
R HH (f) represents a percentage of high-prediction-high-reference (HPHR) modes in an f th frame;
R HL (f) represents a percentage of high-prediction-low-reference (HPLR) modes in an f th frame; and
N is a number of frames used for drifting estimation.
26. The one or more computer-readable media as recited in claim 24 , wherein each of the plurality of encoded video bitstream segments is encoded according to a layered coding scheme.
27. The method as recited in claim 9 wherein the layered coding scheme comprises a fine granularity scalable coding scheme.
28. The method as recited in claim 9 wherein the selecting comprises:
selecting multiple bitstream segments from the plurality of encoded bitstream segments based on an available bandwidth.
29. The method as recited in claim 9 wherein the selecting comprises:
selecting multiple bitstream segments based on contributions to distortion reduction associated with individual ones of the encoded bitstream segments.
30. The method as recited in claim 9 wherein the selecting comprises:
determining a bitrate associated with individual ones of the plurality of encoded bitstream segments;
estimating a contribution to distortion reduction associated with individual ones of the plurality of encoded bitstream segments;
determining an available bandwidth; and
selecting the multiple bitstream segments such that for each bitstream segment that is selected, the associated bitrate is supported by the available bandwidth and the associated contribution to distortion reduction is greater than or equal to a contribution to distortion reduction associated with at least one of the plurality of bitstream segments that is not selected.
31. The method as recited in claim 1 wherein the packetizing further comprises:
raster-scan ordering the bitstream segments in the first data packet according to the distortion reduction contribution of each bitstream segment associated with the first macro block.