System and method for low-latency multimedia streaming
View Patent ↗Systems and methods for transmitting a multimedia stream over a communication link on a network are disclosed. The systems and methods adaptively adjust encoding parameters based on monitoring changing conditions of the network. A transmitter includes an adaptive-rate encoder that adaptively adjusts a video encoding bit rate in response to changing conditions of the communication link. The encoder maintains tight rate control by utilizing slice processing and sub-frame rate adaptation, as well as maintaining a headroom between the channel bit rate and the video encoding bit rate. The adaptive-rate encoder also embeds intra-frame constraints in predictive frames traffic in order to reduce latency.
1. A method of transmitting a multimedia stream over a network, comprising:
receiving a multimedia stream from a source, the multimedia stream comprising video data;
determining a first set of video encoding parameters;
encoding a first portion of the video data into a first encoded video slice using the first set of video encoding parameters;
transmitting the first encoded video slice over a communication link of the network to a receiver;
monitoring conditions of the communication link;
determining a second set of video encoding parameters based on the conditions of the communication link;
encoding a second portion of the video data into a second encoded video slice using the second set of video encoding parameters;
transmitting the second encoded video slice over the communication link to the receiver,
wherein the first encoded video slice and the second encoded video slice each comprise a series of macroblocks constituting a segment of a frame, wherein each segment of the frame is an independently decodable unit that is processed in parallel with other segments of the frame to establish sub-frame level processing; and
adjusting an encoded video bit rate at the sub-frame level to allow for rapid adaptation to changing channel conditions and to minimize latency.
2. The method of claim 1 , wherein determining a first set of video encoding parameters comprises:
determining a current channel bit rate (cur_CBR) of the communication link;
determining a current headroom factor cur_α, wherein cur_α is the difference between the current channel bit rate and an encoded video data rate; and
setting a current video bit rate (cur_VBR) for encoding the multimedia stream such that the cur_CBR−cur_VBR≧cur_α.
3. The method of claim 2 , wherein determining a second set of video encoding parameters based on the conditions of the communication link comprises:
monitoring the communication link to determine a new channel bit rate (new_CBR);
determining a channel bit rate reduction β, wherein β=cur_CBR−new_CBR; and
adjusting cur_VBR if β cur_α.
4. The method of claim 2 , wherein determining a second set of video encoding parameters based on the conditions of the communication link comprises:
monitoring the communication link to determine a new channel bit rate (new_CBR);
determining a channel bit rate reduction β, wherein β=cur_CBR−new_CBR; and
adjusting cur_α if β≧cur_α.
5. The method of claim 2 , wherein determining a second set of video encoding parameters based on the conditions of the communication link comprises:
monitoring the communication link to determine a new channel bit rate (new_CBR);
determining a channel bit rate reduction β, wherein β=cur_CBR−new_CBR;
determining a local maximum drop in channel bit rate loc_max_β, wherein loc_max β is the largest β observed over a time period t; and
adjusting one or more of cur_VBR and cur_α if loc_max β≦a predetermined threshold β_min or if t≧a predetermined threshold t_max.
6. The method of claim 2 , wherein determining a second set of video encoding parameters based on the conditions of the communication link comprises:
monitoring the communication link to determine a new channel bit rate (new_CBR);
determining a channel bit rate reduction β wherein β=cur_CBR−new_CBR;
determining an average drop in channel bit rate avg_β, wherein avg_β is the average β observed over a time period t; and
adjusting one or more of cur_VBR and cur_α if avg_β≦a predetermined threshold β_min or if t≧a predetermined threshold t_max.
7. A method of encoding a multimedia stream for transmission over a network comprising:
determining a current channel bit rate (cur_CBR) of a communication link on the network;
determining a current headroom factor cur_α, wherein cur_α is the difference between the current channel bit rate and an encoded video data rate;
setting a current video bit rate (cur_VBR) for encoding the multimedia stream such that the cur_CBR−cur_VBR≧cur_α;
monitoring the communication link to determine a new channel bit rate (new_CBR);
determining a channel bit rate reduction β, wherein β=cur_CBR−new_CBR; and
adjusting cur_VBR if β>cur_α;
wherein the multimedia stream includes a series of macroblocks constituting a segment of a frame, wherein each segment of the frame is an independently decodable unit that is processed in parallel with other segments of the frame to establish sub-frame level processing; and
adjusting an encoded video bit rate at the sub-frame level to allow for rapid adaptation to changing channel conditions and to minimize latency.
8. The method of claim 7 , further comprising:
adjusting cur_α if β>cur_α.
9. The method of claim 7 , further comprising:
determining a local maximum drop in channel bit rate loc_max_β wherein loc_max_β is the largest β observed over a time period t; and
adjusting one or more of cur_VBR and cur_α if loc_max_β≦a predetermined threshold β_min or if t≧a predetermined threshold t_max.
10. The method of claim 7 , further comprising:
determining an average drop in channel bit rate avg_β, wherein avg_β is the average β observed over a time period t; and
adjusting one or more of cur_VBR and cur_α if avg_β≦a predetermined threshold β_min or if t≧a predetermined threshold t_max.
11. A system for transmitting a multimedia stream over a network comprising:
a transmitter adapted to perform the following:
receive a multimedia stream from a source, the multimedia stream comprising video data;
determine a first set of video encoding parameters;
encode a first portion of the video data into a first encoded video slice using the first set of video encoding parameters;
transmit the first encoded video slice over a communication link of the network to a receiver;
monitor conditions of the communication link;
determine a second set of video encoding parameters based on the conditions of the communication link;
encode a second portion of the video data into a second encoded video slice using the second set of video encoding parameters; and
transmit the second encoded video slice over the communication link to the receiver,
wherein the first encoded video slice and the second encoded video slice each comprise a series of macroblocks constituting a segment of a frame, wherein each segment of the frame is an independently decodable unit that is processed in parallel with other segments of the frame to establish sub-frame level processing; and
wherein the transmitter adjusts an encoded video bit rate at the sub-frame level to allow for rapid adaptation to changing channel conditions and to minimize latency.
12. The system of claim 11 , wherein the transmitter is further adapted to perform the following:
determine a current channel bit rate (cur_CBR) of the communication link;
determine a current headroom factor cur_α; and
set a current video bit rate (cur_VBR) for encoding the multimedia stream such that the cur_CBR−cur_VBR≧cur_α.
13. The system of claim 12 , wherein the transmitter is further adapted to perform the following:
monitor the communication link to determine a new channel bit rate (new_CBR);
determine a channel bit rate reduction β, wherein β=cur_CBR−new_CBR; and
adjust cur_VBR if β>cur_α.
14. The system of claim 12 , wherein the transmitter is further adapted to perform the following:
monitor the communication link to determine a new channel bit rate (new_CBR);
determine a channel bit rate reduction β, wherein β=cur_CBR−new_CBR; and
adjust cur_α if β>cur_α.
15. The system of claim 12 , wherein the transmitter is further adapted to perform the following:
monitor the communication link to determine a new channel bit rate (new_CBR);
determine a channel bit rate reduction β, wherein β=cur_CBR−new_CBR;
determine a local maximum drop in channel bit rate loc_max_β, wherein loc_max_β is the largest β observed over a time period t; and
adjust one or more of cur_VBR and cur_α if loc_max_β≦a predetermined threshold min or if t≧a predetermined threshold t_max.
16. The system of claim 12 , wherein the transmitter is further adapted to perform the following:
monitor the communication link to determine a new channel bit rate (new_CBR);
determine a channel bit rate reduction β, wherein β=cur_CBR−new_CBR;
determine an average drop in channel bit rate avg_β, wherein avg_β is the average β observed over a time period t; and
adjust one or more of cur_VBR and cur_α if avg_β≦a predetermined threshold β_min or if t≧a predetermined threshold t_max.
17. A system for encoding a multimedia stream for transmission over a network comprising:
an encoder adapted to perform the following:
determine a current channel bit rate (cur_CBR) of a communication link on the network;
determine a current headroom factor cur_α;
set a current video bit rate (cur_VBR) for encoding the multimedia stream such that the cur CBR−cur_VBR≧cur_α;
monitor the communication link to determine a new channel bit rate (new_CBR);
determine a channel bit rate reduction β, wherein β=cur_CBR−new_CBR; and
adjust cur_VBR if β>cur_α;
wherein the multimedia stream includes a series of macroblocks constituting a segment of a frame, wherein each segment of the frame is an independently decodable unit that is processed in parallel with other segments of the frame to establish sub-frame level processing; and
wherein the encoder adjusts an encoded video bit rate at the sub-frame level to allow for rapid adaptation to changing channel conditions and to minimize latency.
18. The system of claim 17 , wherein the encoder is further adapted to perform the following:
adjust cur_α if β>cur_α.
19. The system of claim 17 , wherein the encoder is further adapted to perform the following:
determine a local maximum drop in channel bit rate loc_max_β, wherein loc_max_β is the largest β observed over a time period t; and
adjust one or more of cur_VBR and cur_α if loc_m β≦a predetermined threshold β_min or if t≧a predetermined threshold t_max.
20. The system of claim 17 , wherein the encoder is further adapted to perform the following:
determine an average drop in channel bit rate avg_β, wherein avg_β is the average β observed over a time period t; and
adjust one or more of cur_VBR and cur_α if avg_β≦a predetermined threshold β_min or if t≧a predetermined threshold t_max.