Processing a video stream in real time based on binary information of the video stream
View Patent ↗A system (and a method) are disclosed for a video processing system with enhanced entropy coding performance. The system includes an entropy decoder configured to divide decoding of an input video stream into arithmetic decoding and syntax decoding. The entropy decoder includes an arithmetic decoding module, a syntax decoding module, a memory management module and a memory buffer connecting the two decoding modules. The arithmetic decoding module is configured to decode the input video stream into multiple bins of decoded input video stream and the syntax decoding module is configured to decode the bins of arithmetically decoded input videos stream into one or more syntax elements. The memory management module uses the memory buffer to accelerate the coding performances of arithmetic decoding and syntax decoding. The system also includes a corresponding entropy encoder configured to encode a video stream with improved coding performance.
1. A computer-implemented method for decoding an input video stream in real time, comprising:
receiving an input video stream, the input video stream having a plurality of video frames, video coding parameters and a target performance rate;
initializing a memory buffer for storing arithmetically decoded video data of the input video stream;
arithmetically decoding the input video stream, the arithmetically decoded input video stream represented by a plurality of bins which are binary information of the arithmetically decoded input video stream;
storing the plurality of bins of the arithmetically decoded input video stream in the memory buffer;
identifying arithmetic decoding rate of the input video stream, the identified arithmetic decoding rate varying depending on video content of the plurality of video frames of the input video stream;
comparing the identified arithmetic decoding rate with a picture rate, the picture rate being determined by one or more video coding parameters of the input video stream;
determining buffer fullness of the memory buffer based the comparison of the identified arithmetic decoding rate and the picture rate;
responsive to the buffer fullness of the memory buffer exceeding a threshold value, retrieving the plurality of bins stored in the memory buffer; and
decoding the plurality of bins into bits of syntax information of the arithmetically decoded input video stream.
2. The method of claim 1 , the input video stream is a video stream that is suitable for context-adaptive binary arithmetic coding.
3. The method of claim 2 , wherein the input video stream is a context-adaptive binary arithmetic coding (CABAC) video stream.
4. The method of claim 1 , wherein the video coding parameters of the input video stream include a bit-rate, a frame rate and a frame type for each video frame of the input video stream, and the target performance rate is the frame rate of the input video stream.
5. The method of claim 1 , wherein initializing the memory buffer comprises:
determining a size of the memory buffer based on at least in part of the video coding parameters of the input video stream; and
determining an initial delay for receiving the plurality of bins of the arithmetically decoded input video stream and for retrieving the plurality of bins for syntax decoding.
6. The method of claim 1 , wherein decoding the plurality of bins into bits of syntax information comprises:
retrieving bins of arithmetically decoded input video stream; and
de-binarizing the retrieved bins of arithmetically decoded input video stream into bits of syntax elements, wherein performance of de-binarizing supports the target performance rate.
7. The method of claim 1 , wherein decoding the plurality of bins into bits of syntax information of the arithmetically decoded input video stream comprises:
retrieving the plurality of bins of arithmetically decoded input video stream from the memory buffer; and
combining the plurality of bins of arithmetically decoded input video stream into one or more syntax elements of the input video stream.
8. A computer-implemented method for encoding an input video stream in real time, comprising:
receiving the input video stream, the input video stream having a plurality of syntax elements of the input video stream;
initializing a memory buffer associated with the encoding;
encoding the plurality of syntax element into a plurality of bins, the plurality of bins containing binary information of syntax encoded input video stream;
storing the plurality of bins of the syntax encoded input video stream in the memory buffer;
identifying syntax encoding rate of the input video stream;
comparing the identified syntax encoding rate with a picture rate, the picture rate being determined by one or more video coding parameters of the input video stream;
determining buffer fullness of the memory buffer based the comparison of the identified syntax encoding rate and the picture rate;
responsive to the buffer fullness of the memory buffer exceeding a threshold value, retrieving the plurality of bins stored in the memory buffer; and
arithmetically encoding the plurality of bins into an output video stream.
9. A non-transitory computer-readable storage medium comprising instructions executable by one or more processors for processing an input video stream in real time, the instructions when executed cause the one or more processors to:
receive an input video stream, the input video stream having a plurality of video frames, video coding parameters and a target performance rate;
initialize a memory buffer for storing arithmetically decoded video data of the input video stream;
arithmetically decode the input video stream, the arithmetically decoded input video stream represented by a plurality of bins which are binary information of the arithmetically decoded input video stream;
store the plurality of bins of the arithmetically decoded input video stream in the memory buffer;
identify arithmetic decoding rate of the input video stream, the identified arithmetic decoding rate varying depending on video content of the plurality of video frames of the input video stream;
compare the identified arithmetic decoding rate with a picture rate, the picture rate being determined by one or more video coding parameters of the input video stream;
determine buffer fullness of the memory buffer based the comparison of the identified arithmetic decoding rate and the picture rate;
responsive to the buffer fullness of the memory buffer exceeding a threshold value, retrieve the plurality of bins stored in the memory buffer; and
decode the plurality of bins into bits of syntax information of the arithmetically decoded input video stream.
10. The computer-readable storage medium of claim 9 , the input video stream is a video stream that is suitable for context-adaptive binary arithmetic coding.
11. The computer-readable storage medium of claim 10 , wherein the input video stream is a context-adaptive binary arithmetic coding (CABAC) video stream.
12. The computer-readable storage medium of claim 9 , wherein the video coding parameters of the input video stream include a bit-rate, a frame rate and a frame type for each video frame of the input video stream, and the target performance rate is the frame rate of the input video stream.
13. The computer-readable storage medium of claim 9 , further comprising instructions that when executed by the one or more processors cause the one or more processors to:
determine a size of the memory buffer based on at least in part of the video coding parameters of the input video stream; and
determine an initial delay for receiving the plurality of bins of the arithmetically decoded input video stream and for retrieving the plurality of bins for syntax decoding.
14. The computer-readable storage medium of claim 9 , wherein the instructions for decoding the plurality of bins into bits of syntax information comprise instructions that when executed by the one or more processors cause the one or more processors to:
retrieve bins of arithmetically decoded input video stream; and
de-binarize the retrieved bins of arithmetically decoded input video stream into bits of syntax elements, wherein performance of de-binarizing supports the target performance rate.
15. The computer-readable storage medium of claim 9 , wherein the instructions for decoding the plurality of bins into bits of syntax information of the arithmetically decoded input video stream comprise instructions that when executed by the one or more processors cause the one or more processors to:
retrieve the plurality of bins of arithmetically decoded input video stream from the memory buffer; and
combine the plurality of bins of arithmetically decoded input video stream into one or more syntax elements of the input video stream.
16. A non-transitory computer-readable storage medium comprising instructions executable by one or more processors for encoding an input video stream in real time, the instructions when executed cause the one or more processors to:
receive the input video stream, the input video stream having a plurality of syntax elements of the input video stream;
initialize a memory buffer associated with the encoding;
encode the plurality of syntax element into a plurality of bins, the plurality of bins containing binary information of syntax encoded input video stream;
store the plurality of bins of the syntax encoded input video stream in the memory buffer;
identify syntax encoding rate of the input video stream;
compare the identified syntax encoding rate with a picture rate, the picture rate being determined by one or more video coding parameters of the input video stream;
determine buffer fullness of the memory buffer based the comparison of the identified syntax encoding rate and the picture rate;
responsive to the buffer fullness of the memory buffer exceeding a threshold value, retrieve the plurality of bins stored in the memory buffer; and
arithmetically encode the plurality of bins into an output video stream.
17. A computer system for processing an input video stream in real time, comprising:
a computer processor;
a non-transitory computer-readable storage medium storing computer program instructions that when executed by the computer processor cause the computer processor to:
receive an input video stream, the input video stream having a plurality of video frames, video coding parameters and a target performance rate;
initialize a memory buffer for storing arithmetically decoded video data of the input video stream;
arithmetically decode the input video stream, the arithmetically decoded input video stream represented by a plurality of bins which are binary information of the arithmetically decoded input video stream;
store the plurality of bins of the arithmetically decoded input video stream in the memory buffer;
identify arithmetic decoding rate of the input video stream, the identified arithmetic decoding rate varying depending on video content of the plurality of video frames of the input video stream;
compare the identified arithmetic decoding rate with a picture rate, the picture rate being determined by one or more video coding parameters of the input video stream;
determine buffer fullness of the memory buffer based the comparison of the identified arithmetic decoding rate and the picture rate;
responsive to the buffer fullness of the memory buffer exceeding a threshold value, retrieve the plurality of bins stored in the memory buffer; and
decode the plurality of bins into bits of syntax information of the arithmetically decoded input video stream.
18. The system of claim 17 , further comprising computer program instructions that when executed by the computer processor cause the computer processor to:
determine a size of the memory buffer based on at least in part of the video coding parameters of the input video stream; and
determine an initial delay for receiving the plurality of bins of the arithmetically decoded input video stream and for retrieving the plurality of bins for syntax decoding.
19. The system of claim 17 , wherein the instructions for decoding the plurality of bins into bits of syntax information of the arithmetically decoded input video stream comprise instructions that when executed by the one or more processors cause the one or more processors to:
retrieve the plurality of bins of arithmetically decoded input video stream from the memory buffer; and
combine the plurality of bins of arithmetically decoded input video stream into one or more syntax elements of the input video stream.
20. A computer system for encoding an input video stream in real time, comprising:
a computer processor;
a non-transitory computer-readable storage medium storing computer program instructions that when executed by the computer processor cause the computer processor to:
receive the input video stream, the input video stream having a plurality of syntax elements of the input video stream;
initialize a memory buffer associated with the encoding;
encode the plurality of syntax element into a plurality of bins, the plurality of bins containing binary information of syntax encoded input video stream;
store the plurality of bins of the syntax encoded input video stream in the memory buffer;
identify syntax encoding rate of the input video stream;
compare the identified syntax encoding rate with a picture rate, the picture rate being determined by one or more video coding parameters of the input video stream;
determine buffer fullness of the memory buffer based the comparison of the identified syntax encoding rate and the picture rate;
responsive to the buffer fullness of the memory buffer exceeding a threshold value, retrieve the plurality of bins stored in the memory buffer; and
arithmetically encode the plurality of bins into an output video stream.