IP Library Granted Patent US 9,094,685
Granted Patent B2
US 9,094,685 · App. 12/887,022 · Granted Jul 28, 2015

Efficient coding complexity estimation for video transcoding systems

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,094,685
App. No.
12/887,022
Granted
Jul 28, 2015
Kind
B2
Abstract

Systems and methods of transcoding video bitstreams that employ highly efficient look-ahead approaches to video transcoding. The systems and methods perform video transcoding in the pixel domain to make successive adjustments to estimates of the coding complexity of video frames in input video bitstreams. The systems and methods can employ the adjusted estimates of the coding complexity in a rate control function to provide improved bit allocations for the video frames in transcoded output video bitstreams, thereby enhancing overall perceptual quality.

Claims (63)

1. A method of transcoding a video bitstream in a video transcoding system, the video transcoding system including a video bitstream analyzer, a video decoder, and a video encoder, the method comprising the steps of:

receiving, at the video bitstream analyzer, an input video bitstream including a plurality of video frames from an input video sequence, the respective video frames being encoded in a first coding format;

for each of at least some of the plurality of video frames in the input video bitstream:

generating, by the video bitstream analyzer, an estimation representing a coding complexity of the respective video frame; and

adjusting, by the video bitstream analyzer, the estimation representing the coding complexity of the respective video frame, the adjusting including:

obtaining one or more of (1) a first adjustment parameter that is a first function of a current quantization parameter (QP) used to encode the respective video frame, (2) a second adjustment parameter that is a second function of a temporal distance between the respective video frame and a reference video frame used to encode the respective video frame, and, (3) in the event the respective video frame is encoded in the first coding format as a P-frame, a third adjustment parameter that is a third function of a difference between a reference QP used to encode the reference video frame and the current QP;

applying one or more of the first, second, and third adjustment parameters to the estimation representing the coding complexity of the respective video frame to obtain an adjusted coding complexity estimation of the respective video frame; and

in the event the respective video frame is encoded in the first coding format as the P-frame, applying, at least at some times, a fourth function to the adjusted coding complexity estimation of the respective video frame, the fourth function being expressed as,

C F (current_frame)=ω 1 *C F (previous_frame)+ω 2 *C F (current_frame)+ω 3 *C F (next_frame),

wherein “C F (current_frame)” is the adjusted coding complexity estimation of the respective video frame, “C F (previous_frame)” is a predetermined coding complexity of a previous P-frame in the input video bitstream, “C F (next_frame)” is another predetermined coding complexity of a next P-frame in the input video bitstream, and “ω 1 ”, “ω 2 ”, and “ω 3 ” are each a predetermined weighting value;

decoding, by the video decoder, the plurality of video frames to generate a plurality of decoded video frames; and

encoding, by the video encoder, the plurality of decoded video frames in a second coding format to generate a plurality of transcoded video frames in an output video bitstream, the encoding including applying a rate control function to at least some adjusted coding complexity estimations of the respective video frames to determine bit allocations for the plurality of transcoded video frames in the output video bitstream.

2. The method of claim 1 wherein each of the plurality of video frames includes a plurality of macroblocks (MBs), and wherein the generating of the estimation representing the coding complexity of the respective video frame includes generating estimations representing the coding complexities of the plurality of MBs included in the respective video frame.

3. The method of claim 2 wherein the generating of the estimation representing the coding complexity of the respective video frame further includes generating the estimation representing the coding complexity of the respective video frame as a sum of the coding complexities of the plurality of MBs.

4. The method of claim 1 wherein the obtaining of one or more of the first, second, and third adjustment parameters includes obtaining a fifth adjustment parameter that is a fifth function of the coding complexity of the respective video frame, and wherein the coding complexity of the respective video frame is less than or equal to a predetermined maximum value.

5. The method of claim 1 wherein the obtaining of one or more of the first, second, and third adjustment parameters includes obtaining a fifth adjustment parameter that is a fifth function of the coding complexity of the respective video frame, and wherein the coding complexity of the respective video frame is greater than or equal to a predetermined minimum value.

6. The method of claim 1 wherein the predetermined weighting value ω 1 is equal to 15/32, the predetermined weighting value ω 2 is equal to 1/16, and the predetermined weighting value ω 3 is equal to the predetermined weighting value ω 1 .

7. A method of transcoding a video bitstream in a video transcoding system, the video transcoding system including a video bitstream analyzer, a video decoder, and a video encoder, the method comprising the steps of:

receiving, at the video bitstream analyzer, an input video bitstream including a plurality of video frames from an input video sequence, the respective video frames being encoded in a first coding format;

for each of at least some of the plurality of video frames in the input video bitstream:

generating, by the video bitstream analyzer, an estimation representing a coding complexity of the respective video frame; and

adjusting, by the video bitstream analyzer, the estimation representing the coding complexity of the respective video frame, the adjusting including:

obtaining one or more of (1) a first adjustment parameter that is a first function of a current quantization parameter (QP) used to encode the respective video frame, (2) a second adjustment parameter that is a second function of a temporal distance between the respective video frame and a reference video frame used to encode the respective video frame, and, (3) in the event the respective video frame is encoded in the first coding format as a P-frame, a third adjustment parameter that is a third function of a difference between a reference QP used to encode the reference video frame and the current QP; and

applying one or more of the first, second, and third adjustment parameters to the estimation representing the coding complexity of the respective video frame to obtain an adjusted coding complexity estimation of the respective video frame;

decoding, by the video decoder, the plurality of video frames to generate a plurality of decoded video frames;

encoding, by the video encoder, the plurality of decoded video frames in a second coding format to generate a plurality of transcoded video frames in an output video bitstream, the encoding including applying a rate control function to at least some adjusted coding complexity estimations of the respective video frames to determine bit allocations for the plurality of transcoded video frames in the output video bitstream; and

in the event the respective video frame is encoded in the first coding format as the P-frame, and, subsequent to the decoding of the respective video frame, encoded in the second coding format as an I-frame, applying, at least at some times, a fourth function to the adjusted coding complexity estimation of the respective video frame, the fourth function being expressed as,

C F (current_frame)= C F (current_frame)*θ&υ+ I _complexity_prev*(1−υ),

wherein “C F (current_frame)” is the adjusted coding complexity estimation of the respective video frame, “I_complexity_prev” is a predetermined coding complexity of a previous I-frame in the input video bitstream, “0” is a predetermined fourth adjustment parameter, and “υ” is a predetermined weighting parameter.

8. The method of claim 7 further comprising:

determining, from a history of the encoding of the plurality of decoded video frames, one or more of the predetermined fourth adjustment parameter θ and the predetermined weighting parameter υ by obtaining at least one ratio of intra-coded macroblocks in respective I-frames.

9. A non-transitory, computer-readable medium comprising computer-executable instructions which, when executed by a processor in a computing device, cause the computing device to perform a method of transcoding a video bitstream in a video transcoding system, the video transcoding system including a video bitstream analyzer, a video decoder, and a video encoder, the video bitstream analyzer including a data collector and a coding complexity estimator, the method comprising:

receiving, at the data collector, an input video bitstream, the input video bitstream including a plurality of video frames from an input video sequence, the respective video frames being encoded in a first coding format, and

for each of at least some of the plurality of video frames in the input video bitstream:

generating, by the coding complexity estimator, an estimation representing a coding complexity of the respective video frame; and

adjusting, by the coding complexity estimator, the estimation representing the coding complexity of the respective video frame, the adjusting further including:

obtaining one or more of (1) a first adjustment parameter that is a first function of a current quantization parameter (QP) used to encode the respective video frame, (2) a second adjustment parameter that is a second function of a temporal distance between the respective video frame and a reference video frame used to encode the respective video frame, and, (3) in the event the respective video frame is encoded in the first coding format as a P-frame, a third adjustment parameter that is a third function of a difference between a reference QP used to encode the reference video frame and the current QP;

applying one or more of the first, second, and third adjustment parameters to the estimation representing the coding complexity of the respective video frame, thereby obtaining an adjusted coding complexity estimation of the respective video frame; and

in the event the respective video frame is encoded in the first coding format as the P-frame, applying, at least at some times, a fourth function to the adjusted coding complexity estimation of the respective video frame, the fourth function being expressed as,

C F (current_frame)=ω 1 *C F (previous_frame)+ω 2 *C F (current_frame)+ω 3 *C F (next_frame),

wherein “C F (current_frame)” is the adjusted coding complexity estimation of the respective video frame, “C F (previous_frame)” is a predetermined coding complexity of a previous P-frame in the input video bitstream, “C F (next_frame)” is another predetermined coding complexity of a next P-frame in the input video bitstream, and “ω 1 ”, “ω 2 ”, and “ω 3 ” are each a predetermined weighting value;

decoding, by the video decoder, the plurality of video frames, thereby generating a plurality of decoded video frames;

encoding, by the video encoder, the plurality of decoded video frames in a second coding format, thereby generating a plurality of transcoded video frames in an output video bitstream; and

applying, by the video encoder, a rate control function to at least some adjusted coding complexity estimations of the respective video frames to determine bit allocations for the plurality of transcoded video frames in the output video bitstream.

10. The non-transitory, computer-readable medium of claim 9 wherein each of the plurality of video frames includes a plurality of macroblocks (MBs), and wherein the method further comprises:

generating, by the video bitstream analyzer, estimations representing the coding complexities of the plurality of MBs included in the respective video frame.

11. The non-transitory, computer-readable medium of claim 10 wherein the method further comprises:

generating, by the video bitstream analyzer, the estimation representing the coding complexity of the respective video frame as a sum of the coding complexities of the plurality of MBs.

12. The non-transitory, computer-readable medium of claim 9 wherein the predetermined weighting value ω 1 is equal to 15/32, the predetermined weighting value ω 2 is equal to 1/16, and the predetermined weighting value ω 3 is equal to the predetermined weighting value ω 1 .

13. A non-transitory, computer-readable medium comprising computer-executable instructions which, when executed by a processor in a computing device, cause the computing device to perform a method of transcoding a video bitstream in a video transcoding system, the video transcoding system including a video bitstream analyzer, a video decoder, and a video encoder, the video bitstream analyzer including a data collector and a coding complexity estimator, the method comprising:

employing, by the video bitstream analyzer, a look-ahead approach to video transcoding, the employing of the look-ahead approach to video transcoding including:

wherein the data collector is operative to receive receiving, at the data collector, an input video bitstream, the input video bitstream including a plurality of video frames from an input video sequence, the respective video frames being encoded in a first coding format;

for each of at least some of the plurality of video frames in the input video bitstream:

to generate generating, by the coding complexity estimator, an estimation representing a coding complexity of the respective video frame; and

to adjust adjusting, by the coding complexity estimator, the estimation representing the coding complexity of the respective video frame, the adjusting further including:

to obtain obtaining one or more of (1) a first adjustment parameter that is a first function of a current quantization parameter (QP) used to encode the respective video frame, (2) a second adjustment parameter that is a second function of a temporal distance between the respective video frame and a reference video frame used to encode the respective video frame, and, (3) in the event the respective video frame is encoded in the first coding format as a P-frame, a third adjustment parameter that is a third function of a difference between a reference QP used to encode the reference video frame and the current QP;

to apply applying one or more of the first, second, and third adjustment parameters to the estimation representing the coding complexity of the respective video frame, thereby obtaining an adjusted coding complexity estimation of the respective video frame; and

in the event the respective video frame is encoded in the first coding format as the P-frame, applying, at least at some times, a fourth function to the adjusted coding complexity estimation of the respective video frame, the fourth function being expressed as,

C F (current_frame)=ω 1 *C F (previous_frame)+ω 2 *C F (current_frame)+ω 3 *CF (next_frame),

wherein “C F (current_frame)” is the adjusted coding complexity estimation of the respective video frame, “C F (previous_frame)” is a predetermined coding complexity of a previous P-frame in the input video bitstream, “C F (next_frame)” is another predetermined coding complexity of a next P-frame in the input video bitstream, and “ω 1 ”, “ω 2 ”, and “ω 3 ” are each a predetermined weighting value; a video decoder operative to decode the plurality of video frames, thereby generating a plurality of decoded video frames;

a video encoder operative to encode encoding, by the video encoder, the plurality of decoded video frames in a second coding format, thereby generating a plurality of transcoded video frames in an output video bitstream; and

applying, by the video encoder, a rate control function to at least some adjusted coding complexity estimations of the respective video frames to determine bit allocations for the plurality of transcoded video frames in the output video bitstream.

14. The non-transitory, computer-readable medium of claim 13 wherein the predetermined weighting value ω 1 is equal to 15/32, the predetermined weighting value θ 2 is equal to 1/16, and the predetermined weighting value ω 3 is equal to the predetermined weighting value ω 1 .

Assignments (4)
FIRST SUPPLEMENT TO INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Nov 1, 2016
From: DIALOGIC (US) INC.; DIALOGIC INC.; DIALOGIC US HOLDINGS INC.; DIALOGIC DISTRIBUTION LIMITED; DIALOGIC MANUFACTURING LIMITED; DIALOGIC CORPORATION; DIALOGIC GROUP INC.
To: SILICON VALLEY BANK
Reel/Frame 040542/0297 →
SECURITY AGREEMENT Recorded Jun 30, 2015
From: DIALOGIC (US) INC.; DIALOGIC INC.; DIALOGIC US HOLDINGS INC.; DIALOGIC DISTRIBUTION LIMITED; DIALOGIC MANUFACTURING LIMITED; DIALOGIC CORPORATION; DIALOGIC GROUP INC.
To: SILICON VALLEY BANK
Reel/Frame 036037/0165 →
RELEASE OF SECURITY INTEREST Recorded Nov 25, 2014
From: OBSIDIAN, LLC
To: DIALOGIC CORPORATION, F/K/A EICON NETWORKS CORPORATION; DIALOGIC INC.; DIALOGIC (US) INC., F/K/A DIALOGIC INC. AND F/K/A EICON NETWORKS INC.; DIALOGIC DISTRIBUTION LIMITED, F/K/A EICON NETWORKS DISTRIBUTION LIMITED; DIALOGIC MANUFACTURING LIMITED, F/K/A EICON NETWORKS MANUFACTURING LIMITED; DIALOGIC US HOLDINGS INC.; DIALOGIC RESEARCH INC., F/K/A EICON NETWORKS RESEARCH INC.; DIALOGIC JAPAN, INC., F/K/A CANTATA JAPAN, INC.; CANTATA TECHNOLOGY, INC.; EAS GROUP, INC.; SHIVA (US) NETWORK CORPORATION; EXCEL SWITCHING CORPORATION; EXCEL SECURITIES CORPORATION; CANTATA TECHNOLOGY INTERNATIONAL, INC.; BROOKTROUT NETWORKS GROUP, INC.; BROOKTROUT TECHNOLOGY, INC.; SNOWSHORE NETWORKS, INC.; BROOKTROUT SECURITIES CORPORATION
Reel/Frame 034468/0654 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2010
From: YANG, KYEONG H.; WANG, BEIBEI
To: DIALOGIC CORPORATION
Reel/Frame 025028/0546 →