IP Library › Granted Patent US 11,412,229
Granted Patent B2
US 11,412,229 · App. 16/994,183 · Granted Aug 9, 2022

Method and apparatus for video encoding and decoding

Inventors: Xinwei Gao (Shenzhen, CN); Xunan Mao (Shenzhen, CN); Chenchen Gu (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
H04N19/147H04N19/124H04N19/172H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,412,229
App. No.
16/994,183
Granted
Aug 9, 2022
Kind
B2
Abstract

A video coding method includes obtaining an input video frame; determining down-sampling information corresponding to the input video frame in response to determining that a target processing manner corresponding to the input video frame is a down-sampling processing manner; encoding the input video frame according to the down-sampling information to obtain encoded data corresponding to the input video frame; and adding the down-sampling information to the encoded data according to a processing range corresponding to the down-sampling information. The down-sampling information is capable of identifying at least one of a down-sampling proportion and a down-sampling method for the input video frame or identifying a rule for identifying the at least one of a down-sampling proportion and a down-sampling method.

Claims (83)

1. A video coding method, comprising:

obtaining an input video frame;

determining down-sampling information corresponding to the input video frame in response to determining that a target processing manner corresponding to the input video frame is a down-sampling processing manner, wherein the down-sampling information includes a down-sampling method, and the down-sampling method includes a first down-sample method with a first calculation complexity and a second down-sampling method with a second calculation complexity lower than the first calculation complexity;

in response to determining a real-time requirement is a first real-time requirement, employing the first down-sampling method, and in response to determining the real-time requirement is a second real-time requirement higher than the first real-time requirement, employing the second down-sampling method;

encoding the input video frame according to the down-sampling information to obtain encoded data corresponding to the input video frame;

obtaining a processing range corresponding to the down-sampling information, wherein the processing range includes one or more of a sequence level range, a group of pictures (GOP) level range, a frame level range, and a coded block level range, and wherein the sequence level range corresponds to a video sequence including two or more GOPs, the GOP level range corresponds to a GOP including two or more frames, the frame level range corresponds to a frame including two or more coded blocks, and the coded block level range corresponds to a coded block; and

adding the down-sampling information to the encoded data corresponding to the processing range, by:

determining whether the processing range is the sequence level range, the GOP level range, the frame level range, or the coded block level range;

in response to determining the processing range is the sequence level range, adding the down-sampling information to header information of the video sequence;

in response to determining the processing range is the GOP level range, adding the down-sampling information to header information of the GOP;

in response to determining the processing range is the frame level range, adding the down-sampling information to header information of the frame; and

in response to determining the processing range is the coded block level range, adding the down-sampling information to header information of the coded block.

2. The method according to claim 1 , wherein the down-sampling information includes a down-sampling proportion, and the down-sampling proportion is obtained by:

obtaining a current quantization parameter (QP) estimation value corresponding to the input video frame;

obtaining a QP threshold corresponding to the input video frame; and

determining, according to a difference between the current QP estimation value and the QP threshold, the down-sampling proportion corresponding to the input video frame.

3. The method according to claim 2 , wherein the determining, according to a difference between the current QP estimation value and the QP threshold, the down-sampling proportion corresponding to the input video frame comprises:

obtaining a correspondence between a candidate difference range and a candidate down-sampling proportion;

determining, from the candidate difference range according to the difference between the current QP estimation value and the QP threshold, a target difference range corresponding to the difference; and

determining, from the candidate down-sampling proportion according to the target difference range and the correspondence, the down-sampling proportion corresponding to the input video frame.

4. The method according to claim 2 , wherein the obtaining a QP threshold corresponding to the input video frame comprises:

obtaining video frame information corresponding to the input video frame; and

determining, according to the video frame information, the QP threshold corresponding to the input video frame.

5. The method according to claim 1 , wherein the down-sampling information includes a down-sampling method, and the down-sampling method is determined by:

calculating an image content complexity corresponding to the input video frame;

obtaining a correspondence between a candidate image content complexity and a candidate down-sampling method; and

determining, according to the correspondence, the down-sampling method corresponding to the image content complexity.

6. The method according to claim 1 , wherein the down-sampling information includes a down-sampling method, and the down-sampling method is determined by:

obtaining a current coded block of the input video frame;

calculating a module gradient value corresponding to the current coded block;

classifying the current coded block into a target module class in a module set according to the module gradient value, candidate module classes in the module class set respectively corresponding to different module gradient value ranges;

obtaining a correspondence between a candidate module class and a candidate down-sampling method; and

determining, according to the correspondence between the candidate module class and the candidate down-sampling method and the target module class, the down-sampling method corresponding to the current coded block.

7. The method according to claim 1 , wherein the down-sampling information includes a down-sampling proportion, and the encoded data is obtained by:

processing the input video frame according to the down-sampling proportion to obtain a current frame;

obtaining a reference frame corresponding to the current frame; and

encoding the current frame according to the reference frame, to obtain the encoded data corresponding to the input video frame.

8. The method according to claim 7 , wherein the encoding the current frame according to the reference frame, to obtain the encoded data corresponding to the input video frame comprises:

obtaining a down-sampling proportion corresponding to the current frame, and processing the reference frame according to the down-sampling proportion to obtain a target reference frame; and

encoding the current frame according to the target reference frame, to obtain the encoded data corresponding to the input video frame.

9. The method according to claim 1 , wherein the determining down-sampling information corresponding to the input video frame comprises:

obtaining a correspondence between a candidate environmental factor and candidate down-sampling information; and

obtaining a current environmental factor, and determining, according to the correspondence between the candidate environmental factor and the candidate down-sampling information, the down-sampling information corresponding to the current environmental factor.

10. The method according to claim 1 , wherein the down-sampling method includes a first down-sampling proportion and a second down-sampling proportion higher than the first down-sampling proportion, and the method further comprises:

determining whether the input video frame is a B frame or a P frame; and

in response to determining the input video frame is the B frame, employing the second down-sampling proportion; and

in response to determining the input video frame is the P frame, employing the first down-sampling proportion.

11. A video coding method, comprising:

obtaining encoded data corresponding to a current frame to be decoded;

determining a processing range according to the encoded data, wherein the processing range includes one or more of a sequence level range, a group of pictures (GOP) level range, a frame level range, and a coded block level range, and wherein the sequence level range corresponds to a video sequence including two or more GOPs, the GOP level range corresponds to a GOP including two or more frames, the frame level range corresponds to a frame including two or more coded blocks, and the coded block level range corresponds to a coded block;

reading down-sampling information corresponding to the processing range, by:

determining whether the processing range is the sequence level range, the GOP level range, the frame level range, or the coded block level range;

in response to determining the processing range is the sequence level range, reading the down-sampling information from header information of the video sequence;

in response to determining the processing range is the GOP level range, reading the down-sampling information from header information of the GOP;

in response to determining the processing range is the frame level range, reading the down-sampling information from header information of the frame; and

in response to determining the processing range is the coded block level range, reading the down-sampling information from header information of the coded block;

decoding the encoded data according to the down-sampling information corresponding to the current frame, to obtain a decoded video frame, wherein the down-sampling information includes a down-sampling method, and the down-sampling method includes a first down-sample method with a first calculation complexity and a second down-sampling method with a second calculation complexity lower than the first calculation complexity; and

in response to determining a real-time requirement is a first real-time requirement, employing the first down-sampling method, and in response to determining the real-time requirement is a second real-time requirement higher than the first real-time requirement, employing the second down-sampling method.

12. The method according to claim 11 , wherein the down-sampling information includes a down-sampling proportion, and the decoded video frame is obtained by:

decoding the encoded data according to the down-sampling proportion corresponding to the current frame, to obtain a reconstructed video frame corresponding to the current frame; and

up-sampling the reconstructed video frame according to an up-sampling proportion corresponding to the down-sampling proportion, to obtain the decoded video frame.

13. The method according to claim 12 , wherein the decoding the encoded data according to the down-sampling proportion corresponding to the current frame, to obtain a reconstructed video frame corresponding to the current frame comprises:

obtaining a current reference frame corresponding to the current frame; and

decoding the encoded data according to the down-sampling proportion corresponding to the current frame and the current reference frame, to obtain the reconstructed video frame corresponding to the current frame.

14. The method according to claim 13 , wherein the decoding the encoded data according to the down-sampling proportion corresponding to the current frame and the current reference frame, to obtain the reconstructed video frame corresponding to the current frame comprises:

determining, according to the down-sampling proportion corresponding to the current frame, resolution information corresponding to the current frame;

processing the current reference frame according to the resolution information to obtain a target reference frame; and

decoding the encoded data according to the target reference frame, to obtain the reconstructed video frame corresponding to the current frame.

15. The method according to claim 11 , wherein the down-sampling information includes a down-sampling method, and the decoded video frame is obtained by:

decoding the encoded data to obtain a reconstructed video frame corresponding to the current frame; and

processing the reconstructed video frame according to an up-sampling method corresponding to the down-sampling method, to obtain the decoded video frame.

16. A non-transitory computer-readable storage medium storing computer program instructions executable by at least one processor to perform:

obtaining an input video frame;

determining down-sampling information corresponding to the input video frame in response to determining that a target processing manner corresponding to the input video frame is a down-sampling processing manner, wherein the down-sampling information includes a down-sampling method, and the down-sampling method includes a first down-sample method with a first calculation complexity and a second down-sampling method with a second calculation complexity lower than the first calculation complexity;

in response to determining a real-time requirement is a first real-time requirement, employing the first down-sampling method, and in response to determining the real-time requirement is a second real-time requirement higher than the first real-time requirement, employing the second down-sampling method;

encoding the input video frame according to the down-sampling information to obtain encoded data corresponding to the input video frame;

obtaining a processing range corresponding to the down-sampling information, wherein the processing range includes one or more of a sequence level range, a group of pictures (GOP) level range, a frame level range, and a coded block level range, and wherein the sequence level range corresponds to a video sequence including two or more GOPs, the GOP level range corresponds to a GOP including two or more frames, the frame level range corresponds to a frame including two or more coded blocks, and the coded block level range corresponds to a coded block; and

adding the down-sampling information to header information corresponding to the processing range, by:

determining whether the processing range is the sequence level range, the GOP level range, the frame level range, or the coded block level range;

in response to determining the processing range is the sequence level range, adding the down-sampling information to header information of the video sequence;

in response to determining the processing range is the GOP level range, adding the down-sampling information to header information of the GOP;

in response to determining the processing range is the frame level range, adding the down-sampling information to header information of the frame; and

in response to determining the processing range is the coded block level range, adding the down-sampling information to header information of the coded block.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2020
From: GAO, XINWEI; MAO, XUNAN; GU, CHENCHEN
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 053503/0750 →
Priority Claims (1)
CN 201810638919.1 · Jun 20, 2018 · national
Continuity (2)
Continuation PCTCN2019089990 · Jun 4, 2019
Related Publication 20200382793A1 · Dec 3, 2020