Granted Patent B2

US 12,604,018 · App. 18/744,171 · Granted Apr 14, 2026

Inventors: Thomas Alexander Ryder (San Diego, CA); Samuel James Eadie (Munich, DE); Marta Karczewicz (San Diego, CA); Muhammed Zeyd Coban (Carlsbad, CA); Vadim Seregin (San Diego, CA)

Assignee: QUALCOMM Incorporated

H04N19/31H04N19/107H04N19/167H04N19/176

Abstract

Claims (64)

1 . A method of decoding video data, the method comprising:

determining that a first temporal layer identifier of a first picture of video data is included in a first set of temporal layers;

in response to the first temporal layer identifier being included in the first set of temporal layers, decoding blocks of the first picture on a block by block basis, without performing neural network-based decoding of the first picture;

determining that a second temporal layer identifier of a second picture of the video data is included in a second set of temporal layers, the second set of temporal layers being higher than the first set of temporal layers; and

in response to the second temporal layer identifier being included in the second set of temporal layers, decoding the second picture using a neural network-based video decoder, without performing block-based prediction and residual decoding of the second picture.

2 . The method of claim 1 , further comprising:

encoding the blocks of the first picture on a block by block basis; and

encoding the second picture using a neural network-based video encoder.

3 . The method of claim 2 , wherein encoding the blocks of the first picture includes:

for a current block of the first picture, forming a prediction block using inter-prediction, intra-prediction, affine prediction, or intra block copy (IBC) mode;

forming a residual block representing differences between the current block and the prediction block; and

encoding the residual block and prediction information used to form the prediction block.

4 . The method of claim 1 , wherein decoding the blocks of the first picture includes:

for a current block of the blocks of the first picture, forming a prediction block using one of inter-prediction, intra-prediction, affine prediction, or intra block copy (IBC) mode;

decoding a residual block for the current block; and

combining the prediction block with the residual block to form a decoded block for the current block.

5 . The method of claim 1 , wherein at least one picture of the second set of temporal layers is predicted from a reference picture of the first set of temporal layers.

6 . The method of claim 1 , further comprising providing each of the pictures of the first set of temporal layers to the neural network-based video decoder as input for use when decoding pictures in the second set of temporal layers.

7 . The method of claim 1 , wherein decoding the blocks of the first picture and decoding the second picture comprises decoding the first picture before decoding the second picture, the method further comprising:

determining that the second picture has a display order before a display order of the first picture; and

outputting the second picture before outputting the first picture based on the second picture having the display order before the display order of the first picture.

8 . A device for decoding video data, the device comprising:

a memory configured to store video data; and

a processing system comprising one or more processors implemented in circuitry, the processing system being configured to:

determine that a first temporal layer identifier of a first picture of the video data is included in a first set of temporal layers;

in response to the first temporal layer identifier being included in the first set of temporal layers, decode blocks of the first picture on a block by block basis, without performing neural network-based decoding of the first picture;

determine that a second temporal layer identifier of a second picture of the video data is included in a second set of temporal layers, the second set of temporal layers being higher than the first set of temporal layers; and

in response to the second temporal layer identifier being included in the second set of temporal layers, execute a neural network-based video decoder to decode the second picture, without performing block-based prediction and residual decoding of the second picture.

9 . The device of claim 8 , wherein the processing system is further configured to:

encode the blocks of the first picture on a block by block basis; and

execute a neural network-based video encoder to encode the second picture.

10 . The device of claim 9 , wherein to encode the blocks of the first picture, the processing system is configured to:

for a current block of the first picture, form a prediction block using inter-prediction, intra-prediction, affine prediction, or intra block copy (IBC) mode;

form a residual block representing differences between the current block and the prediction block; and

encode the residual block and prediction information used to form the prediction block.

11 . The device of claim 8 , wherein to decode the blocks of the first picture, the processing system is configured to:

for a current block of the blocks of the first picture, form a prediction block using one of inter-prediction, intra-prediction, affine prediction, or intra block copy (IBC) mode;

decode a residual block for the current block; and

combine the prediction block with the residual block to form a decoded block for the current block.

12 . The device of claim 8 , wherein at least one picture of the second set of temporal layers is predicted from a reference picture of the first set of temporal layers.

13 . The device of claim 8 , wherein the processing system is configured to provide each of the pictures of the first set of temporal layers to the neural network-based video decoder as input for use when decoding pictures in the second set of temporal layers.

14 . The device of claim 8 , wherein to decode the blocks of the first picture and to decode the second picture, the processing system is configured to decode the first picture before executing the neural network-based video decoder to decode the second picture, and wherein the processing system is further configured to:

determine that the second picture has a display order before a display order of the first picture; and

output the second picture before outputting the first picture based on the second picture having the display order before the display order of the first picture.

15 . A device for decoding video data, the device comprising:

means for determining that a first temporal layer identifier of a first picture of video data is included in a first set of temporal layers;

means for decoding blocks of the first picture on a block by block basis, without performing neural network-based decoding of the first picture, in response to the first temporal layer identifier being included in the first set of temporal layers;

means for determining that a second temporal layer identifier of a second picture of the video data is included in a second set of temporal layers, the second set of temporal layers being higher than the first set of temporal layers; and

means for decoding the second picture using a neural network-based video decoder, without performing block-based prediction and residual decoding of the second picture, in response to the second temporal layer identifier being included in the second set of temporal layers.

16 . The device of claim 15 , further comprising:

means for encoding the blocks of the first picture on a block by block basis; and

means for encoding the second picture using a neural network-based video encoder.

17 . The device of claim 16 , wherein the means for encoding the blocks of the first picture includes:

means for forming a prediction block using inter-prediction, intra-prediction, affine prediction, or intra block copy (IBC) mode for a current block of the first picture;

means for forming a residual block representing differences between the current block and the prediction block; and

means for encoding the residual block and prediction information used to form the prediction block.

18 . The device of claim 15 , wherein the means for decoding the blocks of the first picture includes:

means for forming a prediction block using one of inter-prediction, intra-prediction, affine prediction, or intra block copy (IBC) mode for a current block of the blocks of the first picture;

means for decoding a residual block for the current block; and

means for combining the prediction block with the residual block to form a decoded block for the current block.

19 . The device of claim 15 , further comprising means for providing each of the pictures of the first set of temporal layers to the neural network-based video decoder as input for use when decoding pictures in the second set of temporal layers.

20 . The device of claim 15 , wherein the means for decoding the blocks of the first picture is configured to decode the blocks of the first picture before the means for decoding the second picture decodes the second picture, further comprising:

means for determining that the second picture has a display order before a display order of the first picture; and

means for outputting the second picture before outputting the first picture based on the second picture having the display order before the display order of the first picture.

Assignments (2)

Continuity (2)

References Cited (18)