Granted Patent B2

US 12,445,640 · App. 18/467,513 · Granted Oct 14, 2025

Inventors: Han Huang (San Diego, CA); Yan Zhang (San Diego, CA); Zhi Zhang (Munich, DE); Vadim Seregin (San Diego, CA); Marta Karczewicz (San Diego, CA)

Assignee: QUALCOMM INCORPORATED

H04N19/52H04N19/105H04N19/119H04N19/127H04N19/139H04N19/159H04N19/176H04N19/56

Abstract

Claims (111)

1. A method of decoding video data, the method comprising:

receiving a block of video data to be decoded using an affine decoder side motion vector refinement (DMVR) mode;

dividing the block into a plurality of subblocks;

subsampling the plurality of subblocks to determine a first subset of the plurality of subblocks, the first subset of subblocks of the plurality of subblocks including fewer subblocks than a number of subblocks within the plurality of subblocks;

determining a final offset for the affine DMVR mode using the first subset of the plurality of subblocks; and

decoding the block of video data using the final offset to generate a decoded block of video data.

2. The method of claim 1 , wherein determining the final offset for the affine DMVR mode comprises:

determining the final offset for the affine DMVR mode using only the first subset of the plurality of subblocks, and skipping a bilateral matching cost calculation on a second subset of the plurality of subblocks different from the first subset of the plurality of subblocks.

3. The method of claim 1 , wherein determining the final offset comprises:

determining respective initial motion vectors for a first prediction direction and a second prediction direction for each subblock of the first subset of the plurality of subblocks;

generating a plurality of candidate motion vectors for each of subblock of the first subset of the plurality of subblocks, wherein generating the plurality of candidate motion vectors includes adding each respective candidate offset, of a plurality of candidate offsets, to the respective initial motion vectors;

calculating respective bilateral matching costs for each subblock of the first subset of the plurality of subblocks for each of the plurality of candidate motion vectors;

accumulating s bilateral matching costs for all subblock of the first subset of the plurality of subblocks for each of the plurality of candidate motion vectors to generate a respective accumulated bilateral matching cost; and

determining the final offset to be the respective candidate offset associated with a lowest respective accumulated bilateral matching cost.

4. The method of claim 3 , wherein determining the respective initial motion vectors comprises:

determining the respective initial motion vectors from an affine candidate merge list.

5. The method of claim 1 , wherein decoding the block of video data using the final offset comprises:

determining control point motion vectors for the block of video data;

adding the final offset to the control point motion vectors to create refined control point motion vectors; and

decoding the block of video data using the refined control point motion vectors.

6. The method of claim 1 , wherein the first subset of the plurality of subblocks is one half of the plurality of subblocks.

7. The method of claim 6 , wherein the first subset of the plurality of subblocks includes every other row of the plurality of subblocks in the block or every other column of subblocks in the block.

8. The method of claim 6 , wherein the first subset of the plurality of subblocks are in a checkerboard pattern in the block.

9. The method of claim 1 , wherein the first subset of the plurality of subblocks is one quarter of the subblocks of the block.

10. The method of claim 9 , wherein first subset of the plurality of subblocks includes subblocks in every other column and every other row of subblocks in the block.

11. The method of claim 1 , further comprising:

displaying a picture that includes the decoded block of video data.

12. An apparatus configured to decode video data, the apparatus comprising:

a memory configured to store a block of video data; and

processing circuitry in communication with the memory, the processing circuitry being configured to:

receive the block of video data to be decoded using an affine decoder side motion vector refinement (DMVR) mode;

divide the block into a plurality of subblocks;

subsample the plurality of subblocks to determine a first subset of the plurality of subblocks, the first subset of subblocks of the plurality of subblocks including fewer subblocks than a number of subblocks within the plurality of subblocks;

determine a final offset for the affine DMVR mode using the first subset of the plurality of subblocks; and

decode the block of video data using the final offset to generate a decoded block of video data.

13. The apparatus of claim 12 , wherein to determine the final offset for the affine DMVR mode, the processing circuitry is further configured to:

determine the final offset for the affine DMVR mode using only the first subset of the plurality of subblocks, and skip a bilateral matching cost calculation on a second subset of the plurality of subblocks different from the first subset of the plurality of subblocks.

14. The apparatus of claim 12 , wherein to determine the final offset, the processing circuitry is further configured to:

determine respective initial motion vectors for a first prediction direction and a second prediction direction for each subblock of the first subset of the plurality of subblocks;

generate a plurality of candidate motion vectors for each of subblock of the first subset of the plurality of subblocks, wherein generating the plurality of candidate motion vectors includes adding each respective candidate offset, of a plurality of candidate offsets, to the respective initial motion vectors;

calculate respective bilateral matching costs for each subblock of the first subset of the plurality of subblocks for each of the plurality of candidate motion vectors;

accumulate the respective bilateral matching costs for all subblock of the first subset of the plurality of subblocks for each of the plurality of candidate motion vectors to generate a respective accumulated bilateral matching cost; and

determine the final offset to be the respective candidate offset associated with a lowest respective accumulated bilateral matching cost.

15. The apparatus of claim 14 , wherein to determine the respective initial motion vectors, the processing circuitry is further configured to:

determine the respective initial motion vectors from an affine candidate merge list.

16. The apparatus of claim 12 , wherein to decode the block of video data using the final offset, the processing circuitry is further configured to:

determine control point motion vectors for the block of video data;

add the final offset to the control point motion vectors to create refined control point motion vectors; and

decode the block of video data using the refined control point motion vectors.

17. The apparatus of claim 12 , wherein the first subset of the plurality of subblocks is one half of the plurality of subblocks.

18. The apparatus of claim 17 , wherein the first subset of the plurality of subblocks includes every other row of the plurality of subblocks in the block or every other column of subblocks in the block.

19. The apparatus of claim 17 , wherein the first subset of the plurality of subblocks are in a checkerboard pattern in the block.

20. The apparatus of claim 12 , wherein the first subset of the plurality of subblocks is one quarter of the subblocks of the block.

21. The apparatus of claim 20 , wherein first subset of the plurality of subblocks includes subblocks in every other column and every other row of subblocks in the block.

22. The apparatus of claim 12 , further comprising:

a display configured to display a picture that includes the decoded block of video data.

23. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to decode video data to:

receive a block of video data to be decoded using an affine decoder side motion vector refinement (DMVR) mode;

divide the block into a plurality of subblocks;

subsample the plurality of subblocks to determine a first subset of the plurality of subblocks, the first subset of subblocks of the plurality of subblocks including fewer subblocks than a number of subblocks within the plurality of subblocks;

determine a final offset for the affine DMVR mode using the first subset of the plurality of subblocks; and

decode the block of video data using the final offset to generate a decoded block of video data.

24. A method of encoding video data, the method comprising:

receiving a block of video data to be encoded using an affine decoder side motion vector refinement (DMVR) mode;

dividing the block into a plurality of subblocks;

subsampling the plurality of subblocks to determine a first subset of the plurality of subblocks, the first subset of subblocks of the plurality of subblocks including fewer subblocks than a number of subblocks within the plurality of subblocks;

determining a final offset for the affine DMVR mode using the first subset of the plurality of subblocks; and

encoding the block of video data using the final offset to generate an encoded block of video data.

25. The method of claim 24 , wherein to determining the final offset for the affine DMVR mode comprises:

determining the final offset for the affine DMVR mode using only the first subset of the plurality of subblocks, and skip a bilateral matching cost calculation on a second subset of the plurality of subblocks different from the first subset of the plurality of subblocks.

26. The method of claim 24 , wherein determining the final offset comprises:

determining respective initial motion vectors for a first prediction direction and a second prediction direction for each subblock of the first subset of the plurality of subblocks;

generating a plurality of candidate motion vectors for each of subblock of the first subset of the plurality of subblocks, wherein generating the plurality of candidate motion vectors includes adding each respective candidate offset, of a plurality of candidate offsets, to the respective initial motion vectors;

calculating respective bilateral matching costs for each subblock of the first subset of the plurality of subblocks for each of the plurality of candidate motion vectors;

accumulating the respective bilateral matching costs for all subblock of the first subset of the plurality of subblocks for each of the plurality of candidate motion vectors to generate a respective accumulated bilateral matching cost; and

determining the final offset to be the respective candidate offset associated with a lowest respective accumulated bilateral matching cost.

27. The method of claim 26 , wherein determining the respective initial motion vectors comprises:

determining the respective initial motion vectors from an affine candidate merge list.

28. The method of claim 24 , wherein encoding the block of video data using the final offset comprises:

determining control point motion vectors for the block of video data;

adding the final offset to the control point motion vectors to create refined control point motion vectors; and

encoding the block of video data using the refined control point motion vectors.

29. An apparatus configured to encode video data, the apparatus comprising:

a memory configured to store a block of video data; and

processing circuitry in communication with the memory, the processing circuitry being configured to:

receive the block of video data to be encoded using an affine decoder side motion vector refinement (DMVR) mode;

divide the block into a plurality of subblocks;

subsample the plurality of subblocks to determine a first subset of the plurality of subblocks, the first subset of subblocks of the plurality of subblocks including fewer subblocks than a number of subblocks within the plurality of subblocks;

determine a final offset for the affine DMVR mode using the first subset of the plurality of subblocks; and

encode the block of video data using the final offset to generate an encoded block of video data.

30. The apparatus of claim 29 , wherein to determine the final offset for the affine DMVR mode, the processing circuitry is further configured to:

determine the final offset for the affine DMVR mode using only the first subset of the plurality of subblocks, and skip a bilateral matching cost calculation on a second subset of the plurality of subblocks different from the first subset of the plurality of subblocks.

31. The apparatus of claim 29 , wherein to determine the final offset, the processing circuitry is further configured to:

determine respective initial motion vectors for a first prediction direction and a second prediction direction for each subblock of the first subset of the plurality of subblocks;

generate a plurality of candidate motion vectors for each of subblock of the first subset of the plurality of subblocks, wherein generating the plurality of candidate motion vectors includes adding each respective candidate offset, of a plurality of candidate offsets, to the respective initial motion vectors;

calculate respective bilateral matching costs for each subblock of the first subset of the plurality of subblocks for each of the plurality of candidate motion vectors;

accumulate the respective bilateral matching costs for all subblock of the first subset of the plurality of subblocks for each of the plurality of candidate motion vectors to generate a respective accumulated bilateral matching cost; and

determine the final offset to be the respective candidate offset associated with a lowest respective accumulated bilateral matching cost.

32. The apparatus of claim 31 , wherein to determine the respective initial motion vectors, the processing circuitry is further configured to:

determine the respective initial motion vectors from an affine candidate merge list.

33. The apparatus of claim 29 , wherein to encode the block of video data using the final offset, the processing circuitry is further configured to:

determine control point motion vectors for the block of video data;

add the final offset to the control point motion vectors to create refined control point motion vectors; and

encode the block of video data using the refined control point motion vectors.

34. The apparatus of claim 29 , wherein the first subset of the plurality of subblocks is one half of the plurality of subblocks.

35. The apparatus of claim 34 , wherein the first subset of the plurality of subblocks includes every other row of the plurality of subblocks in the block or every other column of subblocks in the block.

36. The apparatus of claim 34 , wherein the first subset of the plurality of subblocks are in a checkerboard pattern in the block.

37. The apparatus of claim 29 , wherein the first subset of the plurality of subblocks is one quarter of the subblocks of the block.

38. The apparatus of claim 37 , wherein first subset of the plurality of subblocks includes subblocks in every other column and every other row of subblocks in the block.

39. The apparatus of claim 29 , further comprising:

a camera configured to capture a picture that includes the block of video data.

Assignments (1)

Continuity (2)

References Cited (13)