Granted Patent B2

US 12,256,095 · App. 17/952,193 · Granted Mar 18, 2025

Inventor: Shuqing Fang (Zhejiang, CN)

Assignee: HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO., LTD.

H04N19/52H04N19/176H04N19/577

Abstract

Claims (87)

1. A decoding method, comprising:

obtaining index information of an enhanced temporal motion vector prediction mode from a bit stream of a current block when it is determined that an enhanced temporal motion vector prediction technology is enabled for the current block;

determining one or more matching blocks of the current block, wherein the one or more matching blocks refer to one or more image blocks corresponding to the current block in a target reference picture, comprising:

determining one or more matching blocks of the current block at least based on a target reference picture index, a target reference direction and a position of the current block, wherein the target reference picture index is an index of a first picture in reference picture List 0 , and the target reference direction is a reference direction corresponding to reference picture List 0 ; or, the target reference picture index is an index of a first picture in reference picture List 1 , and the target reference direction is a reference direction corresponding to reference picture List 1 ;

determining candidate enhanced temporal motion vector prediction modes based on the one or more matching blocks and one or more new matching blocks obtained by offsetting each of the one or more matching blocks, and establishing a temporal candidate mode list based on the candidate enhanced temporal motion vector prediction modes;

based on the index information, determining the enhanced temporal motion vector prediction mode from the temporal candidate mode list;

determining motion information of each sub-block in the current block based on the determined enhanced temporal motion vector prediction mode, and performing motion compensation on each sub-block in the current block based on the motion information of each sub-block in the current block,

wherein, determining the motion information of each sub-block in the current block based on the determined enhanced temporal motion vector prediction mode, comprises:

for any sub-block in a target matching block,

if both forward motion information and backward motion information of the sub-block are available, scaling the forward motion information and the backward motion information of the sub-block to the first picture in reference picture List 0 and the first picture in reference picture List 1 respectively, and assigning the scaled forward motion information and the scaled backward motion information to a sub-block at a corresponding position in the current block; or

if forward motion information of the sub-block is available but backward motion information of the sub-block is unavailable, scaling the forward motion information of the sub-block to the first picture in reference picture List 0 , and assigning the scaled forward motion information to a sub-block at a corresponding position in the current block; or

if backward motion information of the sub-block is available but forward motion information of the sub-block is unavailable, scaling the backward motion information of the sub-block to the first picture in reference picture List 1 , and assigning the scaled backward motion information to a sub-block at a corresponding position in the current block,

wherein a sequence parameter set level syntax is used to indicate whether the enhanced temporal motion vector prediction technology is enabled for the current block.

2. The decoding method according to claim 1 , wherein the one or more new matching blocks obtained by offsetting each of the one or more matching blocks are determined by:

for a respective matching block of the one or more matching blocks,

based on one or more offset pairs, horizontally and vertically offsetting the respective matching block to obtain one or more new matching blocks.

3. The decoding method according to claim 1 , wherein after the determining one or more matching blocks of the current block, the decoding method further comprises:

clipping each of the one or more matching blocks to obtain one or more clipped matching blocks.

4. The decoding method according to claim 3 , wherein the one or more new matching blocks obtained by offsetting each of the one or more matching blocks are determined by:

for a respective clipped matching block of the one or more clipped matching blocks,

comparing the motion information of a thirteenth sub-block and the motion information of a fourteenth sub-block, and comparing the motion information of a fifteenth sub-block and the motion information of a sixteenth sub-block, to obtain two comparing results, if at least one of the two comparing results indicates different in motion information, horizontally offsetting the clipped matching block rightward to obtain a new matching block; or

comparing the motion information of a seventeenth sub-block and the motion information of an eighteenth sub-block, and comparing the motion information of a nineteenth sub-block and the motion information of a twentieth sub-block, to obtain two comparing results, if at least one of the two comparing results indicates different in motion information, horizontally offsetting the clipped matching block leftward to obtain a new matching block; or

comparing the motion information of a thirteenth sub-block and the motion information of a twenty-first sub-block, and comparing the motion information of a seventeenth sub-block and the motion information of a twenty-second sub-block, to obtain two comparing results, if at least one of the two comparing results indicates different in motion information, vertically offsetting the clipped matching block downward to obtain a new matching block; or

comparing the motion information of a fifteenth sub-block and the motion information of a twenty-third sub-block, and comparing the motion information of a nineteenth sub-block and the motion information of a twenty-fourth sub-block, to obtain two comparing results, if at least one of the two comparing results indicates different in motion information, vertically offsetting the clipped matching block upward to obtain a new matching block;

wherein the thirteenth sub-block is a sub-block in an up-left side of the clipped matching block, the fourteenth sub-block is a first sub-block in a right side of the clipped matching block, the fifteenth sub-block is a sub-block in a down-left side of the clipped matching block, the sixteenth sub-block is a last sub-block in a right side of the clipped matching block, the seventeenth sub-block is a sub-block in an up-right side of the clipped matching block, the eighteenth sub-block is a first sub-block in a left side of the clipped matching block, the nineteenth sub-block is a sub-block in a down-right side of the clipped matching block, the twentieth sub-block is a last sub-block in a left side of the clipped matching block, the twenty-first sub-block is a first sub-block in a down side of the clipped matching block, the twenty-second sub-block is a last sub-block in a down side of the clipped matching block, the twenty-third sub-block is a first sub-block in an up side of the clipped matching block, and the twenty-fourth sub-block is a last sub-block in an up side of the clipped matching block.

5. The decoding method according to claim 2 , wherein, determining the candidate enhanced temporal motion vector prediction modes based on the one or more matching blocks and the one or more new matching blocks obtained by offsetting each of the one or more matching blocks, comprises:

when there are one or more new matching blocks, determining, prediction modes corresponding to the one or more matching blocks before the offsetting and prediction modes corresponding to the one or more new matching blocks obtained by the offsetting, as the candidate enhanced temporal motion vector prediction modes.

6. The decoding method according to claim 1 , wherein, determining the motion information of each sub-block in the current block based on the determined enhanced temporal motion vector prediction mode, further comprises:

if forward motion information and backward motion information of the sub-block in the target matching block are both unavailable,

when both forward motion information and backward motion information of a surrounding block of the current block are available, scaling the forward motion information and the backward motion information of the surrounding block to the first picture in reference picture List 0 and the first picture in reference picture List 1 respectively, and assigning the scaled forward motion information and the scaled backward motion information to a sub-block at a corresponding position in the current block; or

when forward motion information of the surrounding block is available, but backward motion information of the surrounding block is unavailable, scaling the forward motion information of the surrounding block to the first picture in reference picture List 0 , and assigning the scaled forward motion information to a sub-block at a corresponding position in the current block; or

when backward motion information of the surrounding block is available, but forward motion information of the surrounding block is unavailable, scaling the backward motion information of the surrounding block to the first picture in reference picture List 1 , and assigning the scaled backward motion information to a sub-block at a corresponding position in the current block; or

when forward motion information and backward motion information of the surrounding block are both unavailable, assigning zero motion information to a sub-block at a corresponding position in the current block.

7. The decoding method according to claim 1 , wherein, when the sequence parameter set level syntax is used to indicate whether the enhanced temporal motion vector prediction technology is enabled for the current block, whether the enhanced temporal motion vector prediction technology is enabled for the current block is determined by:

when the enhanced temporal motion vector prediction technology is enabled for an image sequence to which the current block belongs, determining that the enhanced temporal motion vector prediction technology is enabled for the current block; or

when the enhanced temporal motion vector prediction technology is disabled for an image sequence to which the current block belongs, determining that the enhanced temporal motion vector prediction technology is disabled for the current block.

8. A non-volatile machine-readable storage medium having stored thereon machine-executable instructions that, when executed by a processor, implement the decoding method according to claim 1 .

9. An encoding method, comprising:

determining one or more matching blocks of a current block, wherein the one or more matching blocks refer to one or more image blocks corresponding to the current block in a target reference picture, comprising:

determining one or more matching blocks of the current block at least based on a target reference picture index, a target reference direction and a position of the current block, wherein the target reference picture index is an index of a first picture in reference picture List 0 , and the target reference direction is a reference direction corresponding to reference picture List 0 ; or, the target reference picture index is an index of a first picture in reference picture List 1 , and the target reference direction is a reference direction corresponding to reference picture List 1 ;

determining candidate enhanced temporal motion vector prediction modes based on the one or more matching blocks and one or more new matching blocks obtained by offsetting each of the one or more matching blocks, and establishing a temporal candidate mode list based on the candidate enhanced temporal motion vector prediction modes;

traversing the candidate enhanced temporal motion vector prediction modes in the temporal candidate mode list, and for any of the candidate enhanced temporal motion vector prediction modes, determining motion information of each sub-block in the current block based on the candidate enhanced temporal motion vector prediction mode, and performing motion compensation on each sub-block in the current block based on the motion information of each sub-block in the current block,

wherein, determining the motion information of each sub-block in the current block based on the candidate enhanced temporal motion vector prediction mode, comprises:

for any sub-block in a target matching block,

if both forward motion information and backward motion information of the sub-block are available, scaling the forward motion information and the backward motion information of the sub-block to the first picture in reference picture List 0 and the first picture in reference picture List 1 respectively, and assigning the scaled forward motion information and the scaled backward motion information to a sub-block at a corresponding position in the current block; or

if forward motion information of the sub-block is available but backward motion information of the sub-block is unavailable, scaling the forward motion information of the sub-block to the first picture in reference picture List 0 , and assigning the scaled forward motion information to a sub-block at a corresponding position in the current block; or

if backward motion information of the sub-block is available but forward motion information of the sub-block is unavailable, scaling the backward motion information of the sub-block to the first picture in reference picture List 1 , and assigning the scaled backward motion information to a sub-block at a corresponding position in the current block,

determining, based on a rate-distortion cost corresponding to each candidate enhanced temporal motion vector prediction mode in the temporal candidate mode list, a candidate enhanced temporal motion vector prediction mode with a smallest rate-distortion cost as an enhanced temporal motion vector prediction mode of the current block;

carrying index information in a bit stream of the current block, wherein the index information is used to indicate a mode index of the determined enhanced temporal motion vector prediction mode.

10. An encoding device, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is configured to execute the machine-executable instructions to implement the encoding method according to claim 9 .

11. A non-volatile machine-readable storage medium having stored thereon machine-executable instructions that, when executed by a processor, implement the encoding method according to claim 9 .

12. A decoding device, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is configured to execute the machine-executable instructions to perform:

obtaining index information of an enhanced temporal motion vector prediction mode from a bit stream of a current block when it is determined that an enhanced temporal motion vector prediction technology is enabled for the current block;

determining one or more matching blocks of the current block, wherein the one or more matching blocks refer to one or more image blocks corresponding to the current block in a target reference picture, comprising:

determining one or more matching blocks of the current block at least based on a target reference picture index, a target reference direction and a position of the current block, wherein the target reference picture index is an index of a first picture in reference picture List 0 , and the target reference direction is a reference direction corresponding to reference picture List 0 ; or, the target reference picture index is an index of a first picture in reference picture List 1 , and the target reference direction is a reference direction corresponding to reference picture List 1 ;

determining candidate enhanced temporal motion vector prediction modes based on the one or more matching blocks and one or more new matching blocks obtained by offsetting each of the one or more matching blocks, and establishing a temporal candidate mode list based on the candidate enhanced temporal motion vector prediction modes;

based on the index information, determining the enhanced temporal motion vector prediction mode from the temporal candidate mode list;

determining motion information of each sub-block in the current block based on the determined enhanced temporal motion vector prediction mode, and performing motion compensation on each sub-block in the current block based on the motion information of each sub-block in the current block,

wherein, determining the motion information of each sub-block in the current block based on the determined enhanced temporal motion vector prediction mode, comprises:

for any sub-block in a target matching block,

if both forward motion information and backward motion information of the sub-block are available, scaling the forward motion information and the backward motion information of the sub-block to the first picture in reference picture List 0 and the first picture in reference picture List 1 respectively, and assigning the scaled forward motion information and the scaled backward motion information to a sub-block at a corresponding position in the current block; or

if forward motion information of the sub-block is available but backward motion information of the sub-block is unavailable, scaling the forward motion information of the sub-block to the first picture in reference picture List 0 , and assigning the scaled forward motion information to a sub-block at a corresponding position in the current block; or

if backward motion information of the sub-block is available but forward motion information of the sub-block is unavailable, scaling the backward motion information of the sub-block to the first picture in reference picture List 1 , and assigning the scaled backward motion information to a sub-block at a corresponding position in the current block,

wherein a sequence parameter set level syntax is used to indicate whether the enhanced temporal motion vector prediction technology is enabled for the current block.

13. The decoding device according to claim 12 , wherein the one or more new matching blocks obtained by offsetting each of the one or more matching blocks are determined by:

for a respective matching block of the one or more matching blocks,

based on one or more offset pairs, horizontally and vertically offsetting the respective matching block to obtain one or more new matching blocks.

14. The decoding device according to claim 12 , wherein after the determining one or more matching blocks of the current block, the processor is further configured to execute the machine-executable instructions to perform:

clipping each of the one or more matching blocks to obtain one or more clipped matching blocks.

15. The decoding device according to claim 14 , wherein the one or more new matching blocks obtained by offsetting each of the one or more matching blocks are determined by:

for a respective clipped matching block of the one or more clipped matching blocks,

comparing the motion information of a thirteenth sub-block and the motion information of a fourteenth sub-block, and comparing the motion information of a fifteenth sub-block and the motion information of a sixteenth sub-block, to obtain two comparing results, if at least one of the two comparing results indicates different in motion information, horizontally offsetting the clipped matching block rightward to obtain a new matching block; or

comparing the motion information of a seventeenth sub-block and the motion information of an eighteenth sub-block, and comparing the motion information of a nineteenth sub-block and the motion information of a twentieth sub-block, to obtain two comparing results, if at least one of the two comparing results indicates different in motion information, horizontally offsetting the clipped matching block leftward to obtain a new matching block; or

comparing the motion information of a thirteenth sub-block and the motion information of a twenty-first sub-block, and comparing the motion information of a seventeenth sub-block and the motion information of a twenty-second sub-block, to obtain two comparing results, if at least one of the two comparing results indicates different in motion information, vertically offsetting the clipped matching block downward to obtain a new matching block; or

comparing the motion information of a fifteenth sub-block and the motion information of a twenty-third sub-block, and comparing the motion information of a nineteenth sub-block and the motion information of a twenty-fourth sub-block, to obtain two comparing results, if at least one of the two comparing results indicates different in motion information, vertically offsetting the clipped matching block upward to obtain a new matching block;

wherein the thirteenth sub-block is a sub-block in an up-left side of the clipped matching block, the fourteenth sub-block is a first sub-block in a right side of the clipped matching block, the fifteenth sub-block is a sub-block in a down-left side of the clipped matching block, the sixteenth sub-block is a last sub-block in a right side of the clipped matching block, the seventeenth sub-block is a sub-block in an up-right side of the clipped matching block, the eighteenth sub-block is a first sub-block in a left side of the clipped matching block, the nineteenth sub-block is a sub-block in a down-right side of the clipped matching block, the twentieth sub-block is a last sub-block in a left side of the clipped matching block, the twenty-first sub-block is a first sub-block in a down side of the clipped matching block, the twenty-second sub-block is a last sub-block in a down side of the clipped matching block, the twenty-third sub-block is a first sub-block in an up side of the clipped matching block, and the twenty-fourth sub-block is a last sub-block in an up side of the clipped matching block.

16. The decoding device according to claim 13 , wherein, in determining the candidate enhanced temporal motion vector prediction modes based on the one or more matching blocks and the one or more new matching blocks obtained by offsetting each of the one or more matching blocks, the processor is further configured to execute the machine-executable instructions to perform:

when there are one or more new matching blocks, determining, prediction modes corresponding to the one or more matching blocks before the offsetting and prediction modes corresponding to the one or more new matching blocks obtained by the offsetting, as the candidate enhanced temporal motion vector prediction modes.

17. The decoding device according to claim 12 , wherein, in determining the motion information of each sub-block in the current block based on the determined enhanced temporal motion vector prediction mode, the processor is further configured to execute the machine-executable instructions to perform:

if forward motion information and backward motion information of the sub-block in the target matching block are both unavailable,

when both forward motion information and backward motion information of a surrounding block of the current block are available, scaling the forward motion information and the backward motion information of the surrounding block to the first picture in reference picture List and the first picture in reference picture List 1 respectively, and assigning the scaled forward motion information and the scaled backward motion information to a sub-block at a corresponding position in the current block; or

when forward motion information of the surrounding block is available, but backward motion information of the surrounding block is unavailable, scaling the forward motion information of the surrounding block to the first picture in reference picture List 0 , and assigning the scaled forward motion information to a sub-block at a corresponding position in the current block; or

when backward motion information of the surrounding block is available, but forward motion information of the surrounding block is unavailable, scaling the backward motion information of the surrounding block to the first picture in reference picture List 1 , and assigning the scaled backward motion information to a sub-block at a corresponding position in the current block; or

when forward motion information and backward motion information of the surrounding block are both unavailable, assigning zero motion information to a sub-block at a corresponding position in the current block.

18. The decoding device according to claim 12 , wherein, when the sequence parameter set level syntax is used to indicate whether the enhanced temporal motion vector prediction technology is enabled for the current block, whether the enhanced temporal motion vector prediction technology is enabled for the current block is determined by:

when the enhanced temporal motion vector prediction technology is enabled for an image sequence to which the current block belongs, determining that the enhanced temporal motion vector prediction technology is enabled for the current block; or

when the enhanced temporal motion vector prediction technology is disabled for an image sequence to which the current block belongs, determining that the enhanced temporal motion vector prediction technology is disabled for the current block.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (33)