IP Library Granted Patent US 10,291,928
Granted Patent B2
US 10,291,928 · App. 15/402,769 · Granted May 14, 2019

Methods and devices for inter-prediction using motion vectors for video coding

Inventor: Dake He (Waterloo, CA)
Assignee: BLACKBERRY LIMITED
H04N19/513H04N19/13H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,291,928
App. No.
15/402,769
Granted
May 14, 2019
Kind
B2
Abstract

Methods and devices for video coding using inter-coding and removing redundant motion vectors from a set of candidate motion vectors. The methods may determine that at least some candidate motion vectors in a set of candidate motion vectors in a search range around the motion vector predictor identify candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminate said at least some of the candidate motion vectors to reduce the set to a subset. In some cases, a candidate block is redundant if it is less than a threshold different from another of the candidate blocks identified by one of the candidate motion vectors in the set. A function may be applied to the candidate block and the another candidate block. If the output from the function in the two cases is the same, then the blocks are determined to be redundant.

Claims (76)

1. A method of encoding a video in a video encoder to produce a bitstream, the video including a sequence of pictures including a current picture and a previously-reconstructed reference picture, the current picture including a current block to be encoded using inter-prediction, the method comprising:

determining, for the current block, a motion vector predictor that identifies a block in the previously-reconstructed reference picture;

determining that at least some candidate motion vectors in a set of candidate motion vectors in a search range around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminating said at least some of the candidate motion vectors to reduce the set to a subset, and wherein said one of the candidate blocks is determined to be redundant based on applying a function to said one of the candidate blocks and to said another of the candidate blocks and determining that the result of said function is identical in both cases;

selecting, from among the candidate motion vectors in the subset, a current motion vector to identify a predicted block for the current block; and

entropy encoding the current motion vector selected from the subset.

2. The method claimed in claim 1 , wherein applying a function comprises computing a distance between said one of the candidate blocks and said another of the candidate blocks by using a metric selected from L 1 , L 2 , L ∞ , mean absolute error, mean squared error, or structural similarity.

3. The method claimed in claim 1 , wherein the function comprises one of a locality-sensitive hashing, a scalar quantization, a vector quantization, a principle component analysis, or an auto-encoder neural network.

4. The method claimed in claim 1 , wherein determining that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminating said at least some of the candidate motion vectors comprises:

for each candidate motion vector in the set, in turn,

determining a candidate block identified by that candidate motion vector, and

adding that candidate motion vector to the subset when the candidate block identified by that candidate motion vector is determined not to be redundant with any other candidate block identified by candidate motion vectors that are part of the subset.

5. The method claimed in claim 1 , wherein the candidate motion vectors in the set include a first layer of candidate motion vectors and a second layer of candidate motion vectors, and wherein determining that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant comprises:

initializing the subset to contain the first layer of candidate motion vectors without determining that they are redundant, and

determining that at least some of the second layer of candidate motion vectors identify respective candidate blocks that are redundant with respect to candidate blocks identified by respective ones of the first layer of candidate motion vectors.

6. The method claimed in claim 1 , wherein the candidate motion vectors include integer-pel motion vectors each pointing to full pixel location and sub-pel motion vectors each pointing to a sub-pixel position between full pixels, and wherein determining that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant comprises determining that more than a maximum number of the sub-pel motion vectors identify respective candidate blocks that are redundant, and wherein eliminating at least some of the candidate motion vectors comprises eliminating all the sub-pel motion vectors.

7. An encoder for encoding a video to produce a bitstream, the video including a sequence of pictures including a current picture and a previously-reconstructed reference picture, the current picture including a current block to be encoded using inter-prediction, the encoder comprising:

a processor;

memory; and

an encoding application containing instructions executable by the processor that, when executed, cause the processor to

determine, for the current block, a motion vector predictor that identifies a block in the previously-reconstructed reference picture;

determine that at least some candidate motion vectors in a set of candidate motion vectors in a search range around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminate said at least some of the candidate motion vectors to reduce the set to a subset, and wherein said one of the candidate blocks is determined to be redundant based on applying a function to said one of the candidate blocks and to said another of the candidate blocks and determining that the result of said function is identical in both cases;

select, from among the candidate motion vectors in the subset, a current motion vector to identify a predicted block for the current block; and

entropy encode the current motion vector selected from the subset.

8. The encoder claimed in claim 7 , wherein said instructions cause the processor to apply a function by determining a distance between said one of the candidate blocks and said another of the candidate blocks using a metric selected from L 1 , L 2 , L ∞ , mean absolute error, mean squared error, or structural similarity.

9. The encoder claimed in claim 7 , wherein the function comprises one of a locality-sensitive hashing, a scalar quantization, a vector quantization, a principle component analysis, or an auto-encoder neural network.

10. The encoder claimed in claim 7 , wherein the instructions, when executed, cause the processor to determine that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminate said at least some of the candidate motion vectors by:

for each candidate motion vector in the set, in turn,

determining a candidate block identified by that candidate motion vector, and

adding that candidate motion vector to the subset when the candidate block identified by that candidate motion vector is determined not to be redundant with any other candidate block identified by candidate motion vectors that are part of the subset.

11. The encoder claimed in claim 7 , wherein the candidate motion vectors in the set include a first layer of candidate motion vectors and a second layer of candidate motion vectors, and wherein the instructions, when executed, cause the processor to determine that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant by:

initializing the subset to contain the first layer of candidate motion vectors without determining that they are redundant, and

determining that at least some of the second layer of candidate motion vectors identify respective candidate blocks that are redundant with respect to candidate blocks identified by respective ones of the first layer of candidate motion vectors.

12. The encoder claimed in claim 7 , wherein the candidate motion vectors include integer-pel motion vectors each pointing to full pixel location and sub-pel motion vectors each pointing to a sub-pixel position between full pixels, and wherein the instructions, when executed, cause the processor to determine that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant by determining that more than a maximum number of the sub-pel motion vectors identify respective candidate blocks that are redundant, and wherein the instructions, when executed, cause the processor to eliminate at least some of the candidate motion vectors by eliminating all the sub-pel motion vectors.

13. A non-transitory processor-readable medium storing processor-executable instructions for encoding a video to generate a bitstream of encoded data, the video including a sequence of pictures including a current picture and a previously-reconstructed reference picture, the current picture including a current block to be encoded using inter-prediction, wherein the processor-executable instructions, when executed by a processor in a video encoder, cause the processor to:

determine, for the current block, a motion vector predictor that identifies a block in the previously-reconstructed reference picture;

determine that at least some candidate motion vectors in a set of candidate motion vectors in a search range around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminate said at least some of the candidate motion vectors to reduce the set to a subset, and wherein said one of the candidate blocks is determined to be redundant based on applying a function to said one of the candidate blocks and to said another of the candidate blocks and determining that the result of said function is identical in both cases;

select, from among the candidate motion vectors in the subset, a current motion vector to identify a predicted block for the current block; and

entropy encode the current motion vector selected from the subset.

14. A method of decoding a video from a bitstream, the video including a sequence of pictures including a current picture and a previously-reconstructed reference picture, the current picture including a current block to be decoded using inter-prediction, the method comprising:

determining, for the current block, a motion vector predictor that identifies a block in the previously-reconstructed reference picture;

determining that at least some candidate motion vectors in a set of candidate motion vectors in a search range around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminating said at least some of the candidate motion vectors to reduce the set to a subset, and wherein said one of the candidate blocks is determined to be redundant based on applying a function to said one of the candidate blocks and to said another of the candidate blocks and determining that the result of said function is identical in both cases;

entropy decoding a current motion vector from the bitstream on the basis that it is one of the candidate motion vectors from the subset, wherein the current motion vector identifies a predicted block for the current block; and

reconstructing the current block based on the predicted block and a reconstructed residual for the current block.

15. The method claimed in claim 14 , wherein applying a function comprises computing a distance between said one of the candidate blocks and said another of the candidate blocks by using a metric selected from L 1 , L 2 , L ∞ , mean absolute error, mean squared error, or structural similarity.

16. The method claimed in claim 14 , wherein the function comprises one of a locality-sensitive hashing, a scalar quantization, a vector quantization, a principle component analysis, or an auto-encoder neural network.

17. The method claimed in claim 14 , wherein determining that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminating said at least some of the candidate motion vectors comprises:

for each candidate motion vector in the set, in turn,

determining a candidate block identified by that candidate motion vector, and

adding that candidate motion vector to the subset when the candidate block identified by that candidate motion vector is determined not to be redundant with any other candidate block identified by candidate motion vectors that are part of the subset.

18. The method claimed in claim 14 , wherein the candidate motion vectors in the set include a first layer of candidate motion vectors and a second layer of candidate motion vectors, and wherein determining that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant comprises:

initializing the subset to contain the first layer of candidate motion vectors without determining that they are redundant, and

determining that at least some of the second layer of candidate motion vectors identify respective candidate blocks that are redundant with respect to candidate blocks identified by respective ones of the first layer of candidate motion vectors.

19. The method claimed in claim 14 , wherein the candidate motion vectors include integer-pel motion vectors each pointing to full pixel location and sub-pel motion vectors each pointing to a sub-pixel position between full pixels, and wherein determining that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant comprises determining that more than a maximum number of the sub-pel motion vectors identify respective candidate blocks that are redundant, and wherein eliminating at least some of the candidate motion vectors comprises eliminating all the sub-pel motion vectors.

20. A decoder for decoding a video from a bitstream, the video including a sequence of pictures including a current picture and a previously-reconstructed reference picture, the current picture including a current block to be decoded using inter-prediction, the decoder comprising:

a processor;

memory; and

a decoding application containing instructions executable by the processor that, when executed, cause the processor to

determine, for the current block, a motion vector predictor that identifies a block in the previously-reconstructed reference picture;

determine that at least some candidate motion vectors in a set of candidate motion vectors in a search range around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminate said at least some of the candidate motion vectors to reduce the set to a subset, and wherein said one of the candidate blocks is determined to be redundant based on applying a function to said one of the candidate blocks and to said another of the candidate blocks and determining that the result of said function is identical in both cases;

entropy decode a current motion vector from the bitstream on the basis that it is one of the candidate motion vectors from the subset, wherein the current motion vector identifies a predicted block for the current block; and

reconstruct the current block based on the predicted block and a reconstructed residual for the current block.

21. The decoder claimed in claim 20 , wherein said instructions cause the processor to apply a function by determining a distance between said one of the candidate blocks and said another of the candidate blocks using a metric selected from L 1 , L 2 , L ∞ , mean absolute error, mean squared error, or structural similarity.

22. The decoder claimed in claim 20 , wherein the function comprises one of a locality-sensitive hashing, a scalar quantization, a vector quantization, a principle component analysis, or an auto-encoder neural network.

23. The decoder claimed in claim 20 , wherein the instructions, when executed, cause the processor to determine that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminate said at least some of the candidate motion vectors by:

for each candidate motion vector in the set, in turn,

determining a candidate block identified by that candidate motion vector, and

adding that candidate motion vector to the subset when the candidate block identified by that candidate motion vector is determined not to be redundant with any other candidate block identified by candidate motion vectors that are part of the subset.

24. The encoder claimed in claim 20 , wherein the candidate motion vectors in the set include a first layer of candidate motion vectors and a second layer of candidate motion vectors, and wherein the instructions, when executed, cause the processor to determine that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant by:

initializing the subset to contain the first layer of candidate motion vectors without determining that they are redundant, and

determining that at least some of the second layer of candidate motion vectors identify respective candidate blocks that are redundant with respect to candidate blocks identified by respective ones of the first layer of candidate motion vectors.

25. The decoder claimed in claim 20 , wherein the candidate motion vectors include integer-pel motion vectors each pointing to full pixel location and sub-pel motion vectors each pointing to a sub-pixel position between full pixels, and wherein the instructions, when executed, cause the processor to determine that at least some candidate motion vectors in a set of candidate motion vectors around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant by determining that more than a maximum number of the sub-pel motion vectors identify respective candidate blocks that are redundant, and wherein the instructions, when executed, cause the processor to eliminate at least some of the candidate motion vectors by eliminating all the sub-pel motion vectors.

26. A non-transitory processor-readable medium storing processor-executable instructions for decoding a video from a bitstream of encoded data, the video including a sequence of pictures including a current picture and a previously-reconstructed reference picture, the current picture including a current block to be decoded using inter-prediction, wherein the processor-executable instructions, when executed by a processor in a video decoder, cause the processor to:

determine, for the current block, a motion vector predictor that identifies a block in the previously-reconstructed reference picture;

determine that at least some candidate motion vectors in a set of candidate motion vectors in a search range around the motion vector predictor identify respective candidate blocks in the previously-reconstructed reference picture that are redundant and, based on that determination, eliminate said at least some of the candidate motion vectors to reduce the set to a subset, and wherein said one of the candidate blocks is determined to be redundant based on applying a function to said one of the candidate blocks and to said another of the candidate blocks and determining that the result of said function is identical in both cases;

entropy decode a current motion vector from the bitstream on the basis that it is one of the candidate motion vectors from the subset, wherein the current motion vector identifies a predicted block for the current block; and

reconstruct the current block based on the predicted block and a reconstructed residual for the current block.

Assignments (7)
CORRECTIVE ASSIGNMENT TO CORRECT THE ADDED PATENT NUMBER TO REMOVE PATENT NO. 8,873,407 AT PREVIOUSLY RECORDED ON REEL 64066 FRAME 1. ASSIGNOR(S) HEREBY CONFIRMS THE NUNC PRO TUNC ASSIGNMENT EFFECTIVE DATE MARCH 20, 2023. Recorded Feb 2, 2026
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 074921/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE COVER SHEET AT PAGE 50 TO REMOVE 12817157 PREVIOUSLY RECORDED ON REEL 063471 FRAME 0474. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Sep 5, 2023
From: BLACKBERRY LIMITED
To: OT PATENT ESCROW, LLC
Reel/Frame 064806/0669 →
CORRECTIVE ASSIGNMENT TO CORRECT 12817157 APPLICATION NUMBER PREVIOUSLY RECORDED AT REEL: 064015 FRAME: 0001. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Sep 5, 2023
From: OT PATENT ESCROW, LLC
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064807/0001 →
NUNC PRO TUNC ASSIGNMENT Recorded Jun 19, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064066/0001 →
NUNC PRO TUNC ASSIGNMENT Recorded Jun 16, 2023
From: OT PATENT ESCROW, LLC
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064015/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 27, 2023
From: BLACKBERRY LIMITED
To: OT PATENT ESCROW, LLC
Reel/Frame 063471/0474 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2017
From: HE, DAKE
To: BLACKBERRY LIMITED
Reel/Frame 040937/0139 →
Continuity (1)
Related Publication 20180199052A1 · Jul 12, 2018
Cited By (1)
US 12,647,597