IP Library Granted Patent US 10,284,869
Granted Patent B2
US 10,284,869 · App. 15/718,082 · Granted May 7, 2019

Constrained motion field estimation for hardware efficiency

Inventors: Jingning Han (Santa Clara, CA); Yaowu Xu (Saratoga, CA); James Bankoski (Los Gatos, CA); Jia Feng (San Jose, CA)
Assignee: GOOGLE LLC
H04N19/51H04N19/176H04N19/182H04N19/196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,284,869
App. No.
15/718,082
Granted
May 7, 2019
Kind
B2
Abstract

Motion field estimation is used to predict motion within video blocks. A current block of a current frame is identified as a projection of a first reference block of a first reference frame onto the current frame. The projection uses a first motion vector of the reference block with respect to a second reference frame. A temporal motion vector candidate is determined for the current block by projecting the current block onto a second reference frame. The temporal motion vector candidate identifies a second reference block. A motion vector selected from a motion vector candidate list is used to generate a prediction block using the selected motion vector. The current block is coded using the prediction block.

Claims (63)

1. A method for coding a current block of a current frame, the method comprising:

projecting a first reference block of a first reference frame onto the current frame to identify the current block,

wherein the first reference block is inter-predicted from a second reference frame using a first motion vector, and

wherein the projecting uses

the first motion vector,

a first difference between display orders of the current frame and the first reference frame, and

a second difference between display orders of the first reference frame and the second reference frame;

determining a temporal motion vector candidate for the current block by projecting the current block onto a third reference frame, wherein the temporal motion vector candidate identifies a second reference block in the third reference frame, wherein projecting the current block onto the third reference frame uses respective display orders of the current frame, the first reference frame, the second reference frame, and the third reference frame;

adding the temporal motion vector candidate to a motion vector candidate list;

selecting a motion vector from the motion vector candidate list;

generating a prediction block using the selected motion vector; and

coding the current block using the prediction block.

2. The method of claim 1 , wherein the first reference block is within a range of the current block.

3. The method of claim 2 , wherein the range is a −32 to +32 pixels of the current block in a vertical direction and −64 to +64 pixels in a horizontal direction.

4. The method of claim 1 , further comprising:

aligning a result of the projecting such that the current block aligns with a block of the current frame.

5. The method of claim 1 , determining the temporal motion vector candidate for the current block by projecting the current frame onto the second reference frame comprises:

aligning a result of the projecting such that the second reference block aligns with a predefined sub-pixel location of the second reference frame.

6. The method of claim 1 , wherein coding the current block using the prediction block comprises:

decoding, from an encoded bitstream, a packet comprising the current frame and a packet stamp, the packet stamp indicating whether the current frame is to be displayed when packets having the packet stamp are received.

7. The method of claim 1 , wherein coding the current block using the prediction block comprises:

encoding, in an encoded bitstream, a packet wherein the packet comprises the current frame and a packet stamp, the packet stamp indicating whether the current frame is to be displayed by a decoder when packets having the packet stamp are received.

8. An apparatus of decoding a current block of a current frame, the apparatus comprising:

a processor configured to execute instructions stored in a non-transitory memory to:

select reference blocks of a first reference frame, wherein the reference blocks constitute a subset of blocks of the first reference frame;

determine temporal motion vector candidates for the current block using respective motion vectors of the reference blocks by instructions to:

project a reference block of the reference blocks onto the current frame to identify the current block,

wherein the reference block is inter-predicted from a second reference frame using a first motion vector, and

wherein the projecting uses

 the first motion vector,

 a first difference between display orders of the current frame and the first reference frame, and

 a second difference between display orders of first reference frame and the second reference frame;

add the temporal motion vector candidate to a motion vector candidate list;

select a motion vector from the motion vector candidate list;

generate a prediction block using the selected motion vector; and

decode, from an encoded bitstream, the current block using the prediction block.

9. The apparatus of claim 8 , wherein the reference blocks are within a range of the current block.

10. The apparatus of claim 9 , wherein the range is a −32 to +32 pixels of the current block in a vertical direction and −64 to +64 pixels in a horizontal direction.

11. The apparatus of claim 9 , wherein the range is selected, in at least one of a horizontal direction and a vertical direction, based on a size of a processing unit of the current frame.

12. The method of claim 1 , wherein the projection of the first reference block of the first reference frame onto the current frame being a linear projection according to the respective display orders of the current frame, the first reference frame, and the second reference frame.

13. The apparatus of claim 8 , wherein to project the reference block onto the second reference frame comprises to:

project the reference block of the reference blocks onto the second reference frame using a motion vector of the reference block with respect to a third reference frame, a frame index of the current frame, a first reference index of the first reference frame, and a second frame index of the second reference frame.

14. The apparatus of claim 13 , wherein the frame index of the current frame, the first reference index of the first reference frame, and the second frame index of the second reference frame correspond, respectively, to a display order of the current frame, the first reference frame, and the second reference frame.

15. The apparatus of claim 13 , wherein the instructions further include instructions to:

determine the frame index by instructions to:

receive a packet comprising the current frame;

decode, from the packet, a packet stamp indicative of a frame to be displayed;

decode from a header of the current frame, an offset value indicative of a number of packet stamps to be received before the current frame is displayed; and

determine the frame index using the offset value and the packet stamp.

16. An apparatus for decoding an encoded block of an encoded frame processed in processing units, wherein a processing unit of the encoded frame includes the encoded block, the apparatus comprising:

a processor configured to execute instructions stored in a non-transitory memory to:

select motion vectors corresponding to blocks of an extended collocated processing unit in a first reference frame;

identify a block of the blocks, such that the encoded block is a projection, using a motion vector of the block that refers to a third reference frame, onto the encoded block, wherein the projection comprises a scaling of the motion vector of the block using respective display orders of the encoded frame, the first reference frame, and the third reference frame;

on condition that a block is identified,

determine a temporal motion vector candidate for the encoded block in a second reference frame using a motion vector of the identified block and respective display orders of the encoded frame, the first reference frame, the second reference frame, and the third reference frame, and

add the temporal motion vector candidate to a motion vector candidate list;

select a motion vector from the motion vector candidate list;

generate a prediction block using the selected motion vector; and

decode the encoded block using the prediction block.

17. The apparatus of claim 16 , where in the processing unit is 64×64.

18. The apparatus of claim 17 , wherein the extended collocated processing unit extends in a range of −32 to +32 pixels in a height direction and a range of −64 to +64 pixels in a width direction of the processing unit.

19. The apparatus of claim 16 ,

wherein the respective display positions are determined using header information comprising frame offset values and packet stamp values.

Assignments (2)
MERGER AND CHANGE OF NAME Recorded Mar 20, 2018
From: GOOGLE INC.; GOOGLE LLC
To: GOOGLE LLC
Reel/Frame 045643/0031 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2017
From: HAN, JINGNING; XU, YAOWU; BANKOSKI, JAMES; FENG, JIA
To: GOOGLE INC.
Reel/Frame 043913/0594 →
Continuity (1)
Related Publication 20190098329A1 · Mar 28, 2019
Cited By (6)
US 12,368,880 US 12,401,820 US 12,549,756 US 12,556,738 US 12,604,029 US 12,641,216