IP Library Granted Patent US 10,721,465
Granted Patent B2
US 10,721,465 · App. 16/274,889 · Granted Jul 21, 2020

Motion compensation for cubemap packed frames

Inventors: Muhammed Zeyd Coban (Carlsbad, CA); Geert Van der Auwera (Del Mar, CA); Fnu Hendry (San Diego, CA); Marta Karczewicz (San Diego, CA)
Assignee: Qualcomm Incorporated
H04N19/105H04N19/159H04N19/176H04N19/46H04N19/513H04N19/563H04N19/597H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,721,465
App. No.
16/274,889
Granted
Jul 21, 2020
Kind
B2
Abstract

A device for decoding 360-degree video data is configured to store a decoded picture of 360-degree video as a reference frame; derive an extended reference frame from the stored reference frame based on a padding amount by extending a first cube face in the reference frame; inter-predict a block of a current picture from a block of the extended reference frame by determining a motion vector for the block of the current picture; in response to a determination that the motion vector points to a cube face in the extended reference frame other than the first cube face, clipping the motion vector such that the motion vector points to a location in the first cube face; and locating a prediction block for a current block using the clipped motion vector.

Claims (75)

1. A device for decoding 360-degree video data, the device comprising:

one or more memory devices configured to store an encoded picture of the 360-degree video data; and

one or more processors implemented in circuitry and configured to:

receive, in a parameter set of the 360-degree video data, a syntax element indicating a padding amount;

decode an encoded picture of the 360-degree video data to determine a decoded picture of 360-degree video;

store the decoded picture of 360-degree video as a reference frame, wherein the reference frame is packed according to a packing structure, wherein the packing structure comprises one of a cubemap or an adjusted cubemap projection (ACP);

derive an extended reference frame from the stored reference frame based on the padding amount by extending a first cube face in the reference frame;

inter-predict a block of a current picture from a block of the extended reference frame, wherein the current picture is also packed according to the packing structure and wherein to inter-predict the block of the current picture from the block of the extended reference frame, the one or more processors are further configured to:

determine a motion vector for the block of the current picture;

in response to a determination that the motion vector points to a cube face in the extended reference frame other than the first cube face, clip the motion vector such that the motion vector points to a location in the first cube face;

determine a prediction block for a current block of the video data using the clipped motion vector; and

perform overlapped block motion compensation for the prediction block without performing overlapped block motion compensation at a boundary between the first cube face and a second cube face.

2. The device of claim 1 , wherein the determination that the motion vector points to the cube face in the extended reference frame other than the first cube face comprises a determination that the first cube face comprises a cube face in one of a top of the reference frame or a bottom of the reference frame and determining that the cube face in the extended reference frame other than the first cube face comprises a cube face in the other of the top of the reference frame or the bottom of the reference frame.

3. The device of claim 1 , wherein the parameter set comprises a sequence parameter set.

4. The device of claim 1 , wherein the padding amount is equal to 8*(a value of the syntax element plus 1).

5. The device of claim 1 , wherein the syntax element comprises a fixed length coded syntax element.

6. The device of claim 1 , wherein the one or more processors are further configured to:

receive a flag indicating if padding is to be used for motion compensation; and

receive, in the parameter set, the syntax element indicating the padding amount in response to the flag indicating that padding is to be used for motion compensation.

7. The device of claim 1 , wherein the one or more processors are further configured to:

determine the encoded picture of 360-degree video data is packed in a cubemap or adjusted cubemap projection (ACP); and

receive, in the parameter set, the syntax element indicating the padding amount in response to determining that the encoded picture of 360-degree video data is packed in the cubemap or ACP.

8. The device of claim 1 , wherein the device comprises a wireless communication device that comprises a receiver configured to receive the 360-degree video data and demodulate, according to a wireless communication standard, a signal comprising the 360-degree video data.

9. A device for encoding 360-degree video data, the device comprising:

one or more memory devices; and

one or more processors implemented in circuitry and configured to:

decode an encoded picture of the 360-degree video data to determine a decoded picture of 360-degree video;

store the decoded picture of 360-degree video as a reference frame, wherein the reference frame is packed according to a packing structure, wherein the packing structure comprises one of a cubemap or an adjusted cubemap projection (ACP);

determine a padding amount;

derive an extended reference frame from the stored reference frame based on the padding amount by extending a first cube face in the reference frame;

inter-predict a block of a current picture from a block of the extended reference frame, wherein the current picture is also packed according to the packing structure and wherein to inter-predict the block of the current picture from the block of the extended reference frame, the one or more processors are further configured to:

determine a motion vector for the block of the current picture;

in response to a determination that the motion vector points to a cube face in the extended reference frame other than the first cube face, clip the motion vector such that the motion vector points to a location in the first cube face;

determine a prediction block for a current block of the video data using the clipped motion vector; and

perform overlapped block motion compensation for the prediction block without performing overlapped block motion compensation at a boundary between the first cube face and a second cube face.

10. The device of claim 9 , wherein to determine the motion vector for the block of the current picture, the one or more processors are further configured to:

perform motion estimation for the block of the current picture, wherein a search range for performing the motion estimation for the block of the current picture is restricted to the first cube face.

11. The device of claim 9 , wherein the device comprises a wireless communication device, further comprising a transmitter configured to transmit encoded video data.

12. The device of claim 11 , wherein the wireless communication device comprises a telephone handset and wherein the transmitter is configured to modulate, according to a wireless communication standard, a signal comprising the encoded 360-degree video data.

13. A method of decoding 360-degree video data, the method comprising:

receiving, in a parameter set of the 360-degree video data, a syntax element indicating a padding amount;

decoding an encoded picture of the 360-degree video data to determine a decoded picture of 360-degree video;

storing the decoded picture of 360-degree video as a reference frame, wherein the reference frame is packed according to a packing structure, wherein the packing structure comprises one of a cubemap or an adjusted cubemap projection (ACP);

deriving an extended reference frame from the stored reference frame based on the padding amount by extending a first cube face in the reference frame;

inter-predicting a block of a current picture from a block of the extended reference frame, wherein the current picture is also packed according to the packing structure and wherein inter-predicting the block of the current picture from the block of the extended reference frame comprises:

determining a motion vector for the block of the current picture;

in response to a determination that the motion vector points to a cube face in the extended reference frame other than the first cube face, clipping the motion vector such that the motion vector points to a location in the first cube face; and

determining a prediction block for a current block of the video data using the clipped motion vector; and

performing overlapped block motion compensation for the prediction block without performing overlapped block motion compensation at a boundary between the first cube face and a second cube face.

14. The method of claim 13 , wherein the determination that the motion vector points to the cube face in the extended reference frame other than the first cube face comprises determining that the first cube face comprises a cube face in one of a top of the reference frame or a bottom of the reference frame and determining that the cube face in the extended reference frame other than the first cube face comprises a cube face in the other of the top of the reference frame or the bottom of the reference frame.

15. The method of claim 13 , wherein determining the motion vector for the block of the current picture comprises copying a motion vector of a neighboring block of the block of the current picture.

16. The method of claim 13 , wherein the parameter set comprises a sequence parameter set.

17. The method of claim 13 , wherein the padding amount is equal to 8*(a value of the syntax element plus 1).

18. The method of claim 13 , wherein the syntax element comprises a fixed length coded syntax element.

19. The method of claim 13 , further comprising:

receiving a flag indicating if padding is to be used for motion compensation; and

receiving, in the parameter set, the syntax element indicating the padding amount in response to the flag indicating that padding is to be used for motion compensation.

20. The method of claim 13 , further comprising:

determining the encoded picture is packed in a cubemap or adjusted cubemap projection (ACP); and

receiving, in the parameter set, the syntax element indicating the padding amount in response to determining that the encoded picture of 360-degree video data is packed in the cubemap or ACP.

21. A non-transitory computer-readable storage medium storing instructions that when executed by one or more processors cause the one or more processors to:

receive, in a parameter set of the 360-degree video data, a syntax element indicating a padding amount;

decode an encoded picture of the 360-degree video data to determine a decoded picture of 360-degree video;

store the decoded picture of 360-degree video as a reference frame, wherein the reference frame is packed according to a packing structure, wherein the packing structure comprises one of a cubemap or an adjusted cubemap projection (ACP);

derive an extended reference frame from the stored reference frame based on the padding amount by extending a first cube face in the reference frame;

inter-predict a block of a current picture from a block of the extended reference frame, wherein the current picture is also packed according to the packing structure and wherein to inter-predict the block of the current picture from the block of the extended reference frame, the one or more processors are further configured to:

determine a motion vector for the block of the current picture;

in response to a determination that the motion vector points to a cube face in the extended reference frame other than the first cube face, clip the motion vector such that the motion vector points to a location in the first cube face;

determine a prediction block for a current block of the video data using the clipped motion vector; and

perform overlapped block motion compensation for the prediction block without performing overlapped block motion compensation at a boundary between the first cube face and a second cube face.

22. The non-transitory computer-readable storage medium of claim 21 , wherein the determination that the motion vector points to the cube face in the extended reference frame other than the first cube face comprises a determination that the first cube face comprises a cube face in one of a top of the reference frame or a bottom of the reference frame and determining that the cube face in the extended reference frame other than the first cube face comprises a cube face in the other of the top of the reference frame or the bottom of the reference frame.

23. The non-transitory computer-readable storage medium of claim 21 , wherein the parameter set comprises a sequence parameter set.

24. The non-transitory computer-readable storage medium of claim 21 , wherein the padding amount is equal to 8*(a value of the syntax element plus 1).

25. The device of claim 1 , wherein the one or more processors are further configured to:

receive, in a sequence parameter set of the 360-degree video data, a syntax element indicating the padding amount, wherein the syntax element comprises a 5-bit, fixed length coded syntax element, and wherein the padding amount is equal to 8*(a value of the syntax element plus 1).

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 9, 2019
From: COBAN, MUHAMMED ZEYD; VAN DER AUWERA, GEERT; HENDRY, FNU; KARCZEWICZ, MARTA
To: QUALCOMM INCORPORATED
Reel/Frame 049123/0612 →
Continuity (2)
Provisional Application 62630700 · Feb 14, 2018
Related Publication 20190253703A1 · Aug 15, 2019