IP Library Granted Patent US 9,288,507
Granted Patent B2
US 9,288,507 · App. 14/309,711 · Granted Mar 15, 2016

More accurate advanced residual prediction (ARP) for texture coding

Inventors: Li Zhang (San Diego, CA); Ying Chen (San Diego, CA); Marta Karczewicz (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N19/597H04N19/105H04N19/176H04N19/513H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,288,507
App. No.
14/309,711
Granted
Mar 15, 2016
Kind
B2
Abstract

Techniques for advanced residual prediction (ARP) for coding video data may include inter-view ARP. Inter-view ARP may include identifying a disparity motion vector (DMV) for a current video block. The DMV is used for inter-view prediction of the current video block based on an inter-view reference video block. The techniques for inter-view ARP may also include identifying temporal reference video blocks in the current and reference views based on a temporal motion vector (TMV) of the inter-view reference video block, and determining a residual predictor block based on a difference between the temporal reference video blocks.

Claims (91)

1. A method of inter-view advanced residual prediction for decoding video data, the method comprising:

decoding an encoded video bitstream that encodes the video data to identify a disparity motion vector (DMV) and a residual block for a current video block, wherein the current video block is in a current view, and wherein the DMV is used for inter-view prediction of the current video block based on an inter-view reference video block in a reference view and in a same access unit as the current video block;

identifying a temporal motion vector (TMV) and associated reference picture of the inter-view reference video block;

identifying a temporal reference video block in the associated reference picture in the reference view based on the TMV of the inter-view reference video block;

identifying a temporal reference video block in the current view based on the TMV of the inter-view reference video block in the reference view, wherein the temporal reference video block in the current view and the temporal reference video block in the reference view are located in the same access unit;

determining a residual predictor block for the current video block based on a difference between the temporal reference video block in the current view and the temporal reference video block in the reference view; and

applying the residual predictor block and the residual block identified from the encoded video bitstream to the reference video block to reconstruct the current video block.

2. The method of claim 1 , wherein determining the residual predictor block for the current video block based on the difference between the temporal reference video block in the current view and the temporal reference video block in the reference view comprises applying a weighting factor to the difference between the temporal reference video block in the current view and the temporal reference video block in the reference view.

3. The method of claim 1 , further comprising scaling the TMV of the inter-view reference video block to a target reference picture in a target access unit for advanced residual prediction of the current video block, wherein at least one of:

identifying the temporal reference video block in the reference view based on the TMV of the inter-view reference video block comprises identifying the temporal reference video block in the reference view based on the scaled TMV; and

identifying the temporal reference video block in the current view based on the TMV of the inter-view reference video block comprises identifying the temporal reference video block in the reference view based on the scaled TMV.

4. The method of claim 3 , further comprising:

selecting a reference picture list; and

selecting the target reference picture from the selected reference picture list,

wherein selecting the reference picture list comprises one of:

selecting the reference picture list associated with the TMV; or

selecting the reference picture list associated with the DMV.

5. The method of claim 1 , wherein the inter-view reference video block comprises a plurality of prediction units, and identifying the TMV of the inter-view reference video block comprises identifying a TMV associated with one of the plurality of prediction units that contains a center position of the inter-view reference video block.

6. The method of claim 5 , further comprising determining that the prediction unit that contains the center position of the inter-view reference video block does not have a TMV, wherein identifying the TMV of the inter-view reference video block comprises identifying a zero motion vector as the TMV of the inter-view reference video block.

7. The method of claim 6 , wherein identifying the associated reference picture of the inter-view reference video block comprises identifying a reference picture in the reference view which is in the same access unit as a target reference picture used in advanced residual prediction.

8. The method of claim 1 , wherein the inter-view reference video block contains a first set of motion information corresponding to a first reference picture list and a second set of motion information corresponding to a second reference picture list, and identifying the TMV of the inter-view reference video block comprises:

selecting the TMV from the first set of motion information if the first set of motion information includes a TMV; and

selecting the TMV from the second set of motion information if the first set of motion information does not include a TMV.

9. The method of claim 8 , wherein the first reference picture list comprises RefPicList0.

10. The method of claim 8 , wherein an order for considering the first and second sets of motion information is independent of which of the first and second sets of motion information includes the DMV.

11. The method of claim 1 , further comprising:

applying view order difference scaling to scale the identified DMV to a target reference view for advanced residual prediction of the current video block; and

identifying the inter-view reference video block based on the scaled DMV.

12. A method of inter-view advanced residual prediction for encoding video data, the method comprising:

identifying a disparity motion vector (DMV) for a current video block, wherein the current video block is in a current view, and wherein the DMV is used for inter-view prediction of the current video block based on an inter-view reference video block in a reference view and in the same access unit as the current video block;

identifying a temporal motion vector (TMV) and associated reference picture of the inter-view reference video block;

identifying a temporal reference video block in the associated reference picture in the reference view based on the TMV of the inter-view reference video block;

identifying a temporal reference video block in the current view based on the TMV of the inter-view reference video block in the reference view, wherein the temporal reference video block in the current view and the temporal reference video block in the reference view are located in the same access unit;

determining a residual predictor block for the current video block based on a difference between the temporal reference video block in the current view and the temporal reference video block in the reference view; and

encoding an encoded video bitstream that encodes the video data to identify the DMV and a residual block for the current video block, wherein the residual block identified by the encoded video bitstream comprises a difference between the inter-view reference video block and the residual predictor block for the current video block.

13. The method of claim 12 , wherein determining the residual predictor block for the current video block based on the difference between the temporal reference video block in the current view and the temporal reference video block in the reference view comprises applying a weighting factor to the difference between the temporal reference video block in the current view and the temporal reference video block in the reference view.

14. The method of claim 12 , further comprising scaling the TMV of the inter-view reference video block to a target reference picture in a target access unit for advanced residual prediction of the current video block, wherein at least one of:

identifying the temporal reference video block in the reference view based on the TMV of the inter-view reference video block comprises identifying the temporal reference video block in the reference view based on the scaled TMV; and

identifying the temporal reference video block in the current view based on the TMV of the inter-view reference video block comprises identifying the temporal reference video block in the reference view based on the scaled TMV.

15. The method of claim 14 , further comprising:

selecting a reference picture list; and

selecting the target reference picture from the selected reference picture list,

wherein selecting the reference picture list comprises one of:

selecting the reference picture list associated with the TMV; or

selecting the reference picture list associated with the DMV.

16. The method of claim 12 , wherein the inter-view reference video block comprises a plurality of prediction units, and identifying the TMV of the inter-view reference video block comprises identifying a TMV associated with one of the plurality of prediction units that contains a center position of the inter-view reference video block.

17. The method of claim 12 , wherein the inter-view reference video block contains a first set of motion information corresponding to a first reference picture list and a second set of motion information corresponding to a second reference picture list, and identifying the TMV of the inter-view reference video block comprises:

selecting the TMV from the first set of motion information if the first set of motion information includes a TMV; and

selecting the TMV from the second set of motion information if the first set of motion information does not include a TMV.

18. The method of claim 17 , wherein the first reference picture list comprises RefPicList0.

19. The method of claim 17 , wherein an order for considering the first and second sets of motion information is independent of which of the first and second sets of motion information includes the DMV.

20. The method of claim 12 , further comprising:

applying view order difference scaling to scale the identified DMV to a target reference view for advanced residual prediction of the current video block; and

identifying the inter-view reference video block based on the scaled DMV.

21. An apparatus comprising a video coder configured to perform inter-view advanced residual prediction for coding video data, the video coder comprising:

a memory configured to store an encoded video bitstream that encodes the video data; and

one or more processors configured to:

identify a disparity motion vector (DMV) for a current video block, wherein the current video block is in a current view, and wherein the DMV is used for inter-view prediction of the current video block based on an inter-view reference video block in a reference view and in the same access unit as the current video block;

identify a temporal motion vector (TMV) and associated reference picture of the inter-view reference video block;

identifying a temporal reference video block in the associated reference picture in the reference view based on the TMV of the inter-view reference video block;

identifying a temporal reference video block in the current view based on the TMV of the inter-view reference video block in the reference view, wherein the temporal reference video block in the current view and the temporal reference video block in the reference view are located in the same access unit;

determine a residual predictor block for the current video block based on a difference between the temporal reference video block in the current view and the temporal reference video block in the reference view; and

code the encoded video bitstream to identify the DMV and a residual block for the current video block, wherein the residual block identified by coding the encoded video bitstream comprises a difference between the inter-view reference video block and the residual predictor block for the current video block.

22. The apparatus of claim 21 , wherein the one or more processors are further configured to scale the TMV of the inter-view reference video block to a target reference picture in a target access unit for advanced residual prediction of the current video block, and at least one of:

identify the temporal reference video block in the reference view based on the TMV of the inter-view reference video block comprises identifying the temporal reference video block in the reference view based on the scaled TMV; and

identify the temporal reference video block in the current view based on the TMV of the inter-view reference video block comprises identifying the temporal reference video block in the reference view based on the scaled TMV.

23. The apparatus of claim 22 , wherein the one or more processors are further configured to:

select a reference picture list; and

select the target reference picture from the selected reference picture list,

wherein, to select the reference picture list, the one or more processors:

select the reference picture list associated with the TMV; or

select the reference picture list associated with the DMV.

24. The apparatus of claim 21 , wherein the inter-view reference video block comprises a plurality of prediction units, and, to identify the TMV of the inter-view reference video block, the one or more processors identify a TMV associated with one of the plurality of prediction units that contains a center position of the inter-view reference video block.

25. The apparatus of claim 21 , wherein the inter-view reference video block contains a first set of motion information corresponding to a first reference picture list and a second set of motion information corresponding to a second reference picture list, and wherein, to identify the TMV of the inter-view reference video block, the one or more processors:

select the TMV from the first set of motion information if the first set of motion information includes a TMV; and

select the TMV from the second set of motion information if the first set of motion information does not include a TMV.

26. The apparatus of claim 25 , wherein the first reference picture list comprises RefPicList0.

27. The apparatus of claim 21 , wherein the one or more processors are further configured to:

apply view order difference scaling to scale the identified DMV to a target reference view for advanced residual prediction of the current video block; and

identify the inter-view reference video block based on the scaled DMV.

28. The apparatus of claim 21 , wherein the video coder comprises a video decoder, and the one or more processors are configured to:

decode the encoded video bitstream to identify the DMV and the residual block for the current video block; and

apply the residual predictor block and the residual block identified from the encoded video bitstream to the inter-view reference video block to reconstruct the current video block.

29. The apparatus of claim 21 , wherein the video coder comprises a video encoder, and the one or more processors are configured to encode the encoded video bitstream to indicate the DMV and the residual block for the current video block to a video decoder.

30. A computer-readable storage medium having instructions stored thereon that when executed cause one or more processors of a video coder to:

identify a disparity motion vector (DMV) for a current video block, wherein the current video block is in a current view, and wherein the DMV is used for inter-view prediction of the current video block based on an inter-view reference picture in a reference view and in the same access unit as the current video block;

identify a temporal motion vector (TMV) and associated reference picture of the inter-view reference video block;

identifying a temporal reference video block in the associated reference picture in the reference view based on the TMV of the inter-view reference video block;

identifying a temporal reference video block in the current view based on the TMV of the inter-view reference video block in the reference view, wherein the temporal reference video block in the current view and the temporal reference video block in the reference view are located in the same access unit;

determine a residual predictor block for the current video block based on a difference between the temporal reference video block in the current view and the temporal reference video block in the reference view; and

code the encoded video bitstream to identify the DMV and a residual block for the current video block, wherein the residual block identified by coding the encoded video bitstream comprises a difference between the inter-view reference video block and the residual predictor block for the current video block.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 4, 2014
From: ZHANG, LI; CHEN, YING; KARCZEWICZ, MARTA
To: QUALCOMM INCORPORATED
Reel/Frame 033458/0556 →
Continuity (3)
Provisional Application 61838208 · Jun 21, 2013
Provisional Application 61846036 · Jul 14, 2013
Related Publication 20140376633A1 · Dec 25, 2014