IP Library Granted Patent US 11,343,541
Granted Patent B2
US 11,343,541 · App. 17/051,269 · Granted May 24, 2022

Signaling for illumination compensation

Inventors: Chia-Ming Tsai (Hsinchu, TW); Chun-Chia Chen (Hsinchu, TW); Chih-Wei Hsu (Hsinchu, TW); Ching-Yeh Chen (Hsinchu, TW); Tzu-Der Chuang (Hsinchu, TW)
Assignee: HFI INNOVATION INC.
H04N19/85H04N19/139H04N19/159H04N19/176H04N19/52H04N19/96H04N19/117
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,343,541
App. No.
17/051,269
Granted
May 24, 2022
Kind
B2
Abstract

A video coder that implements illumination compensation is provided. The video coder receives a first block of pixels in a first video picture to be coded as a current block, wherein the current block is associated with a motion vector that references a second block of pixels in a second video picture as a reference block. The video coder performs inter-prediction for the current block by using the motion vector to generate a set of motion-compensated pixels for the current block. The video coder modifies the set of motion-compensated pixels of the current block by applying a linear model that is computed based on neighboring samples of the reference block and of the current block. The neighboring samples are identified based on a position of the current block within a larger block.

Claims (32)

1. A method comprising:

receiving a first block of pixels in a first video picture to be coded as a current block, wherein the current block is associated with a motion vector that references a second block of pixels in a second video picture as a reference block;

performing inter-prediction for the current block by using the motion vector to generate a set of motion-compensated pixels for the current block; and

modifying the set of motion-compensated pixels of the current block by applying a linear model that is computed based on neighboring samples of the reference block and of the current block, wherein the neighboring samples are identified based on a position of the current block within a larger block.

2. The method of claim 1 , wherein the linear model is based on a set of illumination compensation parameters comprising a scaling factor and an offset.

3. The method of claim 1 , wherein the larger block comprises a plurality of sub-blocks, wherein a first sub-block is the current block, wherein neighboring pixels of the first sub-block are used as the neighboring samples of the current block to compute the linear model that is used to modify the current block and other sub-blocks of the larger block.

4. The method of claim 1 , wherein the current block is a coding unit (CU) and the larger block is a coding tree unit (CTU) that includes a plurality of CUs, wherein the identified neighboring samples of the current block are derived based on neighboring pixels of the CTU.

5. The method of claim 1 , wherein the current block is a prediction unit (PU) and the larger bock is a coding unit (CU) that includes a plurality of PUs and has a size that is greater than a threshold, wherein the identified neighboring samples of the current block are derived based on neighboring pixels of the CU.

6. The method of claim 1 , wherein the neighboring samples of the current block are neighboring pixels of the larger block that are vertically or horizontally aligned with the current block.

7. The method of claim 1 , wherein the identified neighboring samples of the current block comprises neighboring pixels above the larger block and no other pixels when the current block is at a top edge of the larger block but not at a left edge of the larger block.

8. The method of claim 1 , wherein the identified neighboring samples of the current block comprises neighboring pixels left of the larger block and no other pixels when the current block is at a left edge of the larger block but not at a top edge of the larger block.

9. The method of claim 1 , wherein the identified neighboring samples are computed by using reconstructed pixels based on prediction pixels and residual pixels of a neighboring block when the residual pixels of the neighboring block are available; and by using the prediction pixels of the neighboring block when the residual pixels of the neighboring block are not available.

10. The method of claim 1 , wherein the linear model is inherited from a neighboring block of the current block.

11. The method of claim 1 , wherein the larger block comprises a plurality of sub-blocks that includes the current block, wherein neighboring pixels of the larger block are sub-sampled and used as the neighboring samples to compute the linear model that is used to modify the current block and other sub-blocks of the larger block.

12. The method of claim 1 , wherein performing inter-prediction for the current block comprises generating a set of prediction pixels for the current block, and wherein modifying the set of motion-compensated pixels of the current block comprises applying the linear model to refine the set of prediction pixels of the current block.

13. A method comprising:

receiving a first block of pixels in a first video picture to be coded as a current block, wherein the current block is associated with a motion vector that references a second block of pixels in a second video picture as a reference block;

performing inter-prediction for the current block by using the motion vector to generate a set of motion-compensated pixels for the current block;

determining whether to enable illumination compensation mode based on a prediction setting of the current block; and

modifying the current block by applying a linear model when the illumination compensation mode is enabled.

14. The method of claim 13 , wherein the illumination compensation mode is disabled when a size of the current block is larger than a threshold.

15. The method of claim 13 , wherein determining whether to enable the illumination compensation mode is further based on a position of the current block within a larger block.

16. The method of claim 15 , wherein illumination compensation mode is disabled when the current block is not a top-most block or a left-most block of the larger block.

17. The method of claim 13 , further comprising disabling the illumination compensation mode when at least one of General Bi-prediction (GBi), Bi-directional optical flow (BIO), Sub-PU Temporal Motion Vector Prediction (SPTMVP or ATMVP), Affine mode, Bilateral Template MV Refinement (DMVR), Multi-hypothesis Intra-Merge mode, Multi-hypothesis Merge mode, and Multi-hypothesis Merge+AMVP mode is turned on for the current block.

18. An electronic apparatus comprising:

an encoder circuit capable of:

receiving a first block of pixels in a first video picture to be encoded a current block, wherein the current block is associated with a motion vector that references a second block of pixels in a second video picture as a reference block;

performing inter-prediction for the current block by using the motion vector to generate a set of motion-compensated pixels for the current block;

modifying the set of motion-compensated pixels of the current block by applying a linear model that is computed based on neighboring samples of the reference block and of the current block, wherein the neighboring samples are identified based on a position of the current block within a larger block; and

encoding the modified set of motion-compensated pixels of the current block into a bitstream.

19. The electronic apparatus of claim 18 , wherein the modifying is based on enablement of an illumination compensation mode.

20. The method of claim 1 , wherein the modifying is based on enablement of an illumination compensation mode.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 8, 2022
From: MEDIATEK INC.
To: HFI INNOVATION INC.
Reel/Frame 059339/0015 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2021
From: TSAI, CHIA-MING; CHEN, CHUN-CHIA; HSU, CHIH-WEI; CHEN, CHING-YEH; CHUANG, TZU-DER
To: MEDIATEK INC.
Reel/Frame 054811/0985 →
Continuity (3)
Provisional Application 62785713 · Dec 28, 2018
Provisional Application 62664387 · Apr 30, 2018
Related Publication 20210051345A1 · Feb 18, 2021
Cited By (1)
US 12,684,157