IP Library Granted Patent US 12,192,513
Granted Patent B2
US 12,192,513 · App. 18/460,607 · Granted Jan 7, 2025

Method and apparatus for intra prediction using an interpolation filter

Inventors: Alexey Konstantinovich Filippov (Moscow, RU); Vasily Alexeevich Rufitskiy (Moscow, RU); Jianle Chen (San Diego, CA)
Assignee: Huawei Technologies Co., Ltd.
H04N19/59H04N19/105H04N19/117H04N19/159H04N19/176H04N19/593
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,192,513
App. No.
18/460,607
Granted
Jan 7, 2025
Kind
B2
Abstract

Provided are a method, an apparatus, a computer program product and a non-transitory computer-readable medium for video coding. The method comprises performing an intra-prediction process of a block comprising samples to be predicted, wherein an interpolation filter is applied to reference samples of the block during the intra-prediction process of the block. The interpolation filter is selected on the basis of a subpixel offset between the reference samples and the samples to be predicted, and a size of a main reference side used in the intra-prediction process is determined according to a length of the interpolation filter and an intra-prediction mode that provides, out of a set of available intra-prediction modes, the greatest non-integer value of said subpixel offset, wherein the main reference side comprises the reference samples.

Claims (566)

1. A method of video encoding implemented by an encoding device, the method comprising:

performing an intra-prediction process of a block to obtain predicted sample values of the block, wherein an interpolation filter is applied to reference samples of the block during the intra-prediction process of the block;

obtaining residual information according to sample values of the block and the predicted sample values of the block; and

performing transformation, quantization, and entropy coding on the residual information, to obtain an encoded bitstream,

wherein the interpolation filter is selected on the basis of a subpixel offset between the reference samples and the predicted samples, and

wherein a size of a main reference side used in the intra-prediction process is determined as a sum of:

an integer part of a greatest non-integer value of the subpixel offset, wherein an intra-prediction mode provides, out of a set of available intra-prediction modes, the greatest non-integer value of said subpixel offset,

a size of a side of the block, and

a half of the length of the interpolation filter.

2. The method of claim 1 , wherein

the side of the block of predicted samples is a width of the block when the intra prediction mode is greater than a vertical intra prediction mode (VER_IDX);

or

the side of the block is a height of the block when the intra prediction mode is lesser than a horizontal intra prediction mode (HOR_IDX).

3. The method of claim 1 , wherein, in the main reference side, values of reference samples with positions greater than a doubled size of the side of the block are set to be equal to a value of a sample located at the doubled size of the size of the side of the block.

4. The method of claim 1 , wherein padding is performed by replicating first and/or last reference samples of the main reference side to the left and/or to the right side respectively, in particular as follows: denoting the main reference side as ref, and the size of the main reference side as refS, wherein the padding is represented as:

ref[−1]= p[ 0], and/or

ref[refS+1]= p [refS],

wherein ref [−1] represents a value of the left to the main reference side,

p [0] represents a value of the first reference sample of the main reference side,

ref [refS+1] represents a value of the right to the main reference side, and

p [refS] represents a value of the last reference sample of the main reference side.

5. The method of claim 1 , wherein the interpolation filter used in the intra-prediction processes is a finite impulse response filter and the coefficients of the interpolation filter are fetched from a look-up table.

6. The method of claim 1 , wherein the interpolation filter used in the intra-prediction process is a 4-tap filter.

7. The method of claim 6 , wherein coefficients, c 0 , c 1 , c 2 , and c 3 , of the interpolation filter depend on a non-integer part of the subpixel offset as follows:

Non-integer

part of the

subpixel

offset

c 0

c 1

c 2

c 3

 0 (integer)

0

64

0

0

 1

−1

63

2

0

 2

−2

62

4

0

 3

−2

60

7

−1

 4

−2

58

10

−2

 5

−3

57

12

−2

 6

−4

56

14

−2

 7

−4

55

15

−2

 8

−4

54

16

−2

 9

−5

53

18

−2

10

−6

52

20

−2

11

−6

49

24

−3

12

−6

46

28

−4

13

−5

44

29

−4

14

−4

42

30

−4

15

−4

39

33

−4

16 (half-pel)

−4

36

36

−4

17

−4

33

39

−4

18

−4

30

42

−4

19

−4

29

44

−5

20

−4

28

46

−6

21

−3

24

49

−6

22

−2

20

52

−6

23

−2

18

53

−5

24

−2

16

54

−4

25

−2

15

55

−4

26

−2

14

56

−4

27

−2

12

57

−3

28

−2

10

58

−2

29

−1

7

60

−2

30

0

4

62

−2

31

0

2

63

−1

wherein the “Non-integer part of the subpixel offset” column is defined in 1/32 subpixel resolution.

8. The method of claim 6 , wherein coefficients, c 0 , c 1 , c 2 , and c 3 , of the interpolation filter depend on a non-integer part of the subpixel offset as follows:

Non-integer

part of the

subpixel

offset

c 0

c 1

c 2

c 3

 0 (integer)

16

32

16

0

 1

15

29

17

3

 2

15

29

17

3

 3

14

29

18

3

 4

13

29

18

4

 5

13

28

19

4

 6

13

28

19

4

 7

12

28

20

4

 8

11

28

20

5

 9

11

27

21

5

10

10

27

22

5

11

9

27

22

6

12

9

26

23

6

13

9

26

23

6

14

8

25

24

7

15

8

25

24

7

16 (half-pel)

8

24

24

8

17

7

24

25

8

18

7

24

25

8

19

6

23

26

9

20

6

23

26

9

21

6

22

27

9

22

5

22

27

10

23

5

21

27

11

24

5

20

28

11

25

4

20

28

12

26

4

19

28

13

27

4

19

28

13

28

4

18

29

13

29

3

18

29

14

30

3

17

29

15

31

3

17

29

15

wherein the “Non-integer part of the subpixel offset” column is defined in 1/32 subpixel resolution.

9. The method of claim 6 , wherein coefficients, c 0 , c 1 , c 2 , and c 3 , of the interpolation filter depend on a non-integer part of the subpixel offset as follows:

Non-integer

part of the

subpixel

offset

c 0

c 1

c 2

c 3

 0 (integer)

16

32

16

0

 1

15

29

17

3

 2

15

29

17

3

 3

14

29

18

3

 4

13

29

18

4

 5

13

28

19

4

 6

13

28

19

4

 7

12

28

20

4

 8

11

28

20

5

 9

11

27

21

5

10

10

27

22

5

11

9

27

22

6

12

9

26

23

6

13

9

26

23

6

14

8

25

24

7

15

8

25

24

7

16 (half-pel)

8

24

24

8

17

7

24

25

8

18

7

24

25

8

19

6

23

26

9

20

6

23

26

9

21

6

22

27

9

22

5

22

27

10

23

5

21

27

11

24

5

20

28

11

25

4

20

28

12

26

4

19

28

13

27

4

19

28

13

28

4

18

29

13

29

3

18

29

14

30

3

17

29

15

31

3

17

29

15

wherein the “Non-integer part of the subpixel offset” column is defined in 1/32 subpixel resolution.

10. The method of claim 1 , wherein the interpolation filter is selected from a set of filters used for the intra-prediction process for a given subpixel offset.

11. The method of claim 10 , wherein the set of filters comprises a Gauss filter and a Cubic filter.

12. The method of claim 1 , wherein the quantity of the interpolation filter is N, and wherein the N interpolation filters are used for intra reference sample interpolation and N>=1 and is a positive integer.

13. The method of claim 1 , wherein the reference samples include samples not adjacent to the block.

14. An encoding device, comprising:

at least one processor; and

one or more memories coupled to the at least one processor and storing programming instructions for execution by the at least one processor to cause the encoding device to:

perform an intra-prediction process of a block to obtain predicted sample values of the block,

wherein an interpolation filter is applied to reference samples of the block during the intra-prediction process of the block;

obtain residual information according to sample values of the block and the predicted sample values of the block; and

perform transformation, quantization, and entropy coding on the residual information, to obtain an encoded bitstream,

wherein the interpolation filter is selected on the basis of a subpixel offset between the reference samples and the predicated samples, and

wherein a size of a main reference side used in the intra-prediction process is determined as a sum of:

an integer part of a greatest non-integer value of the subpixel offset, wherein an intra-prediction mode provides, out of a set of available intra-prediction modes, the greatest non-integer value of said subpixel offset,

a size of a side of the block, and

a half of the length of the interpolation filter.

15. The encoding device of claim 14 , wherein

the side of the block of predicted samples is a width of the block when the intra prediction mode is greater than a vertical intra prediction mode (VER_IDX);

or

the side of the block is a height of the block when the intra prediction mode is lesser than a horizontal intra prediction mode (HOR_IDX).

16. The encoding device of claim 14 , wherein, in the main reference side, values of reference samples with positions greater than a doubled size of the side of the block are set to be equal to a value of a sample located at the doubled size of the size of the side of the block.

17. The encoding device of claim 14 , wherein the interpolation filter used in the intra-prediction processes is a finite impulse response filter and the coefficients of the interpolation filter are fetched from a look-up table.

18. The encoding device of claim 14 , wherein the interpolation filter used in the intra-prediction process is a 4-tap filter.

19. The encoding device of claim 14 , wherein the interpolation filter is selected from a set of filters used for the intra-prediction process for a given subpixel offset.

20. A non-transitory computer readable medium having an encoded bitstream for a video signal stored therein, the encoded bitstream is generated by a video encoding method, wherein the method comprises:

performing an intra-prediction process of a block to obtain predicted sample values of the block, wherein an interpolation filter is applied to reference samples of the block during the intra-prediction process of the block;

obtaining residual information according to sample values of the block and predicted sample values of the block; and

performing transformation, quantization, and entropy coding on the residual information, to obtain the encoded bitstream;

wherein the interpolation filter is selected on the basis of a subpixel offset between the reference samples and the predicted samples, and

wherein a size of a main reference side used in the intra-prediction process is determined as a sum of:

an integer part of a greatest non-integer value of the subpixel offset, wherein an intra-prediction mode provides, out of a set of available intra-prediction modes, the greatest non-integer value of said subpixel offset,

a size of a side of the block, and

a half of the length of the interpolation filter.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 25, 2024
From: FILIPPOV, ALEXEY KONSTANTINOVICH; RUFITSKIY, VASILY ALEXEEVICH; CHEN, JIANLE
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 067835/0059 →
Continuity (7)
Continuation 17223758 · Apr 6, 2021
Continuation PCTRU2019050178 · Oct 7, 2019
Provisional Application 62757150 · Nov 7, 2018
Provisional Application 62742300 · Oct 6, 2018
Provisional Application 62744096 · Oct 10, 2018
Provisional Application 62753055 · Oct 30, 2018
Related Publication 20240121432A1 · Apr 11, 2024
References Cited (34)
US 9451254B2 · Joshi et al. · 2016 [cited by applicant]
US 11750837B2 · Filippov et al. · 2023 [cited by applicant]
US 20080123974A1 · Zhu · 2008 [cited by applicant]
US 20080130742A1 · Scheuermann · 2008 [cited by examiner]
US 20120163460A1 · Chien et al. · 2012 [cited by applicant]
US 20120170650A1 · Chong et al. · 2012 [cited by applicant]
US 20120243611A1 · Kondo · 2012 [cited by applicant]
US 20120288004A1 · Sato · 2012 [cited by applicant]
US 20130188702A1 · Li et al. · 2013 [cited by applicant]
US 20140233646A1 · Matsuo et al. · 2014 [cited by applicant]
US 20150023405A1 · Joshi et al. · 2015 [cited by applicant]
US 20150189314A1 · Alshina et al. · 2015 [cited by applicant]
US 20150237358A1 · Alshin et al. · 2015 [cited by applicant]
US 20160373743A1 · Zhao et al. · 2016 [cited by applicant]
US 20180020218A1 · Zhao et al. · 2018 [cited by applicant]
US 20180091825A1 · Zhao et al. · 2018 [cited by applicant]
US 20200014922A1 · Ramasubramonian et al. · 2020 [cited by applicant]
US 20200213590A1 · Kim et al. · 2020 [cited by applicant]
US 20220053213A1 · Yoo et al. · 2022 [cited by applicant]
CN 108605124A · 2018 [cited by applicant]
KR 20170116850A · 2017 [cited by applicant]
ITU-T H.261 (03/93), Line Transmission of Non-Telephone Signals, Video Codec for Audiovisual Services AT p x 64 kbits, total 29 pages. [cited by applicant]
Fabien Racape et al, CE3-related: Wide-angle intra prediction for non-square blocks, JVET-K0500-r3, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG11, 11th Meeting: Ljubljana, SI, Jul. 10-… [cited by applicant]
Document: JVET-L0324-v2, Geert Van der Auwera et al., CE3: Intra reference sample interpolation filter selection using MDIS conditions (Test 3.1.2), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/… [cited by applicant]
Document: JVET-L0628-v1, Alexey Filippov et al, CE3: A combination of tests 3.1.2 and 3.1.4 for intra reference sample interpolation filter, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG… [cited by applicant]
ITU-T H.264(Apr. 2017), Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services-Coding of moving video, Advanced video coding for generic audiovisual services, total 812 pages. [cited by applicant]
Alexey Filippov et al., CE3: A combination of tests 3.1.2 and 3.1.4 for intra reference sample interpolation filter, JVET-L0xxx-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG11, 12th … [cited by applicant]
Pan Zhi-hong et al, A Intra Prediction Mode Algorithm based on AVS, 2010, 3 pages. [cited by applicant]
ITU-T H.265(Feb. 2018), Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services-Coding of moving video, High efficiency video coding, total 692 pages. [cited by applicant]
Sunmi Yoo et al, CE3 Related : Extended reference sample c omtruction for longer interpolation filter in intra prediction, JVET-L0291-v3, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG11,… [cited by applicant]
Benjamin Bross et al, CE3: Multiple reference line intra prediction (Test 1.1.1, 1.1.2 1.1.3 and 1.1.4), JVET-L0283-v2, Joint Video Experts Team (JVET)of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG11, 12th Meeting: Maca… [cited by applicant]
Document: JVET-K1001-v6, Benjamin Bross et al., Versatile Video Coding (Draft 2), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018, 141 pag… [cited by applicant]
Document: JVET-L0275-v1, Alexey Filippov et al., CE3: Intra reference sample interpolation filter (Test 3.1.4), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, MO,… [cited by applicant]
Document: JVET-L0279, Liang Zhao et al., CE3-related: Unification of angular intra prediction for square and non-square blocks, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Mee… [cited by applicant]
Cited By (1)
US 12,348,788