Multiple color channel multiple regression predictor
Inter-color image prediction is based on multi-channel multiple regression (MMR) models. Image prediction is applied to the efficient coding of images and video signals of high dynamic range. MMR models may include first order parameters, second order parameters, and cross-pixel parameters. MMR models using extension parameters incorporating neighbor pixel relations are also presented. Using minimum means-square error criteria, closed form solutions for the prediction parameters are presented for a variety of MMR models.
1. A method comprising:
providing a variety of multi-channel, multiple-regression (MMR) prediction models, each MMR prediction model adapted to approximate an image having a first dynamic range in terms of
an image having a second dynamic range, and
prediction parameters of the respective MMR prediction model, by applying inter-color image prediction;
receiving a first image and a second image, wherein the second image has a different dynamic range than the first image;
selecting a multi-channel, multiple-regression (MMR) prediction model from the variety of MMR models;
determining values of the prediction parameters of the selected MMR model;
computing an output image approximating the first image based on the second image and the determined values of the prediction parameters applied to the selected MMR prediction model;
outputting the determined values of the prediction parameters and the computed output image, wherein the variety of MMR models includes a first order multi-channel, multiple regression prediction model incorporating cross-multiplications between the color components of each pixel according to the formula
{circumflex over (v)} i =sc i {tilde over (C)} (1) +s i {tilde over (M)} (1) +n
wherein
{circumflex over (v)} i =[{circumflex over (v)} i1 {circumflex over (v)} i2 {circumflex over (v)} i3 ] denotes the predicted three color components of the i-th pixel of the first image,
s i =[s i1 s i2 s i3 ] denotes the three color components of the i-th pixel of the second image,
{tilde over (M)} (1) is a 3×3 matrix and n is a 1×3 vector according to
M
~
(
1
)
=
[
m
11
(
1
)
m
12
(
1
)
m
13
(
1
)
m
21
(
1
)
m
22
(
1
)
m
23
(
1
)
m
31
(
1
)
m
32
(
1
)
m
33
(
1
)
]
,
and
n
=
[
n
11
n
12
n
13
]
,
sc
i
=
[
s
i
1
·
s
i
2
s
i
1
·
s
i
3
s
i
2
·
s
i
3
s
i
1
·
s
i
2
·
s
i
3
]
,
and
C
~
(
1
)
=
[
mc
11
(
1
)
mc
12
(
1
)
mc
13
(
1
)
mc
21
(
1
)
mc
22
(
1
)
mc
23
(
1
)
mc
31
(
1
)
mc
32
(
1
)
mc
33
(
1
)
mc
41
(
1
)
mc
42
(
1
)
mc
43
(
1
)
]
,
wherein the prediction parameters of said first order multi-channel, multiple regression prediction model are numerically obtained by minimizing the mean square error between the first image and the output image.
2. The method of claim 1 wherein the first image comprises a VDR image and the second image comprises an SDR image.
3. The method of claim 1 wherein the selected MMR prediction model is at least one of: a first order MMR model, a third order MMR model, a second order with cross-product MMR model, or a third order with cross-product MMR model.
4. The method of claim 3 wherein any of the MMR models further comprises prediction parameters related to neighbor pixels of the i-th pixel.
5. The method of claim 4 wherein the neighbor pixels comprise the left neighbor pixel, the right neighbor pixel, the top neighbor pixel, and the bottom neighbor pixel of the i-th pixel.
6. The method of claim 2 , wherein pixels in the VDR image have more color components than the pixels in the SDR image.
7. The method of claim 1 wherein selecting the MMR prediction model from the variety of MMR prediction models further comprises an iterative selection process comprising:
(a) selecting and applying an initial MMR prediction model;
(b) computing a residual error between the first image and the output image;
(c) selecting the initial MMR model if the residual error is smaller than an error threshold and no further MMR prediction model is selectable; otherwise,
selecting a new MMR prediction model from the variety of MMR prediction models, the new MMR prediction model being different from the previously selected MMR prediction model; and returning to step (b).
8. An image decoding method comprising:
receiving a first image having a first dynamic range;
receiving metadata, wherein said metadata include
a multiple-regression (MMR) prediction model adapted to approximate a second image having a second dynamic range in terms of
the first image, and
prediction parameters of the MMR prediction model, by applying inter-color image prediction, the metadata further including previously determined values of the prediction parameters; and
applying the first image and the previously determined values of the prediction parameters to the MMR prediction model to compute an output image for approximating the second image, wherein the second dynamic range is different from the first dynamic range, wherein the MMR prediction model is a first order multi-channel, multiple regression prediction model incorporating cross-multiplications between the color components of each pixel according to the formula
{circumflex over (v)} i =sc i {tilde over (C)} (1) +s i {tilde over (M)} (1) +n
wherein
{circumflex over (v)} i =[{circumflex over (v)} i1 {circumflex over (v)} i2 {circumflex over (v)} i3 ] denotes the predicted three color components of the i-th pixel of the first image,
s i =[s i1 s i2 s i3 ] denotes the three color components of the i-th pixel of the second image,
{tilde over (M)} (1) is a 3×3 matrix and n is a 1×3 vector according to
M
~
(
1
)
=
[
m
11
(
1
)
m
12
(
1
)
m
13
(
1
)
m
21
(
1
)
m
22
(
1
)
m
23
(
1
)
m
31
(
1
)
m
32
(
1
)
m
33
(
1
)
]
,
and
n
=
[
n
11
n
12
n
13
]
,
sc
i
=
[
s
i
1
·
s
i
2
s
i
1
·
s
i
3
s
i
2
·
s
i
3
s
i
1
·
s
i
2
·
s
i
3
]
,
and
C
~
(
1
)
=
[
mc
11
(
1
)
mc
12
(
1
)
mc
13
(
1
)
mc
21
(
1
)
mc
22
(
1
)
mc
23
(
1
)
mc
31
(
1
)
mc
32
(
1
)
mc
33
(
1
)
mc
41
(
1
)
mc
42
(
1
)
mc
43
(
1
)
]
.
9. The method of claim 8 wherein said first order MMR prediction model is expanded to a second or third order MMR prediction model with cross-pixel multiplications.
10. The method of claim 8 or 9 wherein the MMR prediction model further comprises prediction parameters related to neighbor pixels of the i-th pixel.
11. The method of claim 8 wherein the first image comprises an SDR image and the second image comprises a VDR image.
12. A method comprising:
providing a variety of multi-channel, multiple-regression (MMR) prediction models, each MMR prediction model adapted to approximate an image having a first dynamic range in terms of
an image having a second dynamic range, and
prediction parameters of the respective MMR prediction model, by applying inter-color image prediction;
receiving a first image and a second image, wherein the second image has a different dynamic range than the first image;
selecting a multi-channel, multiple-regression (MMR) prediction model from the variety of MMR models;
determining values of the prediction parameters of the selected MMR model;
computing an output image approximating the first image based on the second image and the determined values of the prediction parameters applied to the selected MMR prediction model;
outputting the determined values of the prediction parameters and the computed output image, wherein the variety of MMR models includes a second order multi-channel, multiple regression prediction according to the formula
{circumflex over (v)} i =s i 2 {tilde over (M)} (2) +s i {tilde over (M)} (1) +n
wherein
{circumflex over (v)} i =[{circumflex over (v)} i1 {circumflex over (v)} i2 {circumflex over (v)} i3 ] denotes the predicted three color components of the i-th pixel of the first image,
s i =[s i1 s i2 s i3 ] denotes the three color components of the i-th pixel of the second image,
{tilde over (M)} (1) and {tilde over (M)} (2) is a 3×3 matrix and n is a 1×3 vector according to:
M
~
(
1
)
=
[
m
11
(
1
)
m
12
(
1
)
m
13
(
1
)
m
21
(
1
)
m
22
(
1
)
m
23
(
1
)
m
31
(
1
)
m
32
(
1
)
m
33
(
1
)
]
,
n
=
[
n
11
n
12
n
13
]
,
M
~
(
2
)
=
[
m
11
(
2
)
m
12
(
2
)
m
13
(
2
)
m
21
(
2
)
m
22
(
2
)
m
23
(
2
)
m
31
(
2
)
m
32
(
2
)
m
33
(
2
)
]
,
and
s
i
2
=
[
s
i
1
2
s
i
2
2
s
i
3
2
]
,
wherein the prediction parameters of said second order multi-channel, multiple regression prediction model are numerically obtained by minimizing the mean square error between the first image and the output image.
13. The method of claim 12 wherein any of the MMR models further comprises prediction parameters related to neighbor pixels of the i-th pixel.
14. The method of claim 13 wherein the neighbor pixels comprise the left neighbor pixel, the right neighbor pixel, the top neighbor pixel, and the bottom neighbor pixel of the i-th pixel.
15. The method of claim 12 wherein selecting the MMR prediction model from the variety of MMR prediction models further comprises an iterative selection process comprising:
(a) selecting and applying an initial MMR prediction model;
(b) computing a residual error between the first image and the output image;
(c) selecting the initial MMR model if the residual error is smaller than an error threshold and no further MMR prediction model is selectable; otherwise,
selecting a new MMR prediction model from the variety of MMR prediction models, the new MMR prediction model being different from the previously selected MMR prediction model; and returning to step (b).
16. An image decoding method comprising:
receiving a first image having a first dynamic range;
receiving metadata, wherein said metadata include
a multiple-regression (MMR) prediction model adapted to approximate a second image having a second dynamic range in terms of
the first image, and
prediction parameters of the MMR prediction model, by applying inter-color image prediction, the metadata further including previously determined values of the prediction parameters; and
applying the first image and the previously determined values of the prediction parameters to the MMR prediction model to compute an output image for approximating the second image, wherein the second dynamic range is different from the first dynamic range, wherein the MMR prediction model is a second order multi-channel, multiple regression prediction according to the formula
{circumflex over (v)} i =s i 2 {tilde over (M)} (2) +s i {tilde over (M)} (1) +n
wherein
{circumflex over (v)} i =[{circumflex over (v)} i1 {circumflex over (v)} i2 {circumflex over (v)} i3 ] denotes the predicted three color components of the i-th pixel of the second image,
s i =[s i1 s i2 s i3 ] denotes the three color components of the i-th pixel of the first image,
{tilde over (M)} (1) and {tilde over (M)} (2) is a 3×3 matrix and n is a 1×3 vector according to:
M
~
(
1
)
=
[
m
11
(
1
)
m
12
(
1
)
m
13
(
1
)
m
21
(
1
)
m
22
(
1
)
m
23
(
1
)
m
31
(
1
)
m
32
(
1
)
m
33
(
1
)
]
,
n
=
[
n
11
n
12
n
13
]
,
M
~
(
2
)
=
[
m
11
(
2
)
m
12
(
2
)
m
13
(
2
)
m
21
(
2
)
m
22
(
2
)
m
23
(
2
)
m
31
(
2
)
m
32
(
2
)
m
33
(
2
)
]
,
and
s
i
2
=
[
s
i
1
2
s
i
2
2
s
i
3
2
]
,
wherein the prediction parameters of said second order multi-channel, multiple regression prediction model are numerically obtained by minimizing the mean square error between the first image and the output image.
17. The method of claim 16 wherein said second order MMR prediction model is expanded to a second or third order MMR prediction model with cross-pixel multiplications.
18. The method of claim 16 or 17 wherein the MMR prediction model further comprises prediction parameters related to neighbor pixels of the i-th pixel.
19. An apparatus comprising a processor and configured to perform any one of the methods recited in claim 1 , 8 , 12 , or 16 .
20. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for executing a method using at least a processor in accordance with any one of claim 1 , 8 , 12 , or 16 .