Method for encoding and method for decoding a light field based image and corresponding devices
A method for encoding a plenoptic image divided into blocks is disclosed. The method comprises: —encoding (S 120 ) a block in a central region of the plenoptic image, called central block, without spatial prediction; and —predictively encoding (S 130 ) blocks immediately surrounding said central block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction.
1. A method for encoding a 4D light field image divided into blocks comprising:
encoding a first block located in a geometrically central region of the 4D light field image compared to other blocks of the 4D light field image without spatial prediction with a reference block in said 4D light field image; and
predictively encoding blocks immediately surrounding said first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction with a reference block in said 4D light field image, wherein the reference block is either the first block itself or a block in said 4D light field image that has been encoded with the first block as a reference block, and wherein
an even number of blocks is comprised in said 4D light field image, and
predictively encoding blocks immediately surrounding the first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction comprises encoding one immediately surrounding block neighboring vertically the first block and one immediately surrounding block neighboring horizontally the first block by spatial unidirectional prediction; encoding the immediately surrounding block neighboring the two immediately surrounding blocks previously encoded by bi-directional prediction; encoding, by unidirectional prediction, the immediately surrounding blocks neighboring vertically or horizontally one of the previously encoded blocks and encoding the remaining immediately surrounding block by bi-directional prediction.
2. The method according to claim 1 , further comprising predictively encoding remaining blocks by processing blocks from the blocks immediately surrounding the geometric center of the 4D light field image up to the borders of the 4D light field image by spatially interleaving spatial unidirectional and spatial bi-directional predictions, wherein the spatial interleaving comprises:
P k and B k respectively denote the sets of blocks to be encoded during first and second sub-steps of step k, and
S k ={I 0 }∪{U l≤k (P l ∪B l )} the set of all blocks encoded up to step k, and
P 0 and B 0 are empty sets, and
the blocks to be encoded during a step k, k≥1, are:
P k for which a horizontal neighbor or a vertical neighbor has just been encoded during step (k−1) are encoded in a first sub-step using unidirectional prediction horizontally and vertically:
P
k
=
{
C
[
i
,
j
]
∉
S
k
-
1
:
∃
C
[
y
,
x
]
∈
S
k
-
1
,
(
(
y
=
i
±
1
)
and
(
x
=
j
)
)
or
(
(
y
=
i
)
and
(
x
=
j
±
1
)
)
}
and
B k located at the corners of the rectangle started at previous sub-step are encoded in a second sub-step using bi-directional prediction:
B
k
=
{
C
[
i
,
j
]
∉
S
k
-
1
:
∃
C
[
y
,
x
]
∈
P
k
,
(
y
=
i
±
1
)
and
(
x
=
j
±
1
)
}
wherein the definition of P 1 differs depending on a parity of n such that if n is odd:
P
1
odd
=
{
C
[
i
,
j
]
∉
S
0
:
∃
C
[
y
,
x
]
∈
S
0
,
(
(
y
=
i
±
1
)
and
(
x
=
j
)
)
or
(
(
y
=
i
)
and
(
x
=
j
±
1
)
)
}
and, if n is even:
P
1
even
=
{
C
[
i
,
j
]
∉
S
0
:
∃
C
[
y
,
x
]
∈
S
0
,
(
(
y
=
i
+
1
)
and
(
x
=
j
)
)
or
(
(
y
=
i
)
and
(
x
=
j
+
1
)
)
}
.
3. The method according to claim 1 , wherein either all of the blocks are a micro-image type or all of the blocks are a view type.
4. The method according to claim 1 , wherein the first block is intra coded using information contained only in the first block.
5. The method according to claim 1 , wherein the first block is temporally predicted from a spatially co-located block in a reconstructed 4D light field image.
6. The method according to claim 1 , wherein the blocks being views images, and during prediction of a block by a reference block, said reference block is weighted by a weight derived from vignetting information.
7. A method for decoding a 4D light field image divided into blocks comprising:
decoding a first block in a geometrically central region of the 4D light field image compared to other blocks of the 4D light field image without spatial prediction with a reference block in the 4D light field image; and
predictively decoding blocks immediately surrounding said block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction with a reference block in said 4D light field image, wherein the reference block is either the first block itself or a block in said 4D light field image that has been encoded with the first block as a reference block, and wherein
said 4D light field image comprises an even number of blocks, and
predictively decoding blocks immediately surrounding the first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction comprises decoding one immediately surrounding block neighboring vertically the first block and one immediately surrounding block neighboring horizontally the first block by spatial unidirectional prediction; decoding the immediately surrounding block neighboring the two immediately surrounding blocks previously decoded by bi-directional prediction; decoding, by unidirectional prediction, the immediately surrounding blocks neighboring vertically or horizontally one of the previously decoded blocks and decoding the remaining immediately surrounding block by bi-directional prediction.
8. The method according to claim 7 , further comprising predictively decoding remaining blocks by processing blocks from the blocks immediately surrounding the geometric center of the 4D light field image up to the borders of the 4D light field image by spatially interleaving spatial unidirectional and spatial bi-directional predictions
wherein spatial interleaving comprises:
P k and B k respectively denote the sets of blocks to be decoded during first and second sub-steps of step k, and
S k ={I 0 }∪{U l≤k (P l ∪B l )} the set of all blocks decoded up to step k, and
P 0 and B 0 are empty sets, and
the blocks to be decoded during a step k, k≥l, are:
P k for which a horizontal neighbor or a vertical neighbor has just been decoded during step (k−1) are decoded in a first sub-step using unidirectional prediction horizontally and vertically:
P
k
=
{
C
[
i
,
j
]
∉
S
k
-
1
:
∃
C
[
y
,
x
]
∈
S
k
-
1
,
(
(
y
=
i
±
1
)
and
(
x
=
j
)
)
or
(
(
y
=
i
)
and
(
x
=
j
±
1
)
)
}
and
B k located at the corners of the rectangle started at previous sub-step are decoded in a second sub-step using bi-directional prediction:
B
k
=
{
C
[
i
,
j
]
∉
S
k
-
1
:
∃
C
[
y
,
x
]
∈
P
k
,
(
y
=
i
±
1
)
and
(
x
=
j
±
1
)
}
wherein the definition of P 1 differs depending on a parity of n such that if n is odd:
P
1
odd
=
{
C
[
i
,
j
]
∉
S
0
:
∃
C
[
y
,
x
]
∈
S
0
,
(
(
y
=
i
±
1
)
and
(
x
=
j
)
)
or
(
(
y
=
i
)
and
(
x
=
j
±
1
)
)
}
and, if n is even:
P
1
even
=
{
C
[
i
,
j
]
∉
S
0
:
∃
C
[
y
,
x
]
∈
S
0
,
(
(
y
=
i
+
1
)
and
(
x
=
j
)
)
or
(
(
y
=
i
)
and
(
x
=
j
+
1
)
)
}
.
9. The method according to claim 7 , wherein either all of the blocks are a micro-image type or all of the blocks are a view type.
10. The method according to claim 7 , wherein the first block is intra coded using information contained only in the first block.
11. The method according to claim 7 , wherein the first block is temporally predicted from a spatially co-located block in a reconstructed 4D light field image.
12. The method according to claim 7 , wherein the blocks being views images, and during prediction of a block by a reference block, said reference block is weighted by a weight derived from vignetting information.
13. A device for encoding a 4D light field image divided into blocks, comprising a processor and at least one memory unit, the processor being configured to:
encode a first block located in a geometrically central region of the 4D light field image compared to other blocks in the 4D light field image without spatial prediction with a reference block in said 4D light field image; and
predictively encode blocks immediately surrounding said first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction with a reference block in said 4D light field image, wherein the reference block is either the central block itself or a block in said 4D light field image that has been encoded with the first block as a reference block, and wherein
an even number of blocks is comprised in said 4D light field image, and
predictively encoding blocks immediately surrounding the first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction comprises encoding one immediately surrounding block neighboring vertically the first block and one immediately surrounding block neighboring horizontally the first block by spatial unidirectional prediction; encoding the immediately surrounding block neighboring the two immediately surrounding blocks previously encoded by bi-directional prediction; encoding, by unidirectional prediction, the immediately surrounding blocks neighboring vertically or horizontally one of the previously encoded blocks and encoding the remaining immediately surrounding block by bi-directional prediction.
14. A device for decoding a 4D light field image divided into blocks, comprising a processor and at least one memory unit, the processor being configured to:
decode a first block in a geometrically central region of the 4D light field image compared to other blocks in the 4D light field image without spatial prediction with a reference block in said 4D light field image; and
predictively decode blocks immediately surrounding said first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction with a reference block in said 4D light field image, wherein the reference block is either the first block itself or a block in said 4D light field image that has been encoded with the first block as a reference block, and wherein
said 4D light field image comprises an even number of blocks, and
predictively decoding blocks immediately surrounding the first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction comprises decoding one immediately surrounding block neighboring vertically the first block and one immediately surrounding block neighboring horizontally the first block by spatial unidirectional prediction; decoding the immediately surrounding block neighboring the two immediately surrounding blocks previously decoded by bi-directional prediction; decoding, by unidirectional prediction, the immediately surrounding blocks neighboring vertically or horizontally one of the previously decoded blocks and decoding the remaining immediately surrounding block by bi-directional prediction.
15. The method of claim 1 , wherein in the case of an odd number of blocks is comprised in said 4D light field image, predictively encoding blocks immediately surrounding the first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction comprises encoding by spatial unidirectional prediction the immediately surrounding blocks neighboring vertically or horizontally the first block and encoding by spatial bi-directional prediction the other immediately surrounding blocks diagonally neighboring the first block.
16. The method of claim 7 , wherein in the case of said 4D light field image comprising an odd number of blocks, predictively decoding blocks immediately surrounding the first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction comprises decoding by spatial unidirectional prediction the immediately surrounding blocks neighboring vertically or horizontally the first block and decoding by spatial bi-directional prediction the other immediately surrounding blocks diagonally neighboring the first block.
17. The device of claim 13 , wherein the processor is further configured for, in the case of an odd number of blocks is comprised in said 4D light field image, predictively encoding blocks immediately surrounding the first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction comprises encoding by spatial unidirectional prediction the immediately surrounding blocks neighboring vertically or horizontally the first block and encoding by spatial bi-directional prediction the other immediately surrounding blocks diagonally neighboring the first block.
18. The device of claim 14 , wherein the processor is further configure for, in the case of said 4D light field image comprising an odd number of blocks, predictively decoding blocks immediately surrounding the first block by spatially interleaving spatial unidirectional prediction and spatial bi-directional prediction comprises decoding by spatial unidirectional prediction the immediately surrounding blocks neighboring vertically or horizontally the first block and decoding by spatial bi-directional prediction the other immediately surrounding blocks diagonally neighboring the first block.