Method and a device for encoding a signal representative of a light-field content
The present disclosure relates to the transmission of sets of data and metadata and more particularly to the transmission of light-field contents. Light-field data take up large amounts of storage space which makes storage cumbersome and processing less efficient. In addition, light-field acquisition devices are extremely heterogeneous and each camera has its own proprietary file format. Since acquired light-field data from different cameras have a diversity of formats a complex processing is induced on the receiver side. To this end, it is proposed a method for encoding a signal representative of a light-field content in which the parameters representing the rays of light sensed by the different pixels of the sensor are mapped on the sensor. A second set of encoded parameters are used to reconstruct the light-field content from the parameters representing the rays of light sensed by the different pixels of the sensor.
1. A method implemented in a device having a processor and memory, the method comprising:
determining a plurality of light rays, wherein each light ray of the plurality of light rays has a trajectory in three-dimensional space and associated color information;
for each light ray, determining a first set of parameters representing the trajectory of the light ray and a second set of parameters representing the color information;
arranging the determined first sets of parameters into a first plurality of images;
compressing the first plurality of images using one of a first image codec and a first video codec;
arranging the determined second sets of parameters into a second plurality of images; and
compressing the second plurality of images using one of a second image codec and a second video codec.
2. The method according to claim 1 , wherein:
each of the determined first sets of parameters represents intersections points of the corresponding light ray with two reference planes in three-dimensional space; or
each of the determined first sets of parameters represents an intersection point of the corresponding light ray with a reference plane in three-dimensional space and a vector indicating an orientation of the corresponding light ray in three-dimensional space.
3. The method according to claim 1 , wherein:
each of the determined first sets of parameters comprises a plurality of parameters;
arranging the determined first sets of parameters into a first plurality of images comprises forming the first plurality of images using the plurality of parameters of each of the determined first sets plurality of parameters; and
a first image of the first plurality of images comprises a first parameter of the plurality of parameters for each light ray of the plurality of light rays.
4. The method according to claim 1 , wherein:
the determined first sets of parameters comprise respective first parameters, second parameters, third parameters, and fourth parameters; and
arranging the determined first sets of parameters into the first plurality of images comprises:
arranging the first parameters into a first image of the first plurality of images;
arranging the second parameters into a second image of the plurality of images;
arranging the third parameters into a third image of the plurality of images; and
arranging the fourth parameters into a fourth image of the plurality of images.
5. The computer implemented method according to claim 1 , wherein at least one of:
the one of a first image codec and a first video codec comprises one of a standard image codec and a standard video codec, including any of a joint photographic experts group (JPEG) image codec, a JPEG2000 image codec, an advanced video coding codec, a high efficiency video coding codec, an H.265 video codec, and H.266 video codec;
the one of a second image codec and a second video codec comprises one of a standard image codec and a standard video codec, including any of a JPEG image codec, a JPEG2000 image codec, an advanced video coding codec, a high efficiency video coding codec, an H.265 video codec, and an H.266 video codec; and
the one of the first image codec and the first video codec and the one of the second image codec and the second video codec are the same.
6. The method of claim 1 , wherein the compressing the first plurality of images comprises:
combining the first plurality of images into a composite image; and
coding the composite image using the one of the first image codec and the first video codec.
7. The method of claim 1 , wherein:
each light ray of the plurality of light rays is determined based on light sensed by one or more pixels of at least a sensor of an optical acquisition device; and
the sensor is a physical sensor device, the optical acquisition device is a physical optical acquisition device, and the plurality of light rays are determined from light entering the physical optical acquisition device; or
the sensor is a simulated sensor device, the optical acquisition device is a simulated optical acquisition device, and the plurality of light rays are determined from a computer graphics image model; or
the sensor is a combination of a physical sensor device and a simulated sensor device, the optical acquisition device is a combination of a physical optical acquisition device and a simulated optical acquisition device, and the plurality of light rays are determined from a combination of from light entering the physical optical acquisition device and a computer graphics image (CGI) model.
8. The method of claim 1 , comprising coding at least one of:
a supplemental enhancement information message comprising parameters necessary for reconstructing the light rays from the decoded images; and
an information message comprising parameters to specify a position for one or more reference planes used to generate the parameters representing the trajectories of the light rays.
9. The method of claim 1 , wherein at least one of:
each light ray of the plurality of light rays is determined based on one or more computer graphics images; and
the computer graphics images are at least partially simulated by a computer for a given scene description.
10. A device comprising circuitry, including a processor and memory, configured to:
determine a plurality of light rays, wherein each light ray of the plurality of light rays has a trajectory in three-dimensional space outside the optical acquisition system and associated color information;
for each light ray, determine a first set of parameters representing the trajectory of the light ray and a second set of parameters representing the color information;
arrange the determined first sets of parameters into a first plurality of images;
compress the first plurality of images using one of a first image codec and a first video codec;
arrange the determined second sets of parameters into a second plurality of images; and
compress the second plurality of images using one of a second image codec and a second video codec.
11. The device of claim 10 , wherein at least one of:
the one of a first image codec and a first video codec comprises one of a standard image codec and a standard video codec, including any of a joint photographic experts group (JPEG) image codec, a JPEG2000 image codec, an advanced video coding codec, a high efficiency video coding codec, an H.265 video codec, and an H.266 video codec;
the one of a second image codec and a second video codec comprises one of a standard image codec and a standard video codec, including any of a JPEG image codec, a JPEG2000 image codec, an advanced video coding codec, a high efficiency video coding codec, an H.265 video codec, and an H.266 video codec; and
the one of the first image codec and the first video codec and the one of the second image codec and the second video codec are the same.
12. The device of claim 10 , wherein the device being configured to compress the first plurality of images comprises the device being configured to:
combine the first plurality of images into a composite image; and
code the composite image using the one of the first image codec and the first video codec.
13. The device of claim 10 , wherein at least one of:
each light ray of the plurality of light rays is determined based on one or more computer graphics images; and
the computer graphics images are at least partially simulated by a computer for a given scene description.
14. The device of claim 10 , wherein the device is configured to code at least one of:
a supplemental enhancement information message comprising parameters for reconstructing the light rays from the decoded images, and
an information message comprising parameters to specify a position for one or more reference planes used to generate the parameters representing the trajectories of the light rays.
15. The device of claim 10 , wherein:
each of the determined first sets of parameters represents intersections points of the corresponding light ray with two reference planes in a three-dimensional space; or
each of the determined first set of parameters represents an intersection of the corresponding light ray with a reference plane in three-dimensional space and a vector indicating an orientation of the light ray in three-dimensional space.
16. The device of claim 10 , wherein:
each of the determined first sets of parameters comprises a plurality of parameters;
the device being configured to arrange the determined first sets of parameters into a first plurality of images comprises the device being configured to form the first plurality of images using the plurality of parameters of each of the determined first sets plurality of parameters; and
a first image of the first plurality of images comprises a first parameter of the plurality of parameters for each light ray of the plurality of light rays.
17. The device of claim 10 , wherein:
the determined first sets of parameters comprise respective first parameters, second parameters, third parameters, and fourth parameters; and
the device being configured to arrange the determined first sets of parameters into a first plurality of images comprises the device being configured to:
arrange the first parameters into a first image of the plurality of images;
arrange the second parameters into a second image of the plurality of images;
arrange the third parameters into a third image of the plurality of images; and
arrange the fourth parameters into a fourth image of the plurality of images.
18. The device of claim 10 , wherein:
each light ray of the plurality of light rays is determined based on light sensed by one or more pixels of at least a sensor of an optical acquisition device; and
the sensor is a physical sensor device, the optical acquisition device is a physical optical acquisition device, and the plurality of light rays are determined from light entering the physical optical acquisition device; or
the sensor is a simulated sensor device, the optical acquisition device is a simulated optical acquisition device, and the plurality of light rays are determined from a computer graphics image (CGI) model; or
the sensor is a combination of a physical sensor device and a simulated sensor device, the optical acquisition device is a combination of a physical optical acquisition device and a simulated optical acquisition device, and the plurality of light rays are determined from a combination of from light entering the physical optical acquisition device and a computer graphics image (CGI) model.
19. The device according to claim 10 , wherein each light ray of the plurality of light rays is determined based on light sensed by one or more pixels of at least a sensor of an optical acquisition device, and wherein each of the trajectories is a trajectory in three-dimensional space outside the optical acquisition device.
20. The method according to claim 1 , wherein each light ray of the plurality of light rays is determined based on light sensed by one or more pixels of at least a sensor of an optical acquisition device, and wherein each of the trajectories is a trajectory in three-dimensional space outside the optical acquisition device.