Method and apparatus for processing virtual object lighting inserted into a 3D real scene
A method and an apparatus for processing a 3D scene are disclosed. A reference image representative of an image of the scene captured under ambient lighting is determined. A texture-free map is determined from said reference image and an input image of the scene. The 3D scene is then processed using the determined texture-free map.
1. A method for processing a 3D scene, comprising:
obtaining an input image of the scene captured by a first camera;
selecting a reference image of the 3D scene among a set of ambient images of the 3D scene captured from different viewpoints of a second camera under ambient lighting, according to a viewpoint corresponding to said input image, said reference image comprising a reference area representative of a reference diffuse surface captured under said ambient lighting;
determining a texture map from said reference image according to an ambient light intensity, the ambient light intensity being determined according to at least one part of said reference area;
determining a texture-free map, a value of a pixel of the texture-free image map being a ratio of an intensity at the corresponding pixel in said input image over a value of an intensity at the same pixel in the texture map; and
processing said 3D scene using said texture-free map.
2. The method according to claim 1 , wherein processing said 3D scene comprises determining lighting parameters of the scene.
3. The method according to claim 1 , further comprising scaling said texture-free map with a scaling factor.
4. The method according to claim 3 , wherein said scaling factor is selected such that a number of pixels with saturated intensity value in the scaled texture-free map is below a value.
5. The method according to claim 1 , further comprising registering said input image with respect to the viewpoint of the selected reference image and wherein determining said texture-free map uses said registered input image.
6. An apparatus configured to process a 3D scene, the apparatus comprising a processor and a memory having stored instructions operative, when executed by the processor, to cause the apparatus to perform:
obtaining an input image of the scene captured by a first camera,
selecting a reference image of the 3D scene among a set of ambient images of the 3D scene captured from different viewpoints of a second camera under ambient lighting, according to a viewpoint corresponding, to said input image, said reference image comprising a reference area representative of a reference diffuse surface captured under said ambient lighting,
determining a texture map from said reference image according to an ambient light intensity, the ambient light intensity being determined according to at least one part of said reference area,
determining a texture-free map, a value of a pixel of the texture-free image map being a ratio of an intensity at the corresponding pixel in said input image over a value of an intensity at the same pixel in the texture map, and
processing said 3D scene using said texture-free map.
7. The apparatus according to claim 6 , wherein processing said 3D scene comprises determining lighting parameters of the scene.
8. The apparatus according to claim 6 , wherein the processor is configured to perform scaling said texture-free map with a scaling factor.
9. The apparatus according to claim 8 , wherein said scaling factor is selected by said processor such that a number of pixels with saturated intensity value in the scaled texture-free map is below a value.
10. The apparatus according to claim 6 , wherein said processor is configured to perform registering said input image with respect to the viewpoint of the selected reference image and wherein determining said texture-free map uses said registered input image.
11. A non-transitory computer-readable storage medium having stored instructions that, when executed by a processor, cause the processor to perform:
obtaining an input image of the scene captured by a first camera,
selecting a reference image of the 3D scene among a set of ambient images of the 3D scene captured from different viewpoints of a second camera under ambient lighting,
according to a viewpoint corresponding to said input image, said reference image comprising a reference area representative of a reference diffuse surface captured under said ambient lighting;
determining a texture map from said reference image according to an ambient light intensity, the ambient light intensity being determined according to at least one part of said reference area;
determining a texture-free map, a value of a pixel of the texture-free map being a ratio of an intensity at the corresponding pixel in said input image over a value of an intensity at the same pixel in the texture map; and
processing said 3D scene using said texture-free map.
12. The non-transitory computer-readable storage medium according to claim 11 , wherein processing the 3D scene comprises determining lighting parameters of the scene.
13. The non-transitory computer-readable storage medium according to claim 11 , further comprising instructions that cause the processor to perform scaling the texture-free map with a scaling factor.
14. The non-transitory computer-readable storage medium according to claim 13 , wherein the scaling factor is selected such that a number of pixels with saturated intensity value in the scaled texture-free map is below a value.
15. The non-transitory computer-readable storage medium according to claim 11 , further comprising instructions that cause the processor to perform registering the input image with respect to the viewpoint of the selected reference image and wherein determining the texture-free map uses the registered input image.