Method of extracting an object on a projected backdrop
A method of extracting from a picked-up image an object that is situated in the foreground of a projected backdrop. The method includes an extraction step comprising the steps of establishing a correspondence relationship between pixels of the projected backdrop and of the background of the picked-up image, and defining said object as the set of picked-up pixels that present a departure from said correspondence relationship. The method is applicable to video conferences, to remote teaching, and to television shows.
1. A method of extracting from a picked-up image an object that is situated in the foreground of a projected backdrop, the projected backdrop being formed by the projection of a known backdrop image I(s), s representing a pixel, wherein the method includes an extraction step comprising the steps of:
establishing a correspondence relationship between pixels of the known backdrop image I(s) and of the background of the image as picked up, said correspondence relationship being described by a geometrical transformation function H and by a light transfer function T, said geometrical transformation function H having an initial value previously estimated by putting particular pixels into correspondence between the known backdrop image I(s) and the image as picked up;
defining said object as the set of picked-up pixels that present a departure from said correspondence relationship; and
iteratively updating at least the geometrical transformation function H to establish an updated correspondence relationship between the pixels of the known backdrop image I(s) and of the background of the image as picked up,
wherein the geometrical transformation function H and the light transfer function T are mathematically estimated by iteratively evaluating a function which includes the geometrical transformation function H, the light transfer function T, the known backdrop image I(s), and the image E as picked up, starting with the light transfer function T having a determined initial value and the geometrical transformation function H having the initial value previously estimated.
2. The method according to claim 1 , wherein said correspondence relationship is described by said geometrical transformation function H and said light transfer function T as follows:
T ( I ( s ))= E ( Hs )
expressing equality between the brightness T(I(s)) at pixel s in the known backdrop image I(s), corrected by the light transfer function T, and the brightness E(Hs) observed at pixel Hs in the image E as picked up, where the pixel Hs is the image of the pixel s after transformation by the geometrical transformation function H.
3. The method according to claim 2 , wherein said departure from the correspondence relationship is represented by an observed noise level ε s at each pixel s:
ε s =T ( I ( s ))− E ( Hs ).
4. The method according to claim 1 , further comprising estimating the geometrical transformation function H and the light transfer function T by minimizing the function F(H,T):
F ( H,T )=Σ s ρ( T ( I ( s ))− E ( Hs )
expressing the accumulated error over the set of pixels s between the brightness T(I(s)) at pixel s of the known backdrop image I(s), corrected by a light transfer function T, and the brightness E(Hs) at pixel Hs of the image E as picked up, where pixel Hs is the image of the pixel s as transformed by the geometrical transformation function H, and where ρ is an error cost function.
5. The method according to claim 1 , wherein said method includes an initialization step comprising:
estimating the geometrical transformation function H from the correspondence between particular pixels of the known backdrop image I(s) and of the image as picked up; and
initializing the light transfer function T with an identity function.
6. The method according to claim 1 , wherein said method includes a post-treatment step comprising regularizing the extracted object.
7. The method according to claim 1 , wherein the extraction step includes a prior step of adjusting the light intensity channels of the backdrop.
8. The method according to claim 1 , wherein the iterative updating of at least the geometrical transformation function H is performed at least once per picked-up image.
9. The method according to claim 1 , wherein the iterative updating of at least the geometrical transformation function H is performed at a frequency greater than 25 Hz.
10. The method according to claim 1 , further comprising iteratively updating the light transfer function T.
11. The method according to claim 1 , wherein the geometrical transformation function H and the light transfer function T are mathematically estimated by iteratively minimizing a function F(H,T) expressing the accumulated error over the set of pixels s between the brightness T(I(s)) at pixel s of the known backdrop image I(s), corrected by the light transfer function T, and the brightness E(Hs) at pixel Hs of the image E as picked up, where pixel Hs is the image of the pixel s as transformed by the geometrical transformation function H.
12. A non-transitory computer-readable medium storing a computer program including program code instructions for implementing the steps of the method according to claim 1 , when said program is executed on a computer.
13. A method of transmitting an image for reproducing on a remote station an image comprising an object situated in the foreground of a projected backdrop, the method implementing an object extraction method according to claim 1 , followed by a step of transmitting the object to the remote station.
14. A method of forming an image on a remote station from an object transmitted by the method according to claim 13 , the image formation method including a step of superposing the object on the backdrop made available on said remote station.
15. A module, comprising a microprocessor configured to control the following means for operations, for extracting from an image as picked up an object situated in the foreground of a projected backdrop, the projected backdrop being formed by the projection of a known backdrop image I(s), s representing a pixel, wherein the module comprises:
means for establishing a correspondence relationship between pixels of the known backdrop image I(s) and of the background of the image as picked up, said correspondence relationship being described by a geometrical transformation function H and by a light transfer function T, said geometrical transformation function H having an initial value previously estimated by putting particular pixels into correspondence between the known backdrop image I(s) and the image as picked up;
means for defining said object as the set of picked-up pixels that present a departure from said correspondence relationship; and
means for iteratively updating at least the geometrical transformation function H to establish an updated correspondence relationship between the pixels of the known backdrop image I(s) and of the background of the image as picked up,
wherein the geometrical transformation function H and the light transfer function T are mathematically estimated by iteratively evaluating a function which includes the geometrical transformation function H, the light transfer function T, the known backdrop image I(s), and the image E as picked up, starting with the light transfer function T having a determined initial value and the geometrical transformation function H having the initial value previously estimated.
16. The module according to claim 15 , wherein said module includes means for computing said geometrical transformation function H and said light transfer function T by minimizing the function F(H,T):
F ( H,T )=Σ s ρ( T ( I ( s ))− E ( Hs )
expressing the accumulated error over the set of pixels s between the brightness T(I(s)) at pixel s of the known backdrop image I(s), corrected by a light transfer function T, and the brightness E(Hs) at pixel Hs of the image E as picked up, where pixel Hs is the image of the pixel s as transformed by the geometrical transformation function H, and where ρ is an error cost function.
17. The module according to claim 15 wherein said module includes post-treatment means suitable for regularizing the extracted object.
18. The module according to claim 15 wherein, said module includes channel adjustment means for adjusting the channels derived from the brightness I of the backdrop.
19. The module according to claim 15 , wherein the geometrical transformation function H and the light transfer function T are mathematically estimated by iteratively minimizing a function F(H,T) expressing the accumulated error over the set of pixels s between the brightness T(I(s)) at pixel s of the known backdrop image I(s), corrected by the light transfer function T, and the brightness E(Hs) at pixel Hs of the image E as picked up, where pixel Hs is the image of the pixel s as transformed by the geometrical transformation function H.