Shape from motion for unknown, arbitrary lighting and reflectance

1. A method for determining three dimensional (3D) shape, comprising:

capturing with a camera a plurality of images of an object in motion;

deriving a general relation that relates spatial and temporal image derivatives to bidirectional reflectance distribution function (BRDF) derivatives;

with isotropic BRDF ρ, determining image intensity I of a 3D point x, imaged at pixel u, as

I ( u,t )=σ( x )ρ( n,x ),

where σ is the albedo and n is the surface normal at the point and cosine fall-off is absorbed within ρ and relating I to depth and gradient;

exploiting rank deficiency to eliminate BRDF terms and recover depth or normal for directional lighting; and

using depth-normal-BRDF relation to recover depth or normal for unknown arbitrary lightings; and

determining the 3D shape from the motion.

2. The method of claim 1 , comprising applying rank deficiency to estimate depth for lighting colocated with the camera.

3. The method of claim 2 , comprising:

using two or more differential pairs of images to eliminate BRDF terms and derive a homogeneous quasilinear PDE in surface depth;

solving the PDE recover level curves of a surface; and

interpolating the level curves to recover dense depth.

4. The method of claim 2 , comprising:

using three or more differential pairs of images to eliminate BRDF terms to extract a first equation that directly yields the surface depth and a second equation that represents a homogeneous quasilinear PDE in surface depth.

5. The method of claim 1 , comprising applying rank deficiency in to estimate depth for an unknown directional point light source.

6. The method of claim 5 , comprising

using three or more differential pairs of images to eliminate BRDF terms and derive a homogeneous quasilinear PDE in surface depth;

solving the PDE recover level curves of a surface; and

interpolating the level curves to recover dense depth.

7. The method of claim 5 , comprising using 4 or more differential pairs of images, we may eliminate BRDF terms to extract two equations. The first directly yields the surface depth, while the second is an inhomogeneous quasilinear PDE in surface depth.

8. The method of claim 1 , comprising detecting an area light source with a diffuse BRDF as a quadratic function of the surface normal, wherein a differential stereo relation is a nonlinear PDE in surface depth to be solved using a nonlinear optimization method.

9. The method of claim 1 , comprising receiving additional depth data with surface normal information from a depth sensor to enhance accuracy.

10. The method of claim 1 , comprising applying rank deficiency in to estimate depth for an unknown arbitrary point light source.

11. The method of claim 10 , comprising

using three or more differential pairs of images to eliminate BRDF terms to derive an inhomogeneous quasilinear PDE in surface depth;

solving the PDE to recover level curves of the surface using a method of characteristics; and

interpolating the level curves to recover dense depth.

12. The method of claim 10 , comprising using four or more differential pairs of images to eliminate BRDF terms to determine a surface depth constraint and a linear constraint on the surface gradient, and combining the constraints to yield a sparse linear system to recover a surface depth.

13. The method of claim 1 , comprising modeling a dependence of surface reflectance on surface normal, lighting and viewing directions.

14. The method of claim 1 , comprising detecting a rank deficiency in differential stereo relations and applying the rank deficiency to eliminate BRDF and lighting dependence.

15. The method of claim 1 , comprising determining a relationship in a depth-normal-BRDF relationship to eliminate BRDF and dependence on arbitrary unknown lighting.

16. The method of claim 1 , comprising eliminating BRDF and a lighting to handle objects that reflects without calibrating the lighting.

17. The method of claim 1 , comprising deriving a BRDF-invariant expressions as quasilinear PDEs, and solving the PDEs.

18. The method of claim 1 , comprising deriving linear constraints on depth and gradient and solving the linear constraints as a sparse linear system to yield surface depth with unknown BRDF and arbitrary unknown illumination.

19. The method of claim 1 , for depth from collocated lighting with an orthographic camera, comprising

recovering level curves; and

interpreting the level curves to recover dense depth.

20. The method of claim 1 , for depth from collocated lighting with an orthographic camera, comprising:

discretizing a quasilinear PDE; and

determining depth as a non-linear optimization problem.