Photometric three-dimensional facial capture and relighting
Embodiments disclosed herein relate to a method and apparatus for generating a three-dimensional surface. In one embodiment, there is a method for generating a three-dimensional surface. The method includes capturing a plurality of images of a target object with at least two cameras, the target object illuminated by at least two sets of red-green-blue (RGB) lights positioned in an array about the target object and generating a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in n-frame intervals.
1. A method for generating a three-dimensional surface, comprising:
capturing a plurality of images of a target object with at least two cameras, the target object illuminated by at least two sets of red-green-blue (RGB) lights positioned in an array about the target object; and
generating a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in n-frame intervals.
2. The method of claim 1 , wherein at least one camera captures the plurality of images of the target object from a first side and at least one camera captures the plurality of images of the target object from a second side.
3. The method of claim 1 , wherein aligning images of the target object using motion estimation comprises:
recovering a backward motion and forward motion within each respective n-frame interval.
4. The method of claim 1 , wherein each frame in the n-frames recovered for the three-dimensional surface is taken from a right side and a left side of the target object.
5. The method of claim 1 , wherein reconstructing the surface estimate comprises:
updating depth maps of the images processed in n-frame intervals.
6. The method of claim 1 , wherein reconstructing the surface estimate of the target object is defined in terms of surface energies of the target object.
7. The method of claim 1 , wherein aligning images of the target object using motion estimation is defined in terms of flow energies of the target object.
8. A method for generating a three-dimensional surface, comprising:
capturing an image of a target object with two cameras, one camera positioned to a left of the target object, one camera position to a right of the target object, wherein the target object is illuminated by three sets of red-green-blue (RGB) lights positioned in an array about the target object; and
processing the image in three frame intervals, wherein processing the image comprises:
recovering an unknown surface at each frame within a three-frame interval; and
recovering a backward motion and forward motion within the three-frame interval.
9. The method of claim 8 , wherein each frame in the three frames recovered for the three-dimensional surface is taken from a right side and a left side of the target object.
10. The method of claim 8 , wherein the image is iteratively processed three frames at a time until images in the three frames converge to a single image of the target object.
11. The method of claim 8 , wherein recovering the unknown surface at each frame within the three-frame interval is defined in terms of surface energies of the target object.
12. The method of claim 8 , wherein recovering the unknown surface at each frame within the three-frame interval comprises:
updating depth maps of the images processed in three frame intervals.
13. The method of claim 12 , where the depth maps are updated between frames generated by the camera positioned to the left of the target object and the camera positioned to the right of the target object.
14. The method of claim 8 , wherein recovering the backward motion and forward motion within the three-frame interval is defined in terms of flow energies of the target object.
15. A system for reconstructing a three-dimensional surface, comprising:
at least two cameras, wherein a first camera of the at least two cameras is positioned to capture a right image of a target object, and wherein a second camera is positioned to capture a left image of the target object;
at least three sets of red-green-blue (RGB) lights positioned in an array about the target object; and
a controller configured to generate a three-dimensional surface of the target object by iteratively reconstructing a surface estimate of the target object and aligning images of the target object using motion estimation until the images converge, wherein the images are processed in three-frame intervals.
16. The system of claim 15 , wherein the three sets of RGB lights are mounted on a circular configuration.
17. The system of claim 15 , wherein the cameras are positioned about 200 cm away from the three-dimensional surface.
18. The system of claim 15 , wherein the sets of RGB lights are positioned about 60 cm away from the cameras.
19. The system of claim 15 , wherein aligning images of the target object using motion estimation is defined in terms of flow energies of the target object.
20. The system of claim 15 , wherein generating the surface estimate of the target object is defined in terms of surface energies of the target object.