Systems and methods for capturing motion in three-dimensional space
View Patent ↗Methods and systems for capturing motion and/or determining the shapes and positions of one or more objects in 3D space utilize cross-sections thereof. In various embodiments, images of the cross-sections are captured using a camera based on edge points thereof.
1. A method of identifying a position and shape of at least a portion of an object (object portion) in three-dimensional (3D) space, the method comprising:
capturing, with a plurality of cameras, each camera having a particular vantage point, two or more images generated by casting an output from at least one source onto the object portion;
analyzing the two or more images captured by the cameras from the particular vantage points to computationally slice the object portion, as captured, into a plurality of two-dimensional (2D) slices, each slice corresponding to a cross-section of the object portion, based at least in part on a plurality of edge points of the object portion in the image; and
reconstructing the position and shape of at least a portion of the object portion in 3D space based at least in part on a plurality of identified cross-sectional positions and sizes.
2. The method of claim 1 , wherein the vantage points are on the same side of the object.
3. The method of claim 1 , wherein the cameras are arranged such that the vantage points lie in a single line.
4. The method of claim 1 , wherein the fields of view of the cameras overlap throughout an area where motion of interest is expected to occur.
5. The method of claim 1 , wherein optical axes of the cameras are parallel.
6. The method of claim 1 , wherein the two or more images comprise images captured as the object moves and articulates.
7. The method of claim 1 , wherein the two or more images are captured such that a time between images is sufficiently small so as to avoid quality degradation due to object movement.
8. The method of claim 1 , wherein the capturing comprises controlling camera settings of the cameras.
9. The method of claim 8 , wherein the camera settings are selected from a group including resolution, depth of field, sensitivity, light frequency and frame rate.
10. The method of claim 8 , wherein the controlling camera settings is conducted according to object parameters assumed to correspond to at least a portion of a type of object.
11. The method of claim 1 , wherein the at least one source comprises one or more light emitting diodes.
12. The method of claim 1 , wherein the at least one source comprises one or more light sources and one or more of the light sources are placed on either side of a camera.
13. The method of claim 1 , wherein the at least one source comprises one or more light sources and the one or more light sources are placed to create a contrast between at least a portion of the object and a background.
14. The method of claim 1 , wherein the at least one source comprises one or more light sources and the one or more light sources are placed to illuminate a region on the at least a portion of the object.
15. The method of claim 1 , wherein the at least one source comprises one or more light sources and at least a subset of the light sources is operable to emit at different wavelengths.
16. The method of claim 1 , wherein the at least one source comprises one or more light sources, at least a subset of the light sources is operable in sequence or simultaneously and one or more of the light sources is operable in a pulsed fashion.
17. The method of claim 1 , wherein the at least one source comprises one or more light sources and at least a subset of the light sources is operable at different brightnesses.
18. The method of claim 1 , wherein the at least one source comprises one or more light sources selected from a group including point light sources, arrays and bar lights.
19. The method of claim 1 , wherein the analyzing comprises applying a pixel threshold to determine portions of the image as corresponding to at least a portion of the object or background.
20. The method of claim 1 , wherein the edge points define a viewed portion of the cross-section such that the portion of the cross-section is within a field of view of at least one of the cameras from at least one of the particular vantage points.
21. The method of claim 20 , wherein the computationally slicing comprises fitting a closed curve to the viewable portion of the object portion, the closed curve defining a 3D model of the object portion.
22. The method of claim 21 , wherein the closed curve is an ellipse.
23. The method of claim 21 , wherein the fitting a closed curve comprises:
determining a family of closed curves corresponding to the viewable portion of the object portion;
selecting a closed curve from the family of closed curves; and
selecting a rotation angle for the closed curve that corresponds with a determined contour of the object portion.
24. The method of claim 21 , wherein the computationally slicing further comprises approximating a contour of the portion of the cross-section within the field of view of at least one of the cameras.
25. The method of claim 1 , wherein light rays cast from the edge points to the image-capturing device are tangent to the cross-section.
26. The method of claim 1 , wherein the at least one of the edge points is defined by a boundary between a shadow region and an illuminated region on the cross-section of the object.
27. The method of claim 1 , wherein the reconstructing comprises assembling the 2D slices to form a 3D model.
28. The method of claim 27 , further comprising using correlation to refine the 3D model.
29. The method of claim 28 , wherein the correlation comprises at least one of:
correlating among the 2D slices; and
correlating a 3D model resulting from stacking the 2D slices to preserve continuity with characteristic information of a corresponding object.
30. The method of claim 29 , wherein the characteristic information includes at least one of continuity in motion and continuity in deformation.
31. The method of claim 1 , wherein:
the casting an output comprises casting an output that causes an illuminated part of the object and a shadow region; and
the analyzing comprises detecting a shadow edge point of the shadow region and determining characteristic parameters of a closed curve corresponding to the shadow edge point.
32. The method of claim 1 , wherein the analyzing comprises determining closed curve characteristics as corresponding to an object type that corresponds to the object portion of the object.
33. The method of claim 32 , wherein the determining closed curve characteristics is conducted by comparing the object to object types in a library including object types.
34. The method of claim 1 , further comprising determining characteristics of an object model determined from the reconstructing; and, according to the characteristics of the object model, at least one of:
revising characteristic parameters in an object library and
defining characteristic parameters in an object library.
35. The method of claim 32 , wherein the closed curve characteristics are selected from a group including closed curve shapes, 3D closed curves and closed curve rotation angle.
36. The method of claim 32 , further comprising modeling at least a portion of the object with a 3D volume.
37. The method of claim 36 , wherein the 3D volume is selected from a group including ellipsoids, hyperboloids and cylinders.
38. The method of claim 36 , wherein the modeling at least a portion of the object comprises modeling at least one of a fingertip, a thumb and one or more finger segments with an ellipsoid.
39. The method of claim 1 , wherein the object portion is moving in 3D space; and the two or more images capture a motion of the object portion.
40. The method of claim 39 , further comprising interpreting the captured motion as user input.
41. The method of claim 40 , wherein the motion of the object portion comprises one or more hand gestures.
42. The method of claim 1 , further comprising:
comparing a reconstructed position and shape of the object portion with characteristic parameters for the object to determine one or more object poses; and
interpreting the one or more object poses as user input.
43. The method of claim 1 , further comprising:
determining that a reconstructed shape of the object portion matches an object model in an object library; and
interpreting the reconstructed shape as user input.
44. The method of claim 1 , wherein the object comprises a hand and the at least a portion of the object comprises at least a portion of a finger.
45. The method of claim 1 , wherein the cameras have overlapping fields of view.
46. The method of claim 1 , wherein the cameras are directed in an upward direction.
47. A system for identifying a position and shape of at least a portion of an object (object portion) in three-dimensional (3D) space, the system comprising:
a plurality of cameras, each camera having a particular vantage point, and oriented toward a field of view;
at least one source to direct illumination onto the object portion in the field of view; and
an image analyzer coupled to the cameras and the at least one source and configured to:
capture two or more images generated by casting an output from at least one source onto the object;
analyze the two or more images captured by the cameras from the particular vantage points to computationally slice the object portion, as captured, into a plurality of two-dimensional (2D) slices, each slice corresponding to a cross-section of the object portion, based at least in part on a plurality of edge points in the image; and
reconstruct the position and shape of at least a portion of the object portion in 3D space based at least in part on a plurality of the identified cross-sectional positions and sizes.
48. The system of claim 47 , wherein the cameras are arranged such that the vantage points lie in a single line.