Systems and methods for capturing motion in three-dimensional space
View Patent ↗Methods and systems for capturing motion and/or determining the shapes and positions of one or more objects in 3D space utilize cross-sections thereof. In various embodiments, images of the cross-sections are captured using a camera based on edge points thereof.
1. A method of identifying a position and shape of at least a portion of an object (object portion) in three-dimensional (3D) space, the method comprising:
capturing, with a single camera having a particular vantage point, one or more images generated by casting an output from at least one source onto the object portion;
analyzing the one or more images captured by the single camera from the particular vantage point to computationally slice the object portion, as captured, into a plurality of two-dimensional (2D) slices, each slice corresponding to a cross-section of the object portion, based at least in part on a plurality of edge points in the image; and
reconstructing the position and shape of at least a portion of the object portion in 3D space based at least in part on a plurality of identified cross-sectional positions and sizes,
wherein light rays cast from the edge points to the image capturing device are tangent to the cross-section.
2. A system for identifying a position and shape of at least a portion of an object (object portion) in three-dimensional (3D) space, the system comprising:
a single camera having a particular vantage point and oriented toward a field of view; at least one source to direct illumination onto the object portion in the field of view; and an image analyzer coupled to the camera and the at least one source and configured to:
capture one or more images generated by casting an output from at least one source onto the object;
analyze the one or more images captured by the single camera from the particular vantage point to computationally slice the object portion, as captured, into a plurality of two-dimensional (2D) slices, each slice corresponding to a cross-section of the object portion, based at least in part on a plurality of edge points in the image; and
reconstruct the position and shape of at least a portion of the object portion in 3D space based at least in part on a plurality of the identified cross-sectional positions and sizes,
wherein light rays cast from the edge points to the image capturing device are tangent to the cross-section.
3. The method of claim 1 , wherein: the one or more images consists of a single image captured by the single camera; and the analyzing is performed on images consisting of the single image.
4. The method of claim 1 , wherein the one or more images comprise images captured as the object moves and articulates.
5. The method of claim 1 , wherein the one or more images are captured such that a time between images is sufficiently small so as to avoid quality degradation due to object movement.
6. The method of claim 1 , wherein the capturing comprises controlling camera settings of the camera.
7. The method of claim 6 , wherein the camera settings are selected from a group including resolution, depth of field, sensitivity, light frequency and frame rate.
8. The method of claim 6 , wherein the controlling camera settings is conducted according to object parameters assumed to correspond to at least a portion of a type of object.
9. The method of claim 1 , wherein the at least one source comprises one or more light emitting diodes.
10. The method of claim 1 , wherein the at least one source comprises one or more light sources and one or more of the light sources are placed on either side of the camera.
11. The method of claim 1 , wherein the at least one source comprises one or more light sources and the one or more light sources are placed to create a contrast between at least a portion of the object and a background.
12. The method of claim 1 , wherein the at least one source comprises one or more light sources and the one or more light sources are placed to illuminate a region on the at least a portion of the object.
13. The method of claim 1 , wherein the at least one source comprises one or more light sources and at least a subset of the light sources is operable to emit at different wavelengths.
14. The method of claim 1 , wherein the at least one source comprises one or more light sources, at least a subset of the light sources is operable in sequence or simultaneously and one or more of the light sources is operable in a pulsed fashion.
15. The method of claim 1 , wherein the at least one source comprises one or more light sources and at least a subset of the light sources is operable at different brightnesses.
16. The method of claim 1 , wherein the at least one source comprises one or more light sources selected from a group including point light sources, arrays and bar lights.
17. The method of claim 1 , wherein the analyzing comprises applying a pixel threshold to determine portions of the image as corresponding to at least a portion of the object or background.
18. The method of claim 1 , wherein the edge points define a viewed portion of the cross-section such that the portion of the cross-section is within a field of view of the camera from the particular vantage point.
19. The method of claim 18 , wherein the computationally slicing comprises fitting a closed curve to the viewable portion of the object portion, the closed curve defining a 3D model of the object portion.
20. The method of claim 19 , wherein the closed curve is an ellipse.
21. The method of claim 19 , wherein the fitting a closed curve comprises:
determining a family of closed curves corresponding to the viewable portion of the object portion;
selecting a closed curve from the family of closed curves; and
selecting a rotation angle for the closed curve that corresponds with a determined contour of the object portion.
22. The method of claim 19 , wherein the computationally slicing further comprises approximating a contour of the portion of the cross-section within the field of view of the camera.
23. The method of claim 1 , wherein the at least one of the edge points is defined by a boundary between a shadow region and an illuminated region on the cross-section of the object.
24. The method of claim 1 , wherein the reconstructing comprises assembling the 2D slices to form a 3D model.
25. The method of claim 24 , further comprising using correlation to refine the 3D model.
26. The method of claim 25 , wherein the correlation comprises at least one of:
correlating among the 2D slices; and
correlating a 3D model resulting from stacking the 2D slices to preserve continuity with characteristic information of a corresponding object.
27. The method of claim 26 , wherein the characteristic information includes at least one of continuity in motion and continuity in deformation.
28. The method of claim 1 , wherein:
the casting an output comprises casting an output that causes an illuminated part of the object and a shadow region; and
the analyzing comprises detecting a shadow edge point of the shadow region and determining characteristic parameters of a closed curve corresponding to the shadow edge point.
29. The method of claim 1 , wherein the analyzing comprises determining closed curve characteristics as corresponding to an object type that corresponds to the object portion of the object.
30. The method of claim 29 , wherein the determining closed curve characteristics is conducted by comparing the object to object types in a library including object types.
31. The method of claim 1 , further comprising determining characteristics of an object model determined from the reconstructing; and, according to the characteristics of the object model, at least one of:
revising characteristic parameters in an object library and
defining characteristic parameters in an object library.
32. The method of claim 29 , wherein the closed curve characteristics are selected from a group including closed curve shapes, 3D closed curves and closed curve rotation angle.
33. The method of claim 29 , further comprising modeling at least a portion of the object with a 3D volume.
34. The method of claim 33 , wherein the 3D volume is selected from a group including ellipsoids, hyperboloids and cylinders.
35. The method of claim 33 , wherein the modeling at least a portion of the object comprises modeling at least one of a finger tip, a thumb and one or more finger segments with an ellipsoid.
36. The method of claim 1 , wherein the object portion is moving in 3D space; and
the one or more images capture a motion of the object portion.
37. The method of claim 36 , further comprising interpreting the captured motion as user input.
38. The method of claim 37 , wherein the motion of the object portion comprises one or more hand gestures.
39. The method of claim 1 , further comprising:
comparing a reconstructed position and shape of the object portion with characteristic parameters for the object to determine one or more object poses; and
interpreting the one or more object poses as user input.
40. The method of claim 1 , further comprising:
determining that a reconstructed shape of the object portion matches an object model in an object library; and
interpreting the reconstructed shape as user input.
41. The method of claim 1 , wherein the object comprises a hand and the at least a portion of the object comprises at least a portion of a finger.
42. The method of claim 1 , wherein the single camera has a field of view in an upward Direction.
43. A method of identifying a position and shape of at least a portion of an object (object portion) in three-dimensional (3D) space, the method comprising:
capturing, with a single camera, an image generated by casting an output from at least one source onto the object portion;
analyzing the image captured by the single camera, the analyzing of the image resulting in computationally slicing the object portion into a plurality of two-dimensional (2D) slices, each slice corresponding to a cross-section of the object portion, based at least in part on a plurality of edge points in the image; and
reconstructing the position and shape of at least a portion of the object portion in 3D space based at least in part on a plurality of identified cross-sectional positions and sizes,
wherein light rays cast from the edge points to the image capturing device are tangent to the cross-section.
44. The method of claim 43 , wherein the single camera has a particular vantage point.