Systems and methods of object shape and position determination in three-dimensional (3D) space
View Patent ↗Methods and systems for capturing motion and/or determining the shapes and positions of one or more objects in 3D space utilize cross-sections thereof. In various embodiments, images of the cross-sections are captured using a camera based on edge points thereof.
1. A smart phone having an interface that identifies a position and a shape of a portion of a human hand moving in a three-dimensional (3D) space, the smart phone comprising:
a fixed function logic circuit storing instructions that, when executed, implement actions including:
analyzing two or more images captured by a camera from a particular vantage point to computationally represent a portion of an object as one or more mathematically represented 3D surfaces, each 3D surface corresponding to a cross-section of the portion of the object, based at least in part on a plurality of edge points of the portion of the object in the image, tangent lines extending from the camera to at least two edge points of the plurality of edge points, and a centerline corresponding to the tangent lines; and
reconstructing the position of, and the shape fitting, at least the portion of the object in the 3D space based at least in part on the plurality of edge points and the centerline.
2. The smart phone of claim 1 , further including: at least one source that casts an output onto the portion of the object.
3. The smart phone of claim 1 , further including transmitting to at least one further process, a signal that includes at least one selected from (i) trajectory information determined from the reconstructed position of, and the shape fitting, the at least a portion of the object that the at least one further process interprets, and (ii) gesture information interpreted from trajectory information for the portion of the object by the smart phone.
4. The smart phone of claim 1 , further comprising a time-of-flight camera, and wherein a plurality of points on at least one surface of the portion of the object are determined from time-of-flight data.
5. The smart phone of claim 4 , wherein the fixed function logic circuit further implements actions that combine time-of-flight data with tangent-line information to provide a more detailed model of the shape of at least the portion of the object.
6. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that analyze the one or more surfaces to produce a 3D model.
7. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that refine a 3D model by correlating a cross-section of an object portion represented by a first surface with a cross-section of at least one different object portion represented by at least one different 3D surface.
8. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that refines a 3D model based on at least one of an object type of an object comprising the portion of the object and characteristics of the object type.
9. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that refines a 3D model by informing results of the analyzing with results of such analyzing as conducted on images obtained at a different time and correspondingly modifying the 3D model.
10. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that refine a position of the portion of the object based upon informing results of the analyzing with results of such analyzing as conducted on images obtained at a different time.
11. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that determine constraints on at least one of a number and position of one or more object portions according an object type.
12. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that resolve an at least partially occluded portion of the object by conducting spatial correlations across surfaces.
13. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that capture one or more additional image sets of two or more images, and resolve an at least partially occluded portion of the object by conducting temporal correlations across image sets.
14. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that resolve an at least partially occluded portion of the object by applying physical constraints of an object type.
15. The smart phone of claim 1 , wherein the fixed function logic circuit further implements actions that refine a 3D model formed by the reconstructing.
16. The smart phone of claim 15 , wherein the 3D model represents a generic hand and the refining of the 3D model comprises texture mapping details from image capture onto a generic hand model.
17. The smart phone of claim 15 , wherein the fixed function logic circuit further implements actions that morph the 3D model according to detected object orientation and motion.
18. The smart phone of claim 15 , wherein the refining of the 3D model includes refining the 3D model based at least in part upon one of continuity in motion and continuity in deformation.
19. A method of using a smart phone to identify a position and a shape of an object moving in a three-dimensional (3D) space, the method comprising:
executing instructions on a fixed function logic circuit of the smart phone to implement actions including:
analyzing two or more images captured by a camera from a particular vantage point to computationally represent a portion of an object as one or more mathematically represented 3D surfaces, each 3D surface corresponding to a cross-section of the portion of the object, based at least in part on a plurality of edge points of the portion of the object in the image, tangent lines extending from the camera to at least two edge points of the plurality of edge points, and a centerline corresponding to the tangent lines; and
reconstructing the position of, and the shape fitting, at least the portion of the object in the 3D space based at least in part on the plurality of edge points and the centerline.
20. A head mounted device, comprising:
an optical assembly to display a displayed environment including at least one of a surrounding three-dimensional (3D) environment and a virtual environment, the surrounding 3D environment including at least a portion of an object; and
a smart phone having an interface that identifies a position and a shape of a portion of the object moving in the surrounding 3D environment, the smart phone including:
a fixed function logic circuit storing instructions that, when executed, implement actions including:
analyzing two or more images captured by a camera from a particular vantage point to computationally represent a portion of an object as one or more mathematically represented 3D surfaces, each 3D surface corresponding to a cross-section of the portion of the object, based at least in part on a plurality of edge points of the portion of the object in the image, tangent lines extending from the camera to at least two edge points of the plurality of edge points, and a centerline corresponding to the tangent lines; and
reconstructing the position of, and the shape fitting, at least the portion of the object in the surrounding 3D environment based at least in part on the plurality of edge points and the centerline.