IP Library Granted Patent US 11,495,002
Granted Patent B2
US 11,495,002 · App. 17/020,126 · Granted Nov 8, 2022

Systems and methods for determining the scale of human anatomy from images

Inventors: Eric J. Varady (San Francisco, CA); Atul Kanaujia (San Francisco, CA)
Assignee: BESPOKE, INC.
G06T19/20G06T3/0068G06T7/50G06T7/536G06T7/62G06T7/74G06T7/80G06T2207/10028G06T2207/30201G06T2207/30204G06T2219/2016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,495,002
App. No.
17/020,126
Granted
Nov 8, 2022
Kind
B2
Abstract

Systems and methods are disclosed for generating a scaled reconstruction for a consumer product. One method includes receiving digital input comprising a calibration target and an object; defining a three-dimensional coordinate system; positioning the calibration target in the three-dimensional coordinate system; based on the digital input, aligning the object to the calibration target in the three-dimensional coordinate system; and generating a scaled reconstruction of the object based on the alignment of the object to the calibration target in the three-dimensional coordinate system.

Claims (67)

1. A computer-implemented method for generating a scaled reconstruction of a user's face, using a computer system, the method comprising:

receiving a digital input comprising a digital representation of the user's face and a calibration target proximate to the user's face;

processing the digital input to detect facial features of the user's face and geometrical parameters of the calibration target;

defining a three-dimensional coordinate system representing a three-dimensional space for scaling the user's face using the calibration target;

aligning the calibration target to the user's face in the three-dimensional coordinate system based on the detected facial features and the geometrical parameters, wherein the calibration target is positioned within a boundary of a predicted location on the digital input, and wherein features of the calibration target outside the boundary are ignored; and

generating the scaled reconstruction of the user's face based on the alignment of the calibration target to the user's face in the three-dimensional coordinate system.

2. The method of claim 1 , further comprising:

determining three-dimensional measurements of the calibration target in the three-dimensional coordinate system;

determining pupillary distance measurements from the detected facial features of the user's face and the three-dimensional measurements of the calibration target, wherein the pupillary distance measurements include distance between pupils focusing on a distant target or a near target; and

further generating the scaled reconstruction of the user's face based on the pupillary distance measurements and the three-dimensional measurements of the calibration target.

3. The method of claim 2 , further comprising:

receiving a three-dimensional reconstruction of the user's face;

aligning the calibration target to the three-dimensional reconstruction of the user's face; and

generating the scaled reconstruction based on the alignment of the calibration target to the three-dimensional reconstruction of the user's face.

4. The method of claim 1 , wherein the calibration target is comprised of a parameterized three-dimensional reconstruction, and wherein the calibration target is distorted based, at least in part, on facial movements, eye movements, or a combination thereof to fit the detected facial features of the user's face.

5. The method of claim 1 , further comprising:

determining, from the digital input, an image including the calibration target and the user's face;

determining, from the image, a three-dimensional location of the user's face, position information of the user's face, or a combination thereof;

predicting the location for the calibration target based, at least in part, on the three-dimensional location of the user's face, the position information of the user's face, or a combination thereof; and

positioning the calibration target in the three-dimensional coordinate system based on the predicted location.

6. The method of claim 1 , further comprising:

determining a scaling measurement based on aligning the user's face to the calibration target in the three-dimensional coordinate system; and

generating the scaled reconstruction of the user's face based on the scaling measurement.

7. The method of claim 1 , wherein the digital input includes one or more of a series of images from a singular image sensor taken from different camera positions, a video taken from different camera positions, a series of images or a video taken from different perspectives with depth information included, a 3D point cloud captured from a depth or 3D sensor, a series of images from multiple 2D sensors, a video captured from multiple 2D sensors, or a combination thereof.

8. The method of claim 1 , further comprising:

generating a plurality of points at pre-determined locations around the detected facial features of the user's face, wherein the detected facial features include center of eyes, corners of the eyes, or a combination thereof; and

tracking the plurality of points and averaging distance between the plurality of points for scaling the user's face in the three-dimensional coordinate system.

9. The method of claim 1 , further comprising:

generating a notification in at least one image capture device to assist in capturing a plurality of images of the calibration target and the user's face, wherein the notifications are generated based, at least in part, on a pre-set timing, feature detection and pose estimation, accelerometer data, gyroscope data, user response, or a combination thereof.

10. A system for generating a scaled reconstruction of a user's face, the system comprising:

a data storage device storing instructions for generating the scaled reconstruction of the user's face to generate a consumer product for the user; and

a processor configured to execute the instructions to perform a method including:

receiving a digital input comprising a digital representation of the user's face and a calibration target proximate to the user's face;

processing the digital input to detect facial features of the user's face and geometrical parameters of the calibration target;

defining a three-dimensional coordinate system representing a three-dimensional space for scaling the user's face using the calibration target;

aligning the calibration target to the user's face in the three-dimensional coordinate system based on the detected facial features and the geometrical parameters, wherein the calibration target is positioned within a boundary of a predicted location on the digital input, and wherein features of the calibration target outside the boundary are ignored;

generating the scaled reconstruction of the user's face based on the alignment of the calibration target to the user's face in the three-dimensional coordinate system.

11. The system of claim 10 , wherein the system is further configured for:

determining three-dimensional measurements of the calibration target in the three-dimensional coordinate system;

determining pupillary distance measurements from the detected facial features of the user's face and the three-dimensional measurements of the calibration target, wherein the pupillary distance measurements include distance between pupils focusing on a distant target or a near target; and

further generating the scaled reconstruction of the user's face based on the pupillary distance measurements and the three-dimensional measurements of the calibration target.

12. The system of claim 11 , wherein the system is further configured for:

receiving a three-dimensional reconstruction of the user's face;

aligning the calibration target to the three-dimensional reconstruction of the user's face; and

generating the scaled reconstruction based on the alignment of the calibration target to the three-dimensional reconstruction of the user's face.

13. The system of claim 10 , wherein the calibration target is comprised of a parameterized three-dimensional reconstruction, and wherein the calibration target is distorted based, at least in part, on facial movements, eye movements, or a combination thereof to fit the detected facial features of the user's face.

14. The system of claim 10 , wherein the system is further configured for:

determining, from the digital input, an image including the calibration target and the user's face;

determining, from the image, a three-dimensional location of the user's face, position information of the user's face, or a combination thereof;

predicting the location for the calibration target based, at least in part, on the three-dimensional location of the user's face, the position information of the user's face, or a combination thereof; and

positioning the calibration target in the three-dimensional coordinate system based on the predicted location.

15. The system of claim 10 , wherein the system is further configured for:

determining a scaling measurement based on aligning the user's face to the calibration target in the three-dimensional coordinate system; and

generating the scaled reconstruction of the user's face based on the scaling measurement.

16. The system of claim 10 , wherein the first digital input includes one or more of a series of images from a singular image sensor taken from different camera positions, a video taken from different camera positions, a series of images or a video taken from different perspectives with depth information included, a 3D point cloud captured from a depth or 3D sensor, a series of images from multiple 2D sensors, a video captured from multiple 2D sensors, or a combination thereof.

17. A non-transitory computer readable medium for use on a computer system containing computer-executable programming instructions for generating a scaled reconstruction of a user's face, the method comprising:

receiving a digital input comprising a digital representation of the user's face and a calibration target proximate to the user's face;

processing the digital input to detect facial features of the user's face and geometrical parameters of the calibration target;

defining a three-dimensional coordinate system representing a three-dimensional space for scaling the user's face using the calibration target;

aligning the calibration target to the user's face in the three-dimensional coordinate system based on the detected facial features and the geometrical, wherein the calibration target is positioned within a boundary of a predicted location on the digital input, and wherein features of the calibration target outside the boundary are ignored;

generating the scaled reconstruction of the user's face based on the alignment of the calibration target to the user's face in the three-dimensional coordinate system.

18. The non-transitory computer readable medium of claim 17 , the method further comprising:

determining three-dimensional measurements of the calibration target in the three-dimensional coordinate system;

determining pupillary distance measurements from the detected facial features of the user's face and the three-dimensional measurements of the calibration target, wherein the pupillary distance measurements include distance between pupils focusing on a distant target or a near target; and

further generating the scaled reconstruction of the user's face based on the pupillary distance measurements and the three-dimensional measurements of the calibration target.

19. The non-transitory computer readable medium of claim 17 , wherein the calibration target is comprised of a parameterized three-dimensional reconstruction, and wherein the calibration target is distorted based, at least in part, on facial movements, eye movements, or a combination thereof to fit the detected facial features of the user's face.

20. The non-transitory computer readable medium of claim 17 , wherein the digital input includes one or more of a series of images from a singular image sensor taken from different camera positions, a video taken from different camera positions, a series of images or a video taken from different perspectives with depth information included, a 3D point cloud captured from a depth or 3D sensor, a series of images from multiple 2D sensors, a video captured from multiple 2D sensors, or a combination thereof.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2020
From: VARADY, ERIC J.; KANAUJIA, ATUL
To: BESPOKE, INC. D/B/A TOPOLOGY EYEWEAR
Reel/Frame 053942/0262 →
Continuity (3)
Continuation 15981495 · May 16, 2018
Provisional Application 62507631 · May 17, 2017
Related Publication 20200410775A1 · Dec 31, 2020