Granted Patent B2

US 10,777,018 · App. 15/981,495 · Granted Sep 15, 2020

Inventors: Eric J. Varady (San Francisco, CA); Atul Kanaujia (San Francisco, CA)

Assignee: Bespoke, Inc.

G06T19/20G06T3/0068G06T7/50G06T7/536G06T7/62G06T7/74G06T7/80G06T2207/10028G06T2207/30201G06T2207/30204G06T2219/2016

Abstract

Claims (70)

1. A computer-implemented method for generating a scaled reconstruction of an object relative to an individual's anatomy, using a computer system, the method comprising:

receiving, from an image capture device, a first digital input comprising a digital representation of a calibration target and an object and/or individual's anatomy proximate to the calibration target;

defining a three-dimensional coordinate system representing a three-dimensional space for scaling the object using the calibration target;

positioning the object in the three-dimensional coordinate system;

positioning the calibration target in the three-dimensional coordinate system;

aligning the object to the calibration target in the three-dimensional coordinate system based on the first digital input;

generating a scaled reconstruction of the object based on the alignment of the object to the calibration target in the three-dimensional coordinate system based on the first digital input;

receiving, from a depth sensor, a second digital input comprising depth information of the object and/or the individual's anatomy; and

generating a digital representation of the scaled reconstruction of the object overlaid on the individual's anatomy based on the first digital input and the second digital input comprising the depth information of the object and/or the individual's anatomy based on the second digital input received from the depth sensor.

2. The method of claim 1 , further comprising:

determining three-dimensional measurements of the calibration target in the three-dimensional coordinate system; and

further generating the scaled reconstruction of the object based on the three-dimensional measurements of the calibration target.

3. The method of claim 1 , wherein the calibration target is comprised of a parameterized three-dimensional reconstruction.

4. The method of claim 1 , further comprising:

receiving a three-dimensional reconstruction of the object;

aligning the calibration target to the three-dimensional reconstruction of the object; and

generating the scaled reconstruction based on the alignment of the calibration target to the three-dimensional reconstruction of the object.

5. The method of claim 4 , further comprising:

generating the three-dimensional reconstruction of the object.

6. The method of claim 1 , further comprising:

determining, from the first digital input, an image including the calibration target and the object and/or the individual's anatomy;

determining, from the image, a three-dimensional location of the calibration target and a three-dimensional location of the object; and

positioning the calibration target in the three-dimensional coordinate system and positioning the object in the three-dimensional coordinate system, based on the three-dimensional location of the calibration target and the three-dimensional location of the object.

7. The method of claim 1 , further comprising:

determining a scaling measurement based on aligning the object to the calibration target in the three-dimensional coordinate system; and

generating the scaled reconstruction of the object based on the scaling measurement.

8. The method of claim 1 , wherein the first digital input includes one or more of a series of images from a singular image sensor taken from different camera positions, a video taken from different camera positions, a series of images or a video taken from different perspectives with depth information included, a 3D point cloud captured from a depth or 3D sensor, a series of images from multiple 2D sensors, a video captured from multiple 2D sensors, or a combination thereof.

9. A system for generating a scaled reconstruction of an object relative to an individual's anatomy, the system comprising:

a data storage device storing instructions for generating a scaled reconstruction for a consumer product; and

a processor configured to execute the instructions to perform a method including:

receiving, from an image capture device, a first digital input comprising a digital representation of a calibration target and an object and/or individual's anatomy proximate to the calibration target;

defining a three-dimensional coordinate system representing a three-dimensional space for scaling the object using the calibration target;

positioning the object in the three-dimensional coordinate system;

positioning the calibration target in the three-dimensional coordinate system;

aligning the object to the calibration target in the three-dimensional coordinate system based on the first digital input;

generating a scaled reconstruction of the object based on the alignment of the object to the calibration target in the three-dimensional coordinate system based on the first digital input;

receiving, from a depth sensor, a second digital input comprising depth information of the object and/or the individual's anatomy; and

generating a digital representation of the scaled reconstruction of the object overlaid on the individual's anatomy based on the first digital input and the second digital input comprising the depth information of the object and/or the individual's anatomy based on the second digital input received from the depth sensor.

10. The system of claim 9 , wherein the system is further configured for:

determining three-dimensional measurements of the calibration target in the three-dimensional coordinate system; and

further generating the scaled reconstruction of the object based on the three-dimensional measurements of the calibration target.

11. The system of claim 9 , wherein the calibration target is comprised of a parameterized three-dimensional reconstruction.

12. The system of claim 11 , wherein the system is further configured for:

receiving a three-dimensional reconstruction of the object;

aligning the calibration target to the three-dimensional reconstruction of the object; and

generating the scaled reconstruction based on the alignment of the calibration target to the three-dimensional reconstruction of the object.

13. The system of claim 12 , wherein the system is further configured for:

generating the three-dimensional reconstruction of the object.

14. The system of claim 9 , wherein the system is further configured for:

determining, from the first digital input, an image including the calibration target and the object and/or the individual's anatomy;

determining, from the image, a three-dimensional location of the calibration target and a three-dimensional location of the object; and

positioning the calibration target in the three-dimensional coordinate system and positioning the object in the three-dimensional coordinate system, based on the three-dimensional location of the calibration target and the three-dimensional location of the object.

15. The system of claim 9 , wherein the system is further configured for:

determining a scaling measurement based on aligning the object to the calibration target in the three-dimensional coordinate system; and

generating the scaled reconstruction of the object based on the scaling measurement.

16. The system of claim 9 , wherein the first digital input includes one or more of a series of images from a singular image sensor taken from different camera positions, a video taken from different camera positions, a series of images or a video taken from different perspectives with depth information included, a 3D point cloud captured from a depth or 3D sensor, a series of images from multiple 2D sensors, a video captured from multiple 2D sensors, or a combination thereof.

17. A non-transitory computer readable medium for use on a computer system containing computer-executable programming instructions for generating a scaled reconstruction of an object relative to an individual's anatomy, the method comprising:

receiving, from an image capture device, a first digital input comprising a digital representation of a calibration target and an object and/or individual's anatomy proximate to the calibration target;

defining a three-dimensional coordinate system representing a three-dimensional space for scaling the object using the calibration target;

positioning the object in the three-dimensional coordinate system;

positioning the calibration target in the three-dimensional coordinate system;

aligning the object to the calibration target in the three-dimensional coordinate system based on the first digital input;

generating a scaled reconstruction of the object based on the alignment of the object to the calibration target in the three-dimensional coordinate system based on the first digital input;

receiving, from a depth sensor, a second digital input comprising depth information of the object and/or the individual's anatomy; and

generating a digital representation of the scaled reconstruction of the object overlaid on the individual's anatomy based on the first digital input and the second digital input comprising the depth information of the object and/or the individual's anatomy based on the second digital input received from the depth sensor.

18. The non-transitory computer readable medium of claim 17 , the method further comprising:

determining three-dimensional measurements of the calibration target in the three-dimensional coordinate system; and

further generating the scaled reconstruction of the object based on the three-dimensional measurements of the calibration target.

19. The non-transitory computer readable medium of claim 17 , wherein the calibration target is comprised of a parameterized three-dimensional reconstruction.

20. The non-transitory computer readable medium of claim 17 , wherein the first digital input includes one or more of a series of images from a singular image sensor taken from different camera positions, a video taken from different camera positions, a series of images or a video taken from different perspectives with depth information included, a 3D point cloud captured from a depth or 3D sensor, a series of images from multiple 2D sensors, a video captured from multiple 2D sensors, or a combination thereof.

Assignments (1)

Continuity (2)

Cited By (9)

US 12,186,017 US 12,216,341 US 12,271,969 US 12,307,621 US 12,406,466 US 12,444,232 US 12,573,865 US 12,678,682 US 12,699,287