IP Library Granted Patent US 9,418,474
Granted Patent B2
US 9,418,474 · App. 12/811,237 · Granted Aug 16, 2016

Three-dimensional model refinement

Inventors: Janos Rohaly (Acton, MA); Ilya A. Kriveshko (Littleton, MA); Dmitriy A. Dedkov (Everett, MA)
Assignee: 3M INNOVATIVE PROPERTIES COMPANY
G06T17/00A61C13/0004G06K9/00201G06K9/00214H04N13/0221G06T2207/10012G06T2210/41
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,418,474
App. No.
12/811,237
Granted
Aug 16, 2016
Kind
B2
Abstract

A three-dimensional measurement is refined by warping two-dimensional images of an object from offset camera positions according to a three-dimensional model of the object, and applying any resulting discrepancies to refine the three-dimensional model, or to refine one of a number of three-dimensional measurements used to create the three-dimensional model.

Claims (36)

1. A method of refining a three-dimensional model comprising the steps of:

providing a three-dimensional model of an object;

obtaining a first two-dimensional image of the object from a first camera pose;

obtaining a second two-dimensional image of the object from a second camera pose, wherein the second two-dimensional image includes a common portion of a surface of the object with the first two-dimensional image;

projecting the first two-dimensional image onto the three-dimensional model, thereby providing a projected two-dimensional image;

deforming the projected two-dimensional image based upon a spatial relationship of the first camera pose, the second camera pose, and the three-dimensional model to obtain an expected image from the second camera pose;

comparing the second two-dimensional image to the expected image to identify one or more discrepancies; and

correcting the three-dimensional model based upon the one or more discrepancies.

2. The method of claim 1 wherein the first camera pose and the second camera pose include a position and an orientation of a single camera in two dependent positions.

3. The method of claim 2 wherein the first camera pose and the second camera pose include a position and an orientation of two offset channels of a multi-aperture camera.

4. The method of claim 1 wherein the first camera pose and the second camera pose include a position and an orientation of a single camera in two independent positions.

5. The method of claim 4 wherein a relationship between the first camera pose and the second camera pose is calculated based upon a three-dimensional measurement of the surface of the object from each of the first camera pose and the second camera pose.

6. The method of claim 1 further comprising deriving the three-dimensional model from a plurality of three-dimensional measurements of the surface of the object from a plurality of camera poses including the first camera pose and the second camera pose.

7. The method of claim 1 further comprising applying the one or more discrepancies to directly refine the three-dimensional model.

8. The method of claim 1 further comprising applying the one or more discrepancies to refine a three-dimensional measurement from one or more of the first camera pose and the second camera pose to provide a refined measurement.

9. The method of claim 8 further comprising refining a camera path calculation for a camera path used to create the three-dimensional model using the refined measurement to provide a refined camera path.

10. The method of claim 9 further comprising using the refined camera path and the refined measurement to refine the three-dimensional model.

11. The method of claim 1 wherein the three-dimensional model includes a point cloud or a polygonal mesh.

12. A computer program product for refining a three-dimensional model of an object comprising computer executable code embodied on a computer readable medium that, when executing on one or more computing devices, performs the steps of:

providing a three-dimensional model of an object;

obtaining a first two-dimensional image of the object from a first camera pose;

obtaining a second two-dimensional image of the object from a second camera pose, wherein the second two-dimensional image includes a common portion of a surface of the object with the first two-dimensional image;

projecting the first two-dimensional image onto the three-dimensional model, thereby providing a projected image;

deforming the projected image based upon a spatial relationship of the first camera pose, the second camera pose, and the three-dimensional model to obtain an expected image from the second camera pose;

comparing the second two-dimensional image to the expected image to identify one or more discrepancies; and

correcting the three-dimensional model based upon the one or more discrepancies.

13. The computer program product of claim 12 wherein the first camera pose and the second camera pose include a position and an orientation of a single camera in two dependent positions.

14. The computer program product of claim 13 wherein the first camera pose and the second camera pose include a position and an orientation of two offset channels of a multi-aperture camera.

15. The computer program product of claim 12 wherein the first camera pose and the second camera pose include a position and an orientation of a single camera in two independent positions.

16. The computer program product of claim 15 wherein a relationship between the first camera pose and the second camera pose is calculated based upon a three-dimensional measurement of the surface of the object from each of the first camera pose and the second camera pose.

17. The computer program product of claim 12 further comprising code for performing the step of deriving the three-dimensional model from a plurality of three-dimensional measurements of the surface of the object from a plurality of camera poses including the first camera pose and the second camera pose.

18. The computer program product of claim 12 further comprising code for performing the step of applying the one or more discrepancies to directly refine the three-dimensional model.

19. The computer program product of claim 12 further comprising code for performing the step of applying the one or more discrepancies to refine a three-dimensional measurement from one or more of the first camera pose and the second camera pose to provide a refined measurement.

20. The computer program product of claim 19 further comprising code for performing the step of refining a camera path calculation for a camera path used to create the three-dimensional model using the refined measurement to provide a refined camera path.

21. The computer program product of claim 20 further comprising code for performing the step of using the refined camera path and the refined measurement to refine the three-dimensional model.

22. The computer program product of claim 12 wherein the three-dimensional model includes a point cloud or a polygonal mesh.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 15, 2022
From: MIDMARK CORPORATION
To: MEDIT CORP.
Reel/Frame 059615/0654 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2019
From: 3M INNOVATIVE PROPERTIES COMPANY
To: MIDMARK CORPORATION
Reel/Frame 049065/0301 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2010
From: ROHALY, JANOS; KRIVESHKO, ILYA A.; DEDKOV, DMITRIY A.
To: 3M INNOVATIVE PROPERTIES COMPANY
Reel/Frame 025024/0988 →
Continuity (2)
Provisional Application 61019159 · Jan 4, 2008
Related Publication 20110043613A1 · Feb 24, 2011