IP Library Granted Patent US 7,605,817
Granted Patent B2
US 7,605,817 · App. 11/270,135 · Granted Oct 20, 2009

Determining camera motion

Assignee: 3M Innovative Properties Company
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,605,817
App. No.
11/270,135
Granted
Oct 20, 2009
Kind
B2
Abstract

Camera motion is determined in a three-dimensional image capture system using a combination of two-dimensional image data and three-dimensional point cloud data available from a stereoscopic, multi-aperture, or similar camera system. More specifically, a rigid transformation of point cloud data between two three-dimensional point clouds may be more efficiently parameterized using point correspondence established between two-dimensional pixels in source images for the three-dimensional point clouds.

Claims (52)

1. A method of operating a three-dimensional imaging system comprising:

receiving a first image set, the first image set including a first plurality of two-dimensional views of a subject from a first position wherein the subject is substantially larger than an image plane of the first image set;

receiving a second image set, the second image set including a second plurality of two-dimensional views of a subject from a second position;

deriving a first point cloud from the first image set;

identifying a number of corresponding pixels between a number of pixels of one of the first plurality of two-dimensional views and a number of pixels of one of the second plurality of two-dimensional views;

deriving a second point cloud from the number of corresponding pixels of the second image set;

determining a motion from the first position to the second position using the first point cloud and the second point cloud; and

repeating steps of receiving the first image set, receiving the second image set, deriving the first point cloud, identifying the number of corresponding pixels, deriving the second point cloud, and determining the motion, for a plurality of consecutive three-dimensional images whereby an aggregate point cloud is constructed for the subject.

2. The method of claim 1 wherein the subject is fixed, the first position and the second position being different positions of a moving camera.

3. The method of claim 1 wherein a camera is fixed and the subject moves, the first position and the second position being different positions of the subject.

4. The method of claim 1 wherein determining motion includes determining a rigid transformation from the first point cloud to the second point cloud.

5. The method of claim 4 wherein determining motion includes weighting one or more points of the point clouds according to confidence.

6. The method of claim 1 wherein the first point cloud is derived from a number of pixels of the first image set using stereo matching.

7. The method of claim 1 wherein the motion includes a change in at least one of a position or an orientation.

8. The method of claim 1 wherein the first point cloud includes a subset of points from the first image set.

9. The method of claim 8 further comprising deriving the subset of points from a sparse mesh of pixels within the first image set.

10. The method of claim 1 wherein the second point cloud includes a subset of points from the second image set.

11. The method of claim 1 wherein one of the plurality of consecutive motions cannot be determined directly from point cloud data of consecutive views, the method further comprising recovering the one of the plurality of consecutive motions through an inverse transformation of one or more next consecutive motions for one or more next consecutive point clouds.

12. The method of claim 1 further comprising supplementing the first point cloud with one or more additional points from the first image set having one or more additional corresponding points in the second image set.

13. The method of claim 1 further comprising supplementing the second point cloud with one or more additional points from the second image set having one or more additional corresponding points in the first image set.

14. The method of claim 1 wherein the first image set and the second image set are obtained from a multi-aperture camera.

15. The method of claim 14 wherein identifying one or more corresponding pixels further includes identifying corresponding pixels between two-dimensional views from a center aperture of the multi-aperture camera.

16. The method of claim 1 wherein the first image set and the second image set are obtained from a camera system including a plurality of cameras in a fixed relationship to one another.

17. The method of claim 1 wherein the subject includes one or more of a nose, an ear, one or more teeth, or a body part.

18. The method of claim 1 wherein the subject includes a dental model.

19. The method of claim 18 wherein the dental model includes one or more of a plaster case, a wax-up, an impression, and a negative impression.

20. The method of claim 1 wherein the subject includes a physical model of an object for use in a digital animation process.

21. A system comprising:

a camera, the camera configured to capture a first image set including a first plurality of two-dimensional views of a subject from a first position and a second image set including a second plurality of two-dimensional views of a subject from a second position; and

a processor coupled in a communicating relationship with the camera, the processor configured to derive a first point cloud from the first image set, identify a number of corresponding pixels between a number of pixels of one of the first plurality of two-dimensional views and a number of pixels of one of the second plurality of two-dimensional views, and derive a second point cloud from the number of corresponding pixels of the second image set, the processor further configured to determine a motion from the first position to the second position using the first point cloud and the second point cloud wherein the subject is substantially larger than an image plane of the first image set, the camera capturing an additional plurality of consecutive image sets and the processor configured to derive a plurality of additional point clouds and determine a plurality of consecutive motions therefrom, whereby an aggregate point cloud is constructed for the subject.

22. The system of claim 21 wherein the subject is fixed, the first position and the second position being different positions of the camera.

23. The system of claim 21 wherein the camera is fixed and the subject moves, the first position and the second position being different positions of the subject.

24. The system of claim 21 wherein the processor is configured to determine motion using a rigid transformation from the first point cloud to the second point cloud.

25. The computer program product of claim 24 wherein the processor is configured to determine motion using weighted values for one or more points of the point clouds.

26. The system of claim 21 wherein the first point cloud is derived from a number of pixels of the first image set using stereo matching.

27. The system of claim 21 wherein the motion includes a change in at least one of a position or an orientation.

28. The system of claim 21 wherein the first point cloud includes a subset of points from the first image set.

29. The system of claim 21 wherein one of the plurality of consecutive motions cannot be determined directly from point cloud data of consecutive image sets, the processor further configured to recover one of the plurality of consecutive motions through an inverse transformation of one or more next consecutive motions for one or more next consecutive point clouds.

30. The system of claim 21 wherein the processor is further configured to supplement the first point cloud with one or more additional points from the first image set having one or more additional corresponding points in the second image set.

31. The system of claim 21 wherein the processor is further configured to supplement the second point cloud with one or more additional points from the second image set having one or more additional corresponding points in the first image set.

32. The system of claim 21 wherein the camera includes a multi-aperture camera.

33. The system of claim 32 wherein the processor is configured to identify corresponding pixels between two-dimensional views from a center aperture of the multi-aperture camera.

34. The system of claim 21 wherein the camera includes a plurality of cameras in a fixed relationship to one another.

35. The system of claim 21 wherein the subject includes one or more of a body part, a nose, an ear, one or more teeth, or a physical model of an object for use in a digital animation process.

36. A system comprising:

first receiving means for receiving a first image set, the first image set including a first plurality of two-dimensional views of a subject from a first position wherein the subject is substantially larger than an image plane of the first image set;

second receiving means for receiving a second image set, the second image including a second plurality of two-dimensional views of a subject from a second position;

deriving means for deriving a first point cloud from the first image set;

first identifying means for identifying a number of corresponding pixels between a number of pixels of one of the first plurality of two-dimensional views and a number of pixels of one of the second plurality of two-dimensional views;

second deriving means for deriving a second point cloud from the number of corresponding pixels of the second image set;

motion determining means for determining a motion from the first position to the second position using the first point cloud and the second point cloud; and

capturing means for capturing an additional plurality of consecutive image sets and deriving a plurality of additional point clouds and determining a plurality of consecutive motions therefrom, whereby an aggregate point cloud is constructed for the subject.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 15, 2022
From: MIDMARK CORPORATION
To: MEDIT CORP.
Reel/Frame 059615/0654 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2019
From: 3M INNOVATIVE PROPERTIES COMPANY
To: MIDMARK CORPORATION
Reel/Frame 049065/0301 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2007
From: 3M COMPANY
To: 3M INNOVATIVE PROPERTIES COMPANY
Reel/Frame 018818/0988 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2006
From: BRONTES TECHNOLOGIES, INC., A CORPORATION OF THE STATE OF DELAWARE
To: 3M COMPANY
Reel/Frame 018656/0864 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2006
From: BRONTES TECHNOLOGIES, INC. A CORPORATION OF THE STATE OF DELAWARE
To: 3M COMPANY
Reel/Frame 018614/0950 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2005
From: ZHANG, TONG; ROHALY, JANOS; WEEKS, STEVEN
To: BRONTES TECHNOLOGIES, INC
Reel/Frame 016799/0524 →
Continuity (1)
Related Publication 20070103460A1 · May 10, 2007