IP Library › Granted Patent US 11,587,268
Granted Patent B2
US 11,587,268 · App. 16/555,982 · Granted Feb 21, 2023

Methods and apparatus for generating a three-dimensional reconstruction of an object with reduced distortion

Inventors: Martin Schaffer (Jena, DE); Marcus Große (Jena, DE)
Assignee: Cognex Corporation
G06T11/003H04N5/72H04N13/243H04N13/296H04N2013/0074
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,587,268
App. No.
16/555,982
Granted
Feb 21, 2023
Kind
B2
Abstract

Methods, systems, and computer readable media for generating a three-dimensional reconstruction of an object with reduced distortion are described. In some aspects, a system includes at least two image sensors, at least two projectors, and a processor. Each image sensor is configured to capture one or more images of an object. Each projector is configured to illuminate the object with an associated optical pattern and from a different perspective. The processor is configured to perform the acts of receiving, from each image sensor, for each projector, images of the object illuminated with the associated optical pattern and generating, from the received images, a three-dimensional reconstruction of the object. The three-dimensional reconstruction has reduced distortion due to the received images of the object being generated when each projector illuminates the object with an associated optical pattern from the different perspective.

Claims (55)

1. A system for generating a three-dimensional reconstruction of an object with reduced distortion, comprising:

at least two image sensors, each image sensor configured to capture one or more images of an object;

at least two projectors, each of the at least two projectors configured to illuminate the object with an associated optical pattern, wherein:

a first projector of the at least two projectors illuminates the object from a first perspective,

a second projector of the at least two projectors illuminates the object from a second perspective that is different from the first perspective, and

an angular separation of the first perspective and the second perspective is larger than a half-angle of a reflectance cone of a surface of the object; and

a processor configured to perform the acts of:

receive, from each of the at least two image sensors, for each projector of the at least two projectors, a plurality of images of the object illuminated with the associated optical pattern for the projector; and

generate, from the received plurality of images, a three-dimensional reconstruction of the object, wherein the three-dimensional reconstruction has reduced distortion due to the received plurality of images of the object being generated when each of the at least two projectors illuminates the object with the associated optical pattern from the first and second perspectives.

2. The system of claim 1 , wherein one or more of the at least two projectors are not synchronized with operation of the at least two image sensors.

3. The system of claim 1 , wherein the at least two image sensors are time-synchronized to capture the plurality of images of the object at a same time.

4. The system of claim 1 , wherein the at least two image sensors are part of at least one subset of a plurality of image sensors, and wherein the at least two image sensors are time-synchronized to capture the plurality of images of the object at a same time.

5. The system of claim 1 , wherein the at least two image sensors comprise:

a first image sensor and a second image sensor time-synchronized with each other to capture one or images of the object at a first time; and

a third image sensor and a fourth image sensor time-synchronized with each other to capture one or images of the object at a second time different from the first time.

6. The system of claim 1 , wherein the at least two image sensors comprise a first image sensor, a second image sensor, and a third image sensor time-synchronized to capture the one or images of the object at a same time.

7. The system of claim 1 , wherein:

the processor is configured to perform the acts of:

receive, from the first image sensor, a first set of images of the object illuminated with the optical pattern from the first projector at a first time;

receive, from the second image sensor, a second set of images of the object illuminated with the optical pattern from the first projector at the first time;

receive, from the first image sensor, a third set of images of the object illuminated with the optical pattern from the second projector at a second time; and

receive, from the second image sensor, a fourth set of images of the object illuminated with the optical pattern from the second projector at the second time.

8. The system of claim 1 , wherein the optical pattern comprises a moving optical pattern.

9. The system of claim 8 , wherein the moving optical pattern comprises an optical pattern translating along a circular path.

10. The system of claim 1 , wherein:

the at least two image sensors comprise three or more image sensors,

wherein a first subset of image sensors, which comprises two or more of the three or more image sensors, are time-synchronized with each other, and

wherein a second subset of image sensors, which comprises one or more of the three or more image sensors not in the first subset of image sensors, are time-synchronized with each other but not time-synchronized with the first subset of image sensors.

11. The system of claim 1 , wherein generating the three-dimensional reconstruction of the object comprises:

identifying a distortion during the generation process; and

modifying, using the plurality of images of the object generated when each of the at least two projectors illuminate the object with its associated optical pattern from at least the first and second perspectives, the distortion.

12. The system of claim 11 , wherein:

identifying the distortion comprises identifying the distortion in one or more images of the first perspective; and

modifying the distortion comprises modifying the distortion based on one or more images of the second perspective.

13. The system of claim 1 , wherein generating the three-dimensional reconstruction of the object comprises:

selecting, from the received plurality of images, one or more image points having a high confidence value, wherein the confidence value is determined based on an oversaturation value, a correlation score with one or more other image points, and/or a modulation value; and

generating, based on the selected image points, the three-dimensional reconstruction of the object.

14. The system of claim 1 , wherein the processor is configured to perform the acts of:

prior to the plurality of images being captured, generating a first instruction for the first projector of the at least two projectors to illuminate the object with an associated optical pattern; and

subsequent to the plurality of images being received, generating a second instruction for the first projector of the at least two projectors to cease illuminating the object with the associated optical pattern.

15. The system of claim 1 , wherein at least one of the at least two image sensors comprises a spectral filter adapted to a projector to block environmental light outside the spectral range of the at least two projectors.

16. The system of claim 15 , wherein the spectral filter supports multiple spectral bands to support the at least two projectors operating at different wavelengths.

17. A method for generating a three-dimensional reconstruction of an object with reduced distortion, comprising:

using a processor to perform the acts of:

receive, from each of at least two image sensors, for each projector of at least two projectors, a plurality of images of an object illuminated with an associated optical pattern, wherein the at least two projectors illuminate the object with the associated optical pattern, wherein:

a first projector of the at least two projectors illuminates the object from a first perspective,

a second projector of the at least two projectors illuminates the object from a second perspective that is different from the first perspective, and

an angular separation of the first perspective and the second perspective is larger than a half-angle of a reflectance cone of a surface of the object; and

generate, from the received plurality of images, a three-dimensional reconstruction of the object, wherein the three-dimensional reconstruction has reduced distortion due to the received plurality of images of the object being generated when each of the at least two projectors illuminates the object with an associated optical pattern from at least the first and second perspectives.

18. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform the acts of:

receive, from each of at least two image sensors, for each projector of at least two projectors, a plurality of images of an object illuminated with an associated optical pattern, wherein the at least two projectors illuminate the object with the associated optical pattern, wherein:

a first projector of the at least two projectors illuminates the object from a first perspective,

a second projector of the at least two projectors illuminates the object from a second perspective that is different from the first perspective, and

an angular separation of the first perspective and the second perspective is larger than a half-angle of a reflectance cone of a surface of the object; and

generate, from the received plurality of images, a three-dimensional reconstruction of the object, wherein the three-dimensional reconstruction has reduced distortion due to the received plurality of images of the object being generated when each of the at least two projectors illuminates the object with an associated optical pattern from at least the first and second perspectives.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE FAMILY NAME OF THE SECOND NAMED INVENTOR PREVIOUSLY RECORDED ON REEL 050217 FRAME 0728. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 29, 2020
From: SCHAFFER, MARTIN; GROSSE, MARCUS
To: COGNEX CORPORATION
Reel/Frame 053080/0775 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2019
From: SCHAFFER, MARTIN; GROBE, MARCUS
To: COGNEX CORPORATION
Reel/Frame 050217/0728 →
Continuity (2)
Provisional Application 62725082 · Aug 30, 2018
Related Publication 20200074698A1 · Mar 5, 2020