IP Library Granted Patent US 9,305,345
Granted Patent B2
US 9,305,345 · App. 14/260,624 · Granted Apr 5, 2016

System and method for image based inspection of an object

Inventors: Ser Nam Lim (Niskayuna, NY); Jose Abiel Garza (McAllen, TX); David Scott Diwinsky (Cincinnati, OH); Li Guan (Clifton Park, NY); Shubao Liu (Niskayuna, NY); Xingwei Yang (Bellevue, WA); Jens Rittscher (Ballston Lake, NY)
Assignee: General Electric Company
G06T7/0004G06K9/52G06T7/001G06T7/0018G06T7/0032G06T7/0046G06T17/00H04N17/00G01N2021/8896G06T2207/30164
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,305,345
App. No.
14/260,624
Granted
Apr 5, 2016
Kind
B2
Abstract

A method for image based inspection of an object includes receiving an image of an object from an image capture device, wherein the image includes a representation of the object with mil-level precision. The method further includes projecting a measurement feature of the object from the image onto a three-dimensional (3D) model of the object based on a final projection matrix; determining a difference between the projected measurement feature and an existing measurement feature on the 3D model; and sending a notification including the difference between the projected measurement feature and the existing measurement feature.

Claims (40)

1. A method comprising:

receiving an image of an object from an image capture device, wherein the image includes a representation of the object with mil-level precision;

projecting a measurement feature of the object from the image onto a three-dimensional (3D) model of the object based on a final projection matrix;

determining a difference between the projected measurement feature and an existing measurement feature on the 3D model;

sending a notification including the difference between the projected measurement feature and the existing measurement feature;

projecting a calibration feature of the object from the 3D model onto the image based on an initial projection matrix;

determining a projection error between the projected calibration feature and an existing calibration feature on the image; and

calculating the final projection matrix based on the projection error.

2. The method of claim 1 , wherein determining the projection error further comprises determining at least one of a distance error and an angular error between the projected calibration feature and the existing calibration feature.

3. The method of claim 2 , further comprising determining a weighting ratio between the distance error and the angular error.

4. The method of claim 1 , wherein the initial projection matrix includes an internal calibration matrix, an initial rotational matrix, and an initial translational matrix.

5. The method of claim 4 , wherein the final projection matrix includes the internal calibration, a final rotational matrix, and a final translational matrix.

6. The method of claim 1 , further comprising:

determining whether the difference between the projected measurement feature and the existing measurement feature exceeds a discrepancy threshold; and

sending the notification to a user in response to determining that the difference exceeds the discrepancy threshold.

7. A system comprising:

at least one processor;

a communication module stored in a memory and executable by the at least one processor, the communication module configured to receive an image of an object from an image capture device, wherein the image includes a representation of the object with mil-level precision;

a back-projection module stored in the memory and executable by the at least one processor, the back-projection module communicatively coupled with the communication module and configured to project a measurement feature of the object from the image onto a three-dimensional (3D) model of the object based on a final projection matrix and determine a difference between the projected measurement feature and an existing measurement feature on the 3D model;

a notification module stored in the memory and executable by the at least one processor, the notification module communicatively coupled with the back-projection module and configured to send a notification including the difference between the projected measurement feature and the existing measurement feature; and

a projection module configured to project a calibration feature of the Object from the 3D model onto the image based on an initial projection matrix and determine a projection error between the projected calibration feature and an existing calibration feature on the image.

8. The system of claim 7 , further comprises a matrix correction module configured to calculate the final projection matrix based on the projection error.

9. The system of claim 8 , wherein the matrix correction module is further configured to calculate the final projection matrix based on at least one of a distance error and an angular error between the projected calibration feature, and the existing calibration feature.

10. The system of claim 9 , wherein the matrix correction module is further configured to calculate the final projection matrix based on a weighting ratio between the distance error and the angular error.

11. The system of claim 7 , wherein the initial projection matrix includes an internal calibration matrix, an initial rotational matrix, and an initial translational matrix.

12. The system of claim 11 , wherein the final projection matrix includes the internal calibration matrix, a final rotational matrix, and a final translational matrix.

13. The system of claim 7 , wherein the notification module is further configured to determine whether the difference between the projected measurement feature and the existing measurement feature exceeds a discrepancy threshold and send the notification to a user in response to determining that the difference exceeds the discrepancy threshold.

14. A computer program product comprising a non-transitory computer readable physical storage device encoding instructions that, in response to execution by at least one processor, cause the processor to perform operations comprising:

receive an image of an object from an image capture device, wherein the image includes a representation of the object with mil-level precision;

project a measurement feature of the object from the image onto a three-dimensional (3D) model of the object based on a final projection matrix;

determine a difference between the projected measurement feature and an existing measurement feature on the 3D model;

send a notification including the difference between the projected measurement feature and the existing measurement feature;

project a calibration feature of the object from the 3D model onto the image based on an initial projection matrix;

determine a projection error between the projected calibration feature and an existing calibration feature on the image; and

calculate the final projection matrix based on the projection error.

15. The computer program product of claim 14 , further causing, the processor to determine at least one of a distance error and an angular error between the projected calibration feature and the existing calibration feature.

16. The computer program product of claim 15 , further causing the processor to determine a weighting ratio between the distance error and the angular error.

17. The computer program product of claim 14 , further causing the processor to perform operations comprising:

determine whether the difference between the projected measurement feature and the existing measurement feature exceeds a discrepancy threshold; and

send the notification to a user in response to determining that the difference exceeds the discrepancy threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 24, 2014
From: LIM, SER NAM; GARZA, JOSE ABIEL; DIWINSKY, DAVID SCOTT; GUAN, LI; LIU, SHUBAO; YANG, XINGWEI; RITTSCHER, JENS
To: GENERAL ELECTRIC COMPANY
Reel/Frame 032748/0716 →
Continuity (1)
Related Publication 20150310604A1 · Oct 29, 2015