IP Library Granted Patent US 9,483,820
Granted Patent B2
US 9,483,820 · App. 14/282,296 · Granted Nov 1, 2016

Method and system for detecting a damaged component of a machine

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,483,820
App. No.
14/282,296
Granted
Nov 1, 2016
Kind
B2
Abstract

A method and system for detecting a damaged machine component during operation of the machine are provided. The damaged machine component detection system includes one or more processors, one or more memory devices communicatively coupled to the one or more processors, an image capture device configured to generate a stream of temporally-spaced images of a scene including a machine component of interest, the images generated in real-time, and a grouping model configured to compare features of a current image of the stream of images to features of previously captured images sorted into a plurality of groups of images having similar features, generate an alert if predetermined features of the current image deviate from corresponding features of the grouped images by a predetermined amount, and update the groups with the current image if the predetermined features of the current image are similar to corresponding features of the grouped images by a predetermined amount. The system also includes an output device configured to output at least one of the alert and the current image.

Claims (36)

1. A damaged machine component detection system comprising:

a processor;

a memory device communicatively coupled to said processor;

an image capture device configured to generate a stream of temporally-spaced images of a scene including a machine component of interest, the images generated in real-time;

a grouping model configured to:

compare predetermined features of at least a portion of a current image of the stream of images to corresponding features of at least a corresponding portion of previously captured images, the previously captured images sorted into a plurality of groups based on a similarity of the predetermined features in the previously captured images;

generate an alert if the predetermined features of the current image and a predetermined number of successive images deviate from corresponding features of the grouped images by greater than a predetermined amount;

update the groups with the current image if the predetermined features of the current image deviate from the corresponding features of the grouped images by less than a predetermined amount; and

a machine component detector configured to receive the images in real-time and detect the machine component of interest in a plurality of images using a Histogram of Gradients (HoG) analysis.

2. The system of claim 1 , wherein the machine component is a shovel bucket and the machine component detector is further configured to detect at least a portion of the shovel bucket including a tooth line of the shovel bucket.

3. The system of claim 1 , wherein the machine component detector is further configured to perform a multi-scale, multi-orientation, sliding window procedure to detect the location of a tooth line in the image based on a tooth line HoG signature.

4. The system of claim 1 , wherein the machine component detector is further configured to perform a multi-scale, multi-orientation, sliding window procedure to detect the location of a tooth line in the image based on a selectable threshold of the sliding window.

5. The system of claim 1 , wherein the grouping model is further configured to add the current image to a group with previously grouped images that have features similar to the current image.

6. The system of claim 1 , wherein the grouping model is further configured to group the current image with previously grouped images that are organized into groups based on at least one of regions of the respective image where the at least a portion of the tooth line was detected, the image size, and an orientation of the at least a portion of the tooth line.

7. The system of claim 1 , wherein the grouping model is further configured to group the current image with a similarity measure of the previously grouped images.

8. The system of claim 1 , wherein the grouping model is embodied in a clustering engine that operates in at least one of a supervised mode and an unsupervised mode.

9. The system of claim 1 , wherein the grouping model comprises a plurality of individual models.

10. A computer-implemented method of detecting a damaged machine component during operation of the machine, the method implemented using a computer device coupled to a user interface and a memory device, said method comprising:

receiving by the computer device a plurality of temporally-spaced images, each image including at least a portion of a machine component and a background, each image comprising a plurality of pixels;

detecting, in real-time, the at least a portion of the machine component in a current one of the plurality of images using a Histogram of Gradients (HOG) analysis;

comparing the detected at least a portion of the machine component in the current image to a detected at least a portion of the machine component in a previous image stored in the memory device;

outputting an alert notification based on the grouping.

11. The method of claim 10 , wherein detecting the at least a portion of the machine component using a HoG analysis further comprises a multi-scale, multi-orientation, sliding window procedure to detect the location of a tooth line in the image based on a tooth line HoG signature.

12. The method of claim 10 , wherein grouping the current image with previously grouped images comprises adding the current image to a group with previously grouped images based on a similarity of features in the current image to features in images of the group.

13. The method of claim 10 , wherein grouping the current image with previously grouped images comprises grouping the current image into groups based on at least one of regions of the respective image where the at least a portion of the machine component was detected, the image size, and an orientation of the at least a portion of the machine component.

14. The method of claim 10 , wherein grouping the current image with previously grouped images comprises grouping the current image with a similarity measure of the previously grouped images.

15. One or more non-transitory computer-readable storage media having computer-executable instructions embodied thereon, wherein when executed by at least one processor, the computer-executable instructions cause the processor to:

receive, in real-time, a plurality of temporally-spaced images of a scene including a shovel bucket having a plurality of teeth spaced along a tooth line;

detect at least a portion of the shovel bucket including the tooth line in a current one of the plurality of images using a Histogram of Gradients (HOG) analysis;

compare the current image to previous captured images sorted into a plurality of groups of images having similar characteristics;

generate an alert if predetermined characteristics of the current image deviate from corresponding characteristics of the grouped images by less than the predetermined amount;

update the groups with the current image if predetermined characteristics of the current image deviate from corresponding characteristics of the grouped images by less than the predetermined amount; and

output at least one of the alert and an image of the damaged component.

16. The computer-readable storage media of claim 15 , wherein the computer-executable instructions further cause the processor to detect the tooth line using a feature vector analysis.

17. The computer-readable storage media of claim 15 , wherein the computer-executable instructions further cause the processor to add the current image to a group with previously grouped images that have features similar to the current image.

18. The computer-readable storage media of claim 15 , wherein the computer-executable instructions further cause the processor to group the current image with previously grouped images that are organized into groups based on at least one of regions of the respective image where the at least a portion of the tooth line was detected, the image size, and an orientation of the at least a portion of the tooth line.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2018
From: GENERAL ELECTRIC COMPANY
To: GE GLOBAL SOURCING LLC
Reel/Frame 048891/0130 →
CORRECTIVE ASSIGNMENT TO CORRECT THE SPELLING OF THE FOURTH INVENTOR'S NAME PREVIOUSLY RECORDED ON REEL 032931 FRAME 0184. ASSIGNOR(S) HEREBY CONFIRMS THE CORRECT SPELLING OF THE FOURTH INVENTOR'S NAME IS CRAIG RONALD HOFFMANN. Recorded Jun 6, 2014
From: LIM, SER NAM; TU, JILIN; GUAN, LI; HOFFMANN, CRAIG RONALD
To: GENERAL ELECTRIC COMPANY
Reel/Frame 033102/0064 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 20, 2014
From: LIM, SER NAM; TU, JILIN; GUAN, LI; HOFFMAN, CRAIG RONALD
To: GENERAL ELECTRIC COMPANY
Reel/Frame 032931/0184 →