IP Library Granted Patent US 11,495,009
Granted Patent B2
US 11,495,009 · App. 16/948,883 · Granted Nov 8, 2022

Track feature detection using machine vision

Inventor: Javier Fernandez (Columbia, SC)
Assignee: HARSCO TECHNOLOGIES LLC
G06V10/56B61L23/042B61L23/045G06V10/225G06V10/751G06V20/56H04N7/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,495,009
App. No.
16/948,883
Granted
Nov 8, 2022
Kind
B2
Abstract

The present disclosure generally relates to automated detection of railroad track features. Images of a railroad track are captured and analyzed to identify track features such as anchors, spikes, rail ties, tie plates, and joints. Various image processing techniques are utilized to accurately distinguish between track features and other objects in the captured images. Track features identified in the images are assigned identifiers and locations and stored in a database so that a status and/or condition of the track features may be monitored for maintenance purposes.

Claims (69)

1. A railroad track feature detection system comprising:

a camera configured to capture a plurality of images,

one or more light sources; and

a computing apparatus comprising:

at least one memory comprising instructions; and

at least one processing device configured to execute the instructions, wherein the instructions cause the at least one processing device to perform operations comprising:

receiving a first image of the plurality of images;

detecting one or more imaged rail features in the first image;

calculating a confidence score for each detected rail feature in the first image; and

based on the confidence score of a first set of detected rail features being above a feature threshold, identifying the detected rail features in the first set and assigning feature identifiers to each of the identified rail features;

identifying a second set of the detected rail features as unwanted objects and filtering out the unwanted objects.

2. The railroad track feature detection system of claim 1 , wherein the camera and the one or more light sources are coupled to a frame of a rail vehicle and positioned such that the plurality of images include a rail track.

3. The railroad track feature detection system of claim 1 , wherein the feature identifiers includes:

feature identification information that corresponds to a respective railroad track feature of a database of railroad track features; and

location information.

4. The railroad track feature detection system of claim 3 , wherein the database of railroad track features includes an anchor, a spike, a rail tie, a tie plate, and a rail joint.

5. The railroad track feature detection system of claim 1 , wherein the operations comprise:

determining at least one color value of each pixel in the first image;

detecting one or more colored areas in the first image; and

calculating a confidence score for each of the detected color areas in the first image; and

based on the confidence score being above a color threshold, identifying the detected color areas as a color marker and assigning a color identifier to the color markers.

6. The railroad track feature detection system of claim 5 , wherein the color identifiers include:

color identification information that corresponds to a respective railroad track feature of a database of railroad track features; and

location information.

7. The railroad track feature detection system of claim 1 , wherein the operations comprise storing the feature identifiers of the identified rail features in the first image.

8. The railroad track feature detection system of claim 7 , wherein the operations comprise:

receiving a second image of the plurality of images;

detecting one or more imaged rail features in the second image;

calculate a confidence score for each detected rail feature in the second image; and

based on the confidence score being above the feature threshold, identifying the detected rail features of the second image and assigning feature identifiers to each of the identified rail features of the second image.

9. The railroad track feature detection system of claim 8 , wherein the operations comprise storing, in the at least one memory, the feature identifiers of the identified rail features in the second image.

10. The railroad track feature detection system of claim 1 , wherein the operations comprise performing a pattern detection technique to detect one or more imaged rail features in the first image.

11. The railroad track feature detection system of claim 10 , wherein the pattern detection technique includes weighting one or more image characteristics.

12. The railroad track feature detection system of claim 1 , wherein the operations comprise:

receiving a second image of the plurality of images;

detecting one or more imaged rail features in the second image;

identifying at least one of the second set of the detected rail features in the second image and filtering out the unwanted objects in the second image.

13. A non-transitory computer readable medium comprising instructions that, when executed by a processor, causes the processor to:

receive a plurality of images from a camera coupled to a frame of a rail vehicle, the camera positioned such that the plurality of images contain a rail track;

detect a plurality of imaged rail features in the plurality of images;

calculate a confidence score for each detected rail feature; and

based on the confidence score of the detected rail features being above a feature threshold, identify the detected rail features and assign feature identifiers to each of the identified rail features

identify a second set of the detected rail features and classifying the second set of the detected rail features as unwanted objects; and

ignore the unwanted objects.

14. The non-transitory computer readable medium of claim 13 , wherein the instructions further cause the processor to:

weight a plurality of image characteristics within the plurality of images; and

based on the weighted image characteristics, perform a pattern detection on the plurality of images to detect the plurality of imaged rail features.

15. The non-transitory computer readable medium of claim 13 , wherein the feature identifiers include:

feature identification information that corresponds to one railroad track feature of a database of railroad track features; and

location information.

16. The non-transitory computer readable medium of claim 15 , wherein the instructions further cause the processor to:

measure a distance between a first identified rail track feature and a second identified rail track feature; and

wherein:

the first identified rail track feature is contained in a first image of the plurality of images; and

the second identified rail track feature is contained in a second image of the plurality of images.

17. The non-transitory computer readable medium of claim 15 , wherein the database of railroad track features includes an anchor, a spike, a rail tie, a tie plate, and a rail joint.

18. The non-transitory computer readable medium of claim 13 , wherein the instructions further cause the processor to:

determine at least one color value of each pixel in the plurality of images;

detect one or more colored areas in the plurality of images; and

calculate a confidence score for each of the detected color areas; and

based on the confidence score being above a color threshold, identify the detected color areas as color markers and assign color identifiers to the identified color markers.

19. The non-transitory computer readable medium of claim 18 , wherein the color identifiers include:

color identification information that corresponds one railroad track features of a database of railroad track features; and

location information.

20. The non-transitory computer readable medium of claim 19 , wherein the instructions further cause the processor to:

measure a distance between a first color marker and a second color marker; and

wherein:

the first color marker is contained in a first image of the plurality of images; and

the second color marker is contained in a second image of the plurality of images.

Assignments (3)
SECURITY INTEREST Recorded Aug 13, 2021
From: HARSCO RAIL, LLC
To: CITIBANK, N.A.
Reel/Frame 057183/0765 →
INTELLECTUAL PROPERTY SECURITY INTEREST ASSIGNMENT AGREEMENT Recorded Aug 13, 2021
From: CITIBANK, N.A.
To: BANK OF AMERICA, N.A.
Reel/Frame 057184/0064 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2020
From: FERNANDEZ, JAVIER
To: HARSCO TECHNOLOGIES LLC
Reel/Frame 053971/0948 →