Granted Patent B2

US 12,430,733 · App. 17/712,038 · Granted Sep 30, 2025

Inventors: Fikret Necati Catbas (Oviedo, FL); Chuanzhi Dong (Orlando, FL)

Assignee: University of Central Florida Research Foundation

G06T7/0002G06T7/74G06V10/225G06T2207/30184G06V20/54H04N23/90

Abstract

Claims (63)

1. A computer-vision-based method for determining load worthiness, safety, and/or serviceability of a structure, said method comprising:

directing one or more cameras at the structure so as to derive therefrom image data defining a series of images of target portions of the structure;

receiving the image data at a computer system;

applying a computer vision process performed by said computer system to said image data so as to derive location data from the image data, the computer vision process being a feature-extraction process in which location data from the target portions are derived from key points in the images such that said location data corresponds to locations of the target portion in each of said images; determining displacement data from the location data for the target portions by comparison of the location data for each of the target portions with respective original location data for each of the target portions; determining a safety and/or serviceability assessment for said structure from said displacement data, wherein determining of the safety and/or serviceability assessment for said structure includes comparison of the displacements of each of the target portions to an estimated displacement distribution, and determination of a structural problem for said assessment responsive to a determination that the displacement of one or more of the target portions deviates from the estimated displacement distribution by more than a predetermined limit; and outputting a report of the safety and/or serviceability assessment.

2. The computer-vision-based method of claim 1 , wherein the determining of the safety and/or serviceability assessment for said structure includes deriving from the location data velocity data and acceleration data indicative of, respectively, a real-world velocity and a real-world acceleration of the of the target portion.

3. The computer-vision-based method of claim 2 , wherein the safety and/or serviceability determination includes comparing said displacement with a predetermined maximum displacement for said target portion, and

wherein said safety and/or serviceability assessment includes an indication of a potential problem with the structure responsive to a determination that the displacement in greater than said predetermined maximum displacement.

4. The computer-vision-based method of claim 1 , wherein the structure is a bridge having a plurality of longitudinally extending, laterally spaced girders, said target portion being at a longitudinal distance along one of the girders, and

said camera or another camera or cameras providing image data defining a series of images of other target portions of the structure that are each located on a respective one of the other girders at the same longitudinal distance therealong such that the target portions are generally aligned laterally; and

said applying of the computer vision process performed by said computer system derives location data for the other target portions from the image data corresponding to locations of the target portion in each of said images, said location data being indicative of a deflection of the respective girder.

5. The computer-vision-based method of claim 1 , wherein the method further comprises calibrating the computer system such that the computer system has stored accessible thereto calibration data providing for conversion of distances of pixels in the images to real-world distance units.

6. The computer-vision-based method of claim 5 , wherein the images are digital data defining pixels of the images, and the location data is derived from a location of the target portion as a pixel location thereof in the image, said pixel location being converted to real-world units using the calibration data.

7. The computer-vision-based method of claim 1 , wherein the computer vision process includes applying computer vision to locate the target portion in the image by searching only within a predetermined searching area smaller than the image.

8. The computer-vision-based method of claim 1 , wherein the target portion is marked with a pattern that makes said target portion readily visible to the computer vision process.

9. The computer-vision-based method of claim 1 , wherein the target portion is a structural component of the structure that is selected due to visibility thereof to the computer vision process.

10. The computer-vision-based method of claim 1 , wherein the system has at least two of said cameras each producing a respective series of images of respective target portions of the structure at a sequence of equally spaced points in time, and the computer vision process comprises

performing a comparison, for each of the target portions, of a pixel location of the target portion in a first of the images with pixel locations of said target portion in the images later in the series of images and deriving for each of the target portions from said comparison one or more displacement data values as a number of pixels in the associated image.

11. The computer-vision-based method of claim 10 , wherein the method further comprises the computer system

performing a comparison, for each of the target portions, of the pixel location of the target portion in each of the images with the pixel locations of said target portion in the next of said images in the series of images and deriving from said comparison a series of velocity data values as a number of pixels in the associated image; and

converting the velocity data to real-world measurement units using calibration data stored by the computer for each of the cameras defining a scaling of the pixels thereof to real-world measurements of movement of the respective target portion; and

calculating a derivative of said velocity data so as to derive therefrom acceleration data corresponding to acceleration of the target portion in real-world measurements; and

wherein the determining of the safety and/or serviceability assessment for said structure includes comparison for each of the target portions of the acceleration indicated by the acceleration data with a predetermined acceleration limit, and determination of a structural problem responsive to the comparison indicating one or more of the accelerations being greater than the predetermined acceleration limit.

12. The computer-vision-based method of claim 10 , wherein the computer has data indicative of the estimated displacement distribution of the target portions subject to loading of the structure during the period of taking of the images of the target portions.

13. The computer-vision-based method of claim 12 , wherein the structure is a bridge, and the method includes

receiving image data from a further camera positioned corresponding to video of a portion of the bridge on which vehicles travel, and

wherein the estimated displacement distribution is selected based on a detected position of a vehicle or positions of vehicles on the structure.

14. A computer-vision-based method for determining load worthiness, safety or serviceability of a structure, said method comprising:

directing one or more cameras at the structure so as to derive therefrom image data defining a series of images of a target portion of the structure;

receiving the image data at a computer system;

applying a computer vision process performed by said computer system to said image data so as to derive location data from the image data, said location data corresponding to locations of the target portion in each of said images;

determining a safety and/or serviceability assessment for said structure from said location data; and

outputting a report of the safety and/or serviceability assessment;

wherein the structure is a bridge having a plurality of longitudinally extending, laterally spaced girders, said target portion being at a longitudinal distance along one of the girders; and

said camera or another camera or cameras providing image data defining a series of images of other target portions of the structure that are each located on a respective one of the other girders at the same longitudinal distance therealong such that the target portions are generally aligned laterally; and

said applying of the computer vision process performed by said computer system derives location data for the other target portions from the image data corresponding to locations of the target portion in each of said images, said location data being indicative of a deflection of the respective girder;

wherein data is derived for the structure that estimates permissible deflection limits for each of the girders using a model of lateral load distribution in the bridge, and

the safety and/or serviceability assessment includes a notification of a problem with the structure responsive to a determination that the location data indicates that one or more of the one of the girders experienced a deflection that is greater than the respective deflection limit estimated therefor.

15. A computer-vision-based method for determining load worthiness or serviceability of a structure, said method comprising:

directing one or more cameras at the structure so as to derive therefrom image data defining a series of images of a target portion of the structure;

receiving the image data at a computer system;

applying a computer vision process performed by said computer system to said image data so as to derive location data from the image data, said location data corresponding to locations of the target portion in each of said images;

determining a safety and/or serviceability assessment for said structure from said location data; and

outputting a report of the safety and/or serviceability assessment;

wherein the structure is a bridge having a plurality of longitudinally extending, laterally spaced girders, said target portion being at a longitudinal distance along one of the girders, and

said camera or another camera or cameras providing image data defining a series of images of other target portions of the structure that are each located on a respective one of the other girders at the same longitudinal distance therealong such that the target portions are generally aligned laterally; and

said applying of the computer vision process performed by said computer system derives location data for the other target portions from the image data corresponding to locations of the target portion in each of said images, said location data being indicative of a deflection of the respective girder;

wherein a further camera is provided that video images from above the bridge that include images of vehicles passing over the bridge, and said images are transmitted to the computer system, wherein the computer system determines using computer vision presence of a vehicle on the bridge and a lateral lane location of the vehicle, and the lateral lane location is used to determine the permissible deflection limits for each of the girders when said vehicle is on the bridge.

16. The computer-vision-based method of claim 15 , wherein the method further comprises synchronizing the images from all of the cameras the computer system is able to determine locations of the target portions at the same point in time, and using data indicative of the presence and lateral lane location of the vehicle at said point in time.

17. A computer-vision-based system for determining load worthiness or serviceability of a structure, said system comprising

a plurality of cameras each directed at a respective target portion of the structure that moves as the structure experiences loads; and

a computer system having computer accessible data storage associated therewith;

the cameras each communicating with said computer system so as to transmit to the computer system image data corresponding to a respective series of images taken spaced at equal time intervals of the respective target portion, said images each being a field of pixels;

the computer system having stored computer-vision software running on which the computer system processes the image data and identifies based thereon a pixel location of each of the target portions in each of the images;

the computer vision software causing the computer system to process the image data with a feature-extraction process in which pixel locations for the target portions are derived from key points in the images;

the computer system performing a comparison, for each of the target portions, of the pixel location of the target portion in a first of the images with the pixel locations of said target portion in the images later in the series of images and deriving from said comparison one or more displacement data values as a number of pixels in the associated image;

the computer having data indicative of an estimated displacement distribution of the target portions of the structure subject to loading thereof during the period of taking of the images of the target portions; and

an additional camera supported so as to capture serial images of loads moving on said structure and transmitting said images to the computer system;

the computer system receiving said images from the additional camera and using a computer vision process to derive therefrom location of a load or loads on the structure, the data indicative of an estimated displacement distribution being modified or selected based on the location or locations of any load or load derived;

the computer system generating assessment data for the structure based on a comparison of the displacement data for the target portions with the estimated displacement distribution, said assessment data indicating a problem with the structure where any of the displacement data indicates that one or more of the target portions have deviated from the estimated displacement distribution by more than a predetermined threshold difference; and

an output device with which the computer system communicates, said computer system transmitting the assessment data to said output device so as to be displayed to a user.

18. The system according to claim 17 , wherein the system further comprises a synchronizer unit connected with all of the cameras, said synchronizer synchronizing all of the cameras so that the displacements of the target portions are linked time-wise to presence of a load or loads on the structure.

19. The system according to claim 17 , wherein the structure is a bridge, and the loads are vehicles traveling on the bridge, and the target portions are each on a respective longitudinal girder of the bridge, wherein all of the target portions are located at a common longitudinal distance along the girders.

20. The system according to claim 17 , where the structure is a vertically extending building and the displacements are lateral.

Assignments (2)

Continuity (2)

References Cited (41)