Granted Patent B2

US 12,682,448 · App. 18/624,667 · Granted Jul 14, 2026

Inventors: Zheng Yi Wu (Watertown, CT); Dian Mo (Watertown, CT); Peng Xiao (Watertown, CT)

Assignee: Bentley Systems, Incorporated

G06T7/001G06F18/22G06V10/25G06V10/751G06V20/48G06T2200/24G06T2207/10016G06T2207/20021G06T2207/30108

Abstract

Claims (75)

1 . A method for non-contact structural health monitoring, comprising:

obtaining, by a structural health monitoring software application executing on one or more computing devices, a video of a structure captured by a video camera;

selecting an area of interest in frames of the video;

dividing the area of interest into a grid of cells;

selecting at least a template matching algorithm that treats each cell of the grid of cells as a template from a set of multiple machine vision algorithms provided by the structural health monitoring application, wherein the set of multiple machine vision algorithms includes at least one phase-based algorithm and at least one template matching algorithm;

applying the template matching algorithm to the frames of the video of the structure to determine a displacement of each cell by:

defining templates in an initial frame of the video,

searching for each template from the initial frame in subsequent frames of the video by sliding the template, comparing the template to an image patch at each possible location, and generating a similarity map based on the comparing, and

determining a best-matching image patch for each template using the similarity map and calculating displacement based on location of the template in the initial frame and the location of the best-matching image patch in the subsequent frame;

detecting defects or damage based on differences in the displacement of cells; and

displaying, by the structural health monitoring software application, an indicator of the detected defects or damage in a user interface on a screen of the one or more computing devices.

2 . The method of claim 1 , wherein the displacement of each cell has a direction and the method further comprises:

displaying, by the structural health monitoring software application, an indicator of the direction of the displacement of each cell in the user interface on the screen.

3 . The method of claim 1 , wherein the displacement of each cell is determined for a period of time encompassed by the video and the method further comprises:

displaying, by the structural health monitoring software application, an indicator of the displacement of one or more cells verses time in the user interface on the screen.

4 . The method of claim 1 , wherein the applying further comprises:

selecting a region of interest (ROI) for each template, and

wherein the searching for each template is confined to be within the ROI of the respective template.

5 . The method of claim 1 , wherein the selecting also selects a phase-based algorithm to have at least two selected machine vision algorithms, and the applying applies the at least two selected machine vision algorithms to determine the displacement of each cell.

6 . The method of claim 5 , wherein the displaying the indicator of the detected defects or damage comprises:

displaying a first indicator based on the at least one phase-based algorithm and displaying a second indicator based on the at least one template matching algorithm to permit comparison of results of the at least one phase-based algorithm and the least one template matching algorithm.

7 . A method for non-contact structural health monitoring, comprising:

obtaining, by a structural health monitoring software application executing on one or more computing devices, a video of a structure captured by a video camera;

selecting an area of interest in frames of the video;

dividing the area of interest into a grid of cells;

selecting at least a phase-based algorithm from a set of multiple machine vision algorithms provided by the structural health monitoring application;

applying the phase-based algorithm to the frames of the video of the structure to determine a displacement of each cell by:

selecting interested points in an initial frame of the video,

selecting an averaging area for each interested point,

calculating local phases of each interested point over subsequent frames of the video, and

tracking phase change for the interested points within respective neighborhoods;

detecting defects or damage based on differences in the displacement of cells; and

displaying, by the structural health monitoring software application, an indicator of the detected defects or damage in a user interface on a screen of the one or more computing devices.

8 . The method of claim 7 , wherein the displacement of each cell has a direction and the method further comprises:

displaying, by the structural health monitoring software application, an indicator of the direction of the displacement of each cell in the user interface on the screen.

9 . The method of claim 7 , wherein the displacement of each cell is determined for a period of time encompassed by the video and the method further comprises:

displaying, by the structural health monitoring software application, an indicator of the displacement of one or more cells verses time in the user interface on the screen.

10 . The method of claim 7 , wherein the selecting also selects a template matching algorithm to have at least two selected machine vision algorithms, and the applying applies the at least two selected machine vision algorithms to determine the displacement of each cell.

11 . The method of claim 10 , wherein the displaying the indicator of the detected defects or damage comprises:

displaying a first indicator based on the at least one phase-based algorithm and displaying a second indicator based on the at least one template matching algorithm to permit comparison of results of the at least one phase-based algorithm and the least one template matching algorithm.

12 . A computing device, comprising:

a processor; and

a memory coupled to the processor and configured to store instructions of a structural health monitoring software application that when executed on the processor are operable to:

obtain a video of a structure captured by a video camera;

select an area of interest in frames of the video;

divide the area of interest into a grid of cells;

select a template matching algorithm that treats each cell of the grid of cells as a template from a set of multiple machine vision algorithms provided by the structural health monitoring application;

apply the template matching algorithm to the frames of the video of the structure to determine a displacement of each cell, wherein the instructions of the structural health monitoring software application to apply the template matching algorithm are operable to:

define templates in an initial frame of the video;

search for each template from the initial frame in subsequent frames of the video by sliding the template, comparing the template to an image patch at each possible location, and generating a similarity map based on the comparing, and

determine a best-matching image patch for each template using the similarity map and calculate displacement based on location of the template in the initial frame and the location of the best-matching image patch in the subsequent frame;

detect defects or damage based on differences in the displacement of cells, wherein a defect is detected in response to cells within the area of interest having displacements in different directions; and

display an indicator of the detected defects or damage in a user interface on a screen of the one or more computing devices.

13 . The computing device of claim 12 , wherein the instructions of the structural health monitoring software application when executed are further operable to:

display an indicator of a direction of the displacement of each cell in the user interface on the screen.

14 . The computing device of claim 12 , wherein the displacement of each cell is determined for a period of time encompassed by the video and the instructions of the structural health monitoring software application when executed are further operable to:

display an indicator of the displacement of one or more cells verses time in the user interface on the screen.

15 . A computing device, comprising:

a processor; and

a memory coupled to the processor and configured to store instructions of a structural health monitoring software application that when executed on the processor are operable to:

obtain a video of a structure captured by a video camera;

select an area of interest in frames of the video;

divide the area of interest into a grid of cells;

select a phase-based algorithm from a set of multiple machine vision algorithms provided by the structural health monitoring application;

apply the phase-based algorithm to the frames of the video of the structure to determine a displacement of each cell, wherein the instructions of the structural health monitoring software application to apply the phase-based algorithm are operable to:

select interested points in an initial frame of the video;

select an averaging area for each interested point;

calculate local phases of each interested point over subsequent frames of the video, and

track phase change for the interested points within respective neighborhoods;

detect defects or damage based on differences in the displacement of cells, wherein a defect is detected in response to cells within the area of interest having displacements in different directions; and

display an indicator of the detected defects or damage in a user interface on a screen of the one or more computing devices.

16 . The computing device of claim 15 , wherein the instructions of the structural health monitoring software application when executed are further operable to:

display an indicator of a direction of the displacement of each cell in the user interface on the screen.

17 . The computing device of claim 15 , wherein the displacement of each cell is determined for a period of time encompassed by the video and the instructions of the structural health monitoring software application when executed are further operable to:

display an indicator of the displacement of one or more cells verses time in the user interface on the screen.

Assignments (2)

Continuity (2)

References Cited (62)