Identifying temporal changes of industrial objects by matching images
Technology for matching images (for example, video images, still images) of an identical infrastructure object (for example, a tower component of a tower supporting power lines) for purposes of comparing the infrastructure object to itself at different points in time to detect a potential anomaly and the potential need for maintenance of the infrastructure object. In some embodiments, this matching of images is done using creation of a three dimensional (#D) computer model of the infrastructure object and by tagging captured images with location on the 3D model across multiple videos taken at different points in time.
1. A computer-implemented method comprising:
receiving a first video image that includes a first initial version infrastructure object image showing a first infrastructure object with the first initial version infrastructure object image being characterized by a first viewing vector;
receiving a second video image that includes a second initial version infrastructure object image showing the first infrastructure object with the second initial version infrastructure object image being characterized by a second viewing vector that is at least approximately parallel to the first viewing vector;
selecting the first initial version infrastructure image from the first video;
analyzing, by machine logic, the second video image to determine that the second initial version infrastructure object image is a match with an identical instance of the first initial version infrastructure object image, with the analysis of the second video image including:
constructing, by machine logic, a set of three dimensional (3D) data model(s) of at least a portion of the environment around the first infrastructure object based, at least in part, upon the first and second video images; and
determining the match based, at least in part, upon the respective relationships of the first and second initial infrastructure object images to the set of 3D data model(s); and
determining, by machine logic, that a potential maintenance condition exists regarding the first infrastructure object based, at least in part, upon the first and second initial version infrastructure object images.
2. The method of claim 1 further comprising:
responsive to the determination of the existence of the potential maintenance condition, sending out, automatically and by machine logic, a notification of the potential maintenance condition.
3. The method of claim 1 wherein the set of 3D data model(s) include:
a first 3D data model constructed based upon spatial data of the first video image; and
a second 3D data model constructed based upon spatial data of the second video image.
4. The method of claim 3 wherein the determination of the match based, at least in part, upon the respective relationships of the first and second initial infrastructure object images to the 3D data model includes:
for at least some given images of the first video including the first initial version infrastructure object image, tagging the given image with data indicative of a location within the 3D model shown by the given image;
for at least some given images of the second video including the second initial version infrastructure object image, tagging the given image with tagging information indicative of a location within the 3D model shown by the given image; and
comparing the tagging information to determine the match between the first initial version infrastructure object image from the first video image and the second initial version infrastructure object image from the second video image.
5. The method of claim 4 wherein the tagging information is in the form of 3D coordinates given with respect to a frame of reference defined by the 3D model.
6. The method of claim 3 wherein the first infrastructure object is a tower component.
7. The method of claim 6 wherein the portion of the environment is a power line support tower.
8. The method of claim 1 further comprising:
capturing the first video image by unmanned aerial vehicle(s) equipped with video camera(s); and
capturing the second video image by unmanned aerial vehicle(s) equipped with video camera(s).
9. A computer-implemented method comprising:
receiving a first video image that includes a first initial version infrastructure object image showing a first infrastructure object with the first initial version infrastructure object image being characterized by a first viewing vector;
receiving a second video image that includes a second initial version infrastructure object image showing the first infrastructure object with the second initial version infrastructure object image being characterized by a second viewing vector that is at least approximately parallel to the first viewing vector;
selecting the first initial version infrastructure image from the first video;
analyzing, by machine logic, the second video image to determine that the second initial version infrastructure object image is a match with an identical instance of the first initial version infrastructure object image, with the analysis of the second video image including:
constructing, by machine logic, a three dimensional (3D) data model of at least a portion of the environment around the first infrastructure object based, at least in part, upon the first and second video images; and
determining the match based, at least in part, upon the respective relationships of the first and second initial infrastructure object images to the 3D data model;
adjusting, by machine logic, at least one of the first and second initial version infrastructure object image to obtain a plurality of adjusted infrastructure image objects respectively corresponding to the first and second initial version infrastructure object images, with the plurality of adjusted infrastructure object images showing the first infrastructure object aligned with itself across the plurality of adjusted infrastructure object images;
comparing, by machine logic, the adjusted infrastructure object images with each other to determine a difference data set corresponding to a set of differences between at least two of the plurality of adjusted infrastructure object images; and
analyzing, by machine logic, the difference data set to determine that a potential maintenance condition exists regarding the first infrastructure object.
10. The method of claim 9 further comprising:
responsive to the determination of the existence of the potential maintenance condition, sending out, automatically and by machine logic, a notification of the potential maintenance condition.
11. The method of claim 9 wherein the set of 3D data model(s) include:
a first 3D data model constructed based upon spatial data of the first video image; and
a second 3D data model constructed based upon spatial data of the second video image.
12. The method of claim 9 wherein the adjustment of the at least one initial version infrastructure object images to obtain a plurality of adjusted infrastructure object images includes:
translating, by machine logic, in the first initial version infrastructure object image to provide translational alignment of the first infrastructure object as respectively shown in the plurality of adjusted infrastructure images.
13. The method of claim 9 wherein the adjustment of the at least one initial version infrastructure object images to obtain a plurality of adjusted infrastructure object images includes:
adjusting, by machine logic, an angular orientation of an image plane of the first initial version infrastructure object image for angular alignment of the first infrastructure object as respectively shown in the plurality of adjusted infrastructure images; and
translating, by machine logic, in the first initial version infrastructure object image to provide translational alignment of the first infrastructure object as respectively shown in the plurality of adjusted infrastructure images.