Technology configured to enable fault detection and condition assessment of underground stormwater and sewer pipes
The present disclosure relates to technology configured to enable fault detection and condition assessment of underground stormwater and sewer pipes. Embodiments of the present disclosure have been developed to allow automated processing of video captured by pipe inspection robots and the like thereby to identify and categorize artefacts in pipes.
1 . A computer implemented method configured to facilitate maintenance of pipe infrastructure for a pipe infrastructure network, the method including:
performing a data ingestion and processing method for a plurality of locations within the pipe infrastructure network, wherein each data ingestion and processing method includes:
receiving input video data defining a pipe inspection video collected via pipe inspection infrastructure for a defined pipe at a defined location within the pipe infrastructure network;
for each frame in a subset of frames defined in the video data:
(i) processing the frame via one or more image classification models, thereby to identify presence of one or more artefacts belonging to a defined set of pipe condition artefact classifications;
(ii) determining position data associated with the frame; and
(iii) defining, for each identified artefact, an artefact data set representative of a combination of a position and an artefact classification;
executing a duplication identification process configured to identify artefact data sets having combinations of position and an artefact classification from distinct frames that are predicted to represent a common real-world artefact in the pipe; and
outputting pipe condition data derived from the deduplicated artefact data;
compiling the pipe condition data for the plurality of locations within the pipe infrastructure network into a pipe network artefact data repository; and
executing processing in respect of the pipe network artefact data repository thereby to autonomously define maintenance facilitation data including one or more of the following:
(i) a report which includes prioritization of recommended pipe maintenance remediation actions across the pipe infrastructure network;
(ii) a report which includes budgeting information of recommended pipe maintenance remediation actions across the pipe infrastructure network; and
(iii) a report which predicts future failure events or timeframes across the pipe infrastructure network.
2 . The method of claim 1 , wherein processing the frame via one or more image classification modules, thereby to identify presence of one or more pipe condition artefacts belonging to a defined set of pipe condition artefact classifications includes:
(i) processing the frame via a primary classification model, thereby to identify one or more artefact categories; and
(ii) for each identified artefact category having an associated secondary classification model, executing the associated secondary classification model thereby to determine an artefact classification.
3 . The method of claim 2 , further comprising, for each identified artefact category without an associated secondary classification model, determining an artefact classification based on the primary classification model.
4 . The method of claim 1 , wherein the defined set of pipe condition artefact classifications includes one or more pipe condition artefact classifications associated with an OCR extraction trigger, and wherein for each identified artefact having a classification associated with the OCR extraction trigger, the method includes performing an OCR extraction of text data from the video frame.
5 . The method of claim 4 , wherein the text data includes a positional measurement.
6 . The method of claim 5 , wherein the positional measurement is used to determine the positioning data associated with the frame.
7 . The method of claim 4 , wherein the text data includes data describing the defined pipe and/or pipe location.
8 . The method of claim 1 , further comprising extracting positional measurement data from a data file associated with the video data, wherein the data file associates video timecodes and/or frames with positional data.
9 . The method of claim 1 , wherein the duplication identification process configured to identify combinations of position and an artefact classification from distinct frames that are predicted to represent a common real-world artefact in the pipe includes:
(i) sorting artefact data sets based on their positional information; and
(ii) identifying as duplicates artefact data sets having (A) a common artefact classification; and
(B) positional information within a threshold range.
10 . The method of claim 1 , wherein the duplication identification process configured to identify combinations of position and an artefact classification from distinct frames that are predicted to represent a common real-world artefact in the pipe includes:
(i) sorting artefact data sets based on their positional information;
(ii) segregating the artefact data sets into blocks of a predefined distance based on the positional information; and
(iii) identifying as duplicates artefact data sets having (A) a common artefact classification; and
(B) segregation into a common block.
11 . The method of claim 10 , wherein the predefined distance is between 100 mm and 500 mm.
12 . The method of claim 1 , wherein the duplication identification process includes a frame-to-frame comparison process that uses: (i) image similarity; (ii) relative capture positions; and (iii) artefact classification data for each frame.
13 . The method of claim 12 , wherein the image similarity uses a structural similarity (SSIM) comparison process.
14 . The method of claim 12 , wherein the frame-to-frame comparison process uses a trained machine learning model.
15 . The method of claim 1 , further comprising enabling accessing by a user of a user interface that is configured to enable a user to manually review and adjust artefact classifications by reference to display of relevant video frames.
16 . The method of claim 1 , further comprising outputting reporting data for the pipe in accordance with a predefined reporting standard.
17 . The method of claim 1 , wherein determining position data associated with a given frame includes performing an OCR extraction of position data superimposed on the frame.
18 . The method of claim 17 , wherein the position data is a relative position based on a start point.
19 . The method of claim 1 , wherein determining position data associated with a given frame includes performing a simultaneous location and mapping (SLAM) process.
20 . The method of claim 1 , wherein determining position data associated with a given frame includes performing a position tracking process based on image processing techniques.
21 . The method of claim 1 , wherein determining position data associated with a given frame includes performing a position tracking process based on an optical flow method.
22 . The method of claim 1 , further comprising
for each frame in the subset or another subset of frames defined in the video data, processing the frame via a computer vision technique thereby to identify a pipe joint; and
for each joint identified in the pipe: determining a water level at the joint.
23 . The method of claim 22 , wherein the processing the frame via a computer vision technique thereby to identify a pipe joint includes applying a computer vision technique configured to identify partial circles.
24 . The method of claim 23 , wherein the computer vision technique configured to identify partial circles includes a Hough transform.
25 . The method of claim 23 , wherein identifying water level at the joint is based upon analysis of attributes of the partial circle.
26 . The method of claim 23 , wherein identifying water level at the joint is based upon analysis of a height of the pipe for which the joint is not visible.
27 . The method of claim 23 , wherein identifying water level at the joint is based upon analysis of a comparison between width of the identified partial circle and height of the identified partial circle, with the difference between width and height representing a height defining predicted water level.
28 . The method of claim 22 , further comprising applying a deduplication process thereby to identify frames showing common joints thereby to enable identification of unique joints.
29 . The method of claim 28 , wherein the deduplication process includes an image similarity comparison process.
30 . The method according to claim 1 wherein the maintenance facilitation data includes two or more of the following:
(i) a report which includes prioritization of recommended pipe maintenance remediation actions across the pipe infrastructure network;
(ii) a report which includes budgeting information of recommended pipe maintenance remediation actions across the pipe infrastructure network; and
(iii) a report which predicts future failure events or timeframes across the pipe infrastructure network.
31 . The method according to claim 1 wherein the maintenance facilitation data includes a report which includes prioritization of recommended pipe maintenance remediation actions across the pipe infrastructure network.