Smart road surface detection method and edge collection device, cloud-based road surface recognition module and system thereof

1 . A non-transitory computer-readable recording medium having recorded thereon a cloud-based road surface recognition module for installation within a cloud server, the road surface recognition module comprising software code executable by at least one processor to identify and store pavement condition index (PCI) eigenvalues of roads with poor pavement conditions separately based on data of PCI characteristics in a bounding box of a road surface captured from video streaming and a corresponding inertial attitude thereof measured by a front inertial sensor installed at a front of a vehicle and a rear inertial sensor installed at a rear of the vehicle,

wherein associations are created between the PCI eigenvalues, front inertial attitude data, and rear inertial attitude data, each front inertial attitude value in the data having a corresponding rear inertial attitude value measured concurrently therewith.

2 . The recording medium as claimed in claim 1 ,

wherein the road surface recognition module identifies and stores the PCI eigenvalues of roads with poor pavement conditions from a plurality of streamed videos individually based on metadata of roads with poor pavement conditions.

3 . The recording medium as claimed in claim 2 ,

wherein the road surface recognition module adapts a supervised deep learning model framework and uses a “You Only Look Once” (YOLO) computation method to identify data of the PCI eigenvalues of roads with poor pavement conditions from a plurality of training streamed videos individually, and assigns the corresponding front inertial attitude and rear inertial attitude thereof as tag files of a training sample for supervised learning.

4 . The recording medium as claimed in claim 3 ,

wherein the road surface recognition module is trained and tested by performing, in sequential order: a data preprocessing step, a machine learning model development and training step, a model evaluation step, and a prediction step, and

wherein a sample set is split in the data preprocessing step, placing 75% of training samples within a training data set and 25% of training samples within a testing data set.

5 . The recording medium as claimed in claim 4 , wherein the tag files of the training samples further comprise GPS speed data.

6 . The recording medium as claimed in claim 1 , wherein the PCI eigenvalues comprise 19 distress types based on the definitions of distress items specified in ASTM D6433-11, including alligator cracking, bleeding, block cracking, bumps and sags, corrugation, depressions, edge cracking, joint reflections cracking, lane/shoulder drop off, longitudinal and transversal cracking, patching and utility cut patching, polished aggregate, potholes, railroad crossing, rutting, shoving, slippage cracking, swell, and weathering and raveling.

7 . The recording medium as claimed in claim 1 , wherein the PCI eigenvalues are identified based at least in part on at least one characteristic waveform storing a front inertial attitude value together with the corresponding rear inertial attitude value.

8 . A smart road surface detection method comprising:

using at least one edge mobile device to collect a plurality of streamed videos of roadways with poor pavements and corresponding data of a front inertial attitude, a rear inertial attitude, and GPS data thereof, the front inertial attitude measured by a front inertial sensor installed at a front of a vehicle, the rear inertial attitude measured by a rear inertial sensor installed at a rear of the vehicle;

conducting a preliminary recognition process on the plurality of collected streamed videos separately in order to select a pavement condition index (PCI) characteristic from bounding boxes of a road surface;

organizing the plurality of collected streamed videos and the corresponding inertial attitude, rear inertial attitude, and GPS data thereof into a plurality of metadata to be uploaded and stored in a cloud server, wherein each front inertial attitude value in the metadata has a corresponding rear inertial attitude value measured concurrently therewith;

by a road surface recognition module executing on the cloud server:

identifying and storing PCI eigenvalues of at least one of the plurality of collected streamed videos based on a plurality of testing streamed videos of selected roads with poor pavements and the corresponding front inertial attitude and rear inertial attitude thereof separately, and

deriving surface recognition results for the at least one of the plurality of collected streamed videos from the PCI eigenvalues, front inertial attitude, and rear inertial attitude thereof; and

displaying the derived surface recognition results for the at least one of the plurality of streamed videos in a geographic information system according to the GPS data thereof.

9 . The smart road surface detection method as claimed in claim 8 , wherein the bounding boxes for the PCI characteristics are recognized and created separately by processing, by an object detector, a plurality of streamed videos of roadways, while storing the GPS data, front inertial attitude, and rear inertial attitude thereof in order to compile into a plurality of metadata.

10 . The smart road surface detection method as claimed in claim 8 , wherein the PCI eigenvalues comprise 19 distress types based on the definitions of distress items specified in ASTM D6433-11, including alligator cracking, bleeding, block cracking, bumps and sags, corrugation, depressions, edge cracking, joint reflections cracking, lane/shoulder drop off, longitudinal and transversal cracking, patching and utility cut patching, polished aggregate, potholes, railroad crossing, rutting, shoving, slippage cracking, swell, and weathering and raveling.

11 . A smart road surface detection system configured to perform the smart road surface detection method as claimed in claim 8 , comprising:

an edge mobile device, installed at a front end of a vehicle, wherein the edge mobile device comprises a lens unit, a display unit, a GPS positioning unit, a front inertial measurement unit, a memory unit, and a communication unit, which are electrically connected with a computation processor separately;

an inertial measurement device, installed at a rear end of the vehicle, wherein the inertial measurement device comprises a rear inertial measurement unit and a transmission unit;

an integrated application unit that is pre-stored in the memory unit of the edge mobile device and is called by the computation processor to execute operations of recording poor pavement and boxing the pavement condition index (PCI) characteristics in order to produce a series of streamed videos, wherein the metadata of the streamed videos, including front inertial attitude, rear inertial attitude, and GPS data, are stored therein and transmitted through the edge mobile device, each front inertial attitude value in the metadata having a corresponding rear inertial attitude value measured concurrently therewith; and

a cloud server that receives the streamed videos, front inertial attitude, rear inertial attitude, and GPS data transmitted by the edge mobile device, the cloud server comprising a road surface recognition module executing thereon, wherein the road surface recognition module identifies and stores the PCI eigenvalues of roads with poor pavement conditions based on data of PCI characteristics in a bounding box of a road surface from video streaming and the corresponding inertial attitude thereof separately, information of recognized results being displayed in a geographic information system according to the GPS location thereof.

12 . The smart road surface detection system as claimed in claim 11 , wherein the PCI eigenvalues comprise 19 distress types based on the definitions of distress items specified in ASTM D6433-11, including alligator cracking, bleeding, block cracking, bumps and sags, corrugation, depressions, edge cracking, joint reflections cracking, lane/shoulder drop off, longitudinal and transversal cracking, patching and utility cut patching, polished aggregate, potholes, railroad crossing, rutting, shoving, slippage cracking, swell, and weathering and raveling.

13 . An edge collection device configured to perform the smart road surface detection method as claimed in claim 8 , comprising:

an edge mobile device, installed at a front end of a vehicle, wherein the edge mobile device comprises a lens unit, a display unit, a GPS positioning unit, a front inertial measurement unit, a memory unit, and a communication unit, which are electrically connected with a computation processor separately;

an inertial measurement device, installed at a rear end of the vehicle, wherein the inertial measurement device comprises a rear inertial measurement unit and a transmission unit; and

an integrated application unit that is pre-stored in the memory unit of the edge mobile device and is called by the computation processor to execute operations of recording poor pavement and boxing the PCI characteristics in order to produce a series of streamed videos, wherein the metadata of the streamed videos, including front inertial attitude, rear inertial attitude, and GPS data, are stored therein and uploaded to a cloud server through the edge mobile device, each front inertial attitude value in the metadata having a corresponding rear inertial attitude value measured concurrently therewith.

14 . The edge collection device as claimed in claim 13 , wherein the PCI eigenvalues comprise 19 distress types based on the definitions of distress items specified in ASTM D6433-11, including alligator cracking, bleeding, block cracking, bumps and sags, corrugation, depressions, edge cracking, joint reflections cracking, lane/shoulder drop off, longitudinal and transversal cracking, patching and utility cut patching, polished aggregate, potholes, railroad crossing, rutting, shoving, slippage cracking, swell, and weathering and raveling.