System and method for automatic treadwear classification
A system and method are provided for automatically alerting drivers to potential tread ware problems to enable them to avoid the danger hazards associated with worn treads. A tread-evaluation station is placed at a location where images of tires may be captured. Images of tires are recorded when a vehicle is at or near the tread-evaluation station. An automated analysis is performed on the images. Based on the automated analysis, tires depicted in the captured images are classified into categories of wear. The automated analysis may include a detection component trained to detect tires in images, and a classification model trained to assign a classification to the wear status of the tires identified by the detection component. The tread-evaluation station may further be trained to predict when tires that do not currently need replacing will need replacing.
1 . A method comprising:
capturing one or more digital images that include a depiction of a tire, wherein the one or more digital images are captured by one or more cameras incorporated into an electric vehicle charging station as a vehicle that includes the tire approaches or is parked at the electric vehicle charging station, wherein the electric vehicle charging station includes a computing device;
generating, by the computing device of the electric vehicle charging station using an object detection model trained on a first dataset of images containing tires, a representation of a location of the tire for each digital image in the one or more digital images, wherein:
the object detection model comprises a semantic segmentation model; and
the representation of the location of the tire comprises a pixel-wise segmentation mask that highlights the location of the tire in the digital image;
providing, by the computing device of the electric vehicle charging station, the one or more digital images and corresponding representations of the location of the tire to a classification model trained on a second dataset of images consisting of close-up views of tires;
generating, by the computing device of the electric vehicle charging station using the classification model, a classification of the tread of the tire based on the one or more digital images and the corresponding representations of the location of the tire;
determining, by the computing device of the electric vehicle charging station using a trained machine learning engine based at least in part on the classification of the tread of the tire, a predicted remaining life of the tire representing a duration of time until the tread of the tire will be worn;
generating, by the computing device of the electric vehicle charging station, an alert that is based, at least in part, on a classification of the tread of the tire and the predicted remaining life of the tire; and
causing the alert to be displayed on a display associated with the electric vehicle charging station.
2 . The method of claim 1 wherein the semantic segmentation model operates substantially in the same manner as the UNet++ architecture.
3 . The method of claim 1 wherein the classification model operates substantially in the same manner as a trained EfficientNetV2 neural network model.
4 . The method of claim 1 further comprising pre-processing the first dataset by performing one or more augmentation procedures on the one or more images, the one or more augmentation procedures comprising at least one of:
mosaic augmentation,
normalization,
random perspective augmentation, or
colorspace augmentation.
5 . The method of claim 1 further comprising training the classification model to assign one of a predetermined plurality of tread wear classifications to depictions of tires.
6 . The method of claim 5 wherein the classification model uses a convolutional neural network with residual blocks.
7 . The method of claim 1 further comprising transforming at least some images in the first dataset prior to training the classification model, wherein the transforming includes at least one of:
resizing the at least some images;
performing a random horizontal flip of the at least some images;
performing a random vertical flip of the at least some images;
performing a Gaussian blur of the at least some images; or
performing a random crop of the at least some images.
8 . The method of claim 1 further comprising performing one or more transformations on the one or more images prior to providing the one or more digital images to the classification model, wherein the one or more transformations include at least one deterministic transformation and no random transformations.
9 . The method of claim 1 wherein the alert is generated and sent at a time that is based, at least in part, on the predicted remaining life of the tire.
10 . The method of claim 1 , wherein causing the alert to be displayed on a display associated with the electric vehicle charging station comprises causing the alert to be displayed on a mobile app used by a driver of the vehicle to check-in, authenticate, or pay for a charge session at the electric vehicle charging station.
11 . A method comprising:
capturing one or more digital images that include a depiction of a tire, wherein the one or more digital images are captured by one or more cameras incorporated into an electric vehicle charging station as a vehicle that includes the tire approaches or is parked at the electric vehicle charging station, wherein the electric vehicle charging station includes a computing device;
generating, by the computing device of the electric vehicle charging station using an object detection model trained on a first dataset of images containing tires, a representation of a location of the tire for each digital image in the one or more digital images, wherein the representation of the location of the tire comprises a predicted bounding box that highlights a predicted location of the tire in the digital image;
providing, by the computing device of the electric vehicle charging station, the one or more digital images and corresponding representations of the location of the tire to a classification model trained on a second dataset of images consisting of close-up views of tires;
generating, by the computing device of the electric vehicle charging station using the classification model, a classification of the tread of the tire based on the one or more digital images and the corresponding representations of the location of the tire;
determining, by the computing device of the electric vehicle charging station using a trained machine learning engine based at least in part on the classification of the tread of the tire, a predicted remaining life of the tire representing a duration of time until the tread of the tire will be worn;
generating, by the computing device of the electric vehicle charging station, an alert that is based, at least in part, on a classification of the tread of the tire and the predicted remaining life of the tire; and
causing the alert to be displayed on a display associated with the electric vehicle charging station.
12 . The method of claim 11 wherein capturing the one or more digital images is performed by capturing multiple images of the tire over a period of time, the method further comprising:
identifying an inconsistency between predicted bounding boxes, for the tire, determined by the object detection model for two or more consecutive images of the tire; and
dropping the predicted bounding boxes for the two or more consecutive images of the tire.
13 . One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause:
capturing one or more digital images that include a depiction of a tire, wherein the one or more digital images are captured by one or more cameras incorporated into an electric vehicle charging station as a vehicle that includes the tire approaches or is parked at the electric vehicle charging station, wherein the electric vehicle charging station includes a computing device;
generating, by the computing device of the electric vehicle charging station using an object detection model trained on a first dataset of images containing tires, a representation of a location of the tire for each digital image in the one or more digital images, wherein:
the object detection model comprises a semantic segmentation model; and
the representation of the location of the tire comprises a pixel-wise segmentation mask that highlights the location of the tire in the digital image;
providing, by the computing device of the electric vehicle charging station, the one or more digital images and corresponding representations of the location of the tire to a classification model trained on a second dataset of images consisting of close-up views of tires;
generating, by the computing device of the electric vehicle charging station using the classification model, a classification of the tread of the tire based on the one or more digital images and the corresponding representations of the location of the tire;
determining, by the computing device of the electric vehicle charging station using a trained machine learning engine based at least in part on the classification of the tread of the tire, a predicted remaining life of the tire representing a duration of time until the tread of the tire will be worn; and
generating, by the computing device of the electric vehicle charging station, an alert that is based, at least in part, on the classification of the tread of the tire and the predicted remaining life of the tire.
14 . The one or more non-transitory storage media of claim 13 wherein:
capturing the one or more digital images is performed by capturing multiple images of the tire over a period of time; and
the instructions include instructions for:
identifying an inconsistency between predicted bounding boxes, for the tire, determined by the object detection model for two or more consecutive images of the tire; and
dropping the predicted bounding boxes for the two or more consecutive images of the tire.
15 . The one or more non-transitory storage media of claim 13 wherein the alert is generated and sent at a time that is based, at least in part, on the predicted remaining life of the tire.
16 . The one or more non-transitory storage media of claim 13 wherein causing the alert to be displayed on a display associated with the electric vehicle charging station comprises causing the alert to be displayed on a mobile app used by a driver of the vehicle to check-in, authenticate, or pay for a charge session at the electric vehicle charging station.
17 . The one or more non-transitory storage media of claim 13 wherein the instructions, when executed by the one or more computing devices, further cause pre-processing the first dataset by performing one or more augmentation procedures on the one or more images, the one or more augmentation procedures comprising at least one of:
mosaic augmentation,
normalization,
random perspective augmentation, or
colorspace augmentation.
18 . The one or more non-transitory storage media of claim 13 wherein the instructions, when executed by the one or more computing devices, further cause transforming at least some images in the first dataset prior to training the classification model, wherein the transforming includes at least one of:
resizing the at least some images;
performing a random horizontal flip of the at least some images;
performing a random vertical flip of the at least some images;
performing a Gaussian blur of the at least some images; or
performing a random crop of the at least some images.
19 . The one or more non-transitory storage media of claim 13 further comprising performing one or more transformations on the one or more images prior to providing the one or more digital images to the classification model, wherein the one or more transformations include at least one deterministic transformation and no random transformations.