System and methods for automatically validating evidence of traffic violations using automatically detected context features
Disclosed herein are methods and systems for automatically validating evidence of traffic violations. One instance of a method comprises receiving an evidence package comprising video frames showing a vehicle involved in a potential traffic violation. The video frames can be input into one or more deep learning models to obtain a plurality of classification results. The method can further comprise generating a score based in part on the classification results and evaluating the score against one or more thresholds to determine whether the evidence package is automatically approved, is automatically rejected, or requires further review.
1. A method of automatically evaluating evidence of a potential traffic violation, comprising:
receiving, at a server, an evidence package of the potential traffic violation from an edge device, wherein the evidence package comprises one or more event video frames and one or more license plate video frames of videos captured by the edge device showing a vehicle involved in the potential traffic violation;
inputting the one or more event video frames and license plate video frames into one or more deep learning models running on the server to obtain one or more classification results, wherein each of the classification results is associated with one of a plurality of features;
inputting one or more of the classification results and their associated features into a decision tree algorithm to obtain a plurality of contributing scores, wherein each of the contributing scores is associated with one of the plurality of features;
calculating a final score based on the contributing scores; and
evaluating the final score against one or more predetermined thresholds to determine whether the evidence package is automatically approved, is automatically rejected, or requires further review.
2. The method of claim 1 , further comprising inputting the one or more license plate video frames into a license plate classifier running on the server, wherein the classification results comprise confidence scores obtained from the license plate classifier concerning license plate-related features of the vehicle.
3. The method of claim 2 , wherein the license plate classifier comprises a convolutional neural network backbone comprising multiple prediction heads connected to the convolutional neural network backbone.
4. The method of claim 2 , wherein one of the plurality of features is a prediction concerning whether license plate characters on the license plate are arranged in a stacked arrangement, and wherein one of the classification results is a confidence score associated with the prediction concerning whether the license plate characters on the license plate are arranged in the stacked arrangement.
5. The method of claim 1 , wherein the event video frames are captured by an event camera of the edge device coupled to a carrier vehicle while the carrier vehicle is in motion, and wherein the license plate video frames are captured by a license plate recognition (LPR) camera of the edge device coupled to the carrier vehicle while the carrier vehicle is in motion.
6. The method of claim 1 , wherein the final score is calculated by incrementing or decrementing an initial score using the plurality of contributing scores, wherein each of the contributing scores is associated with one of the features, and wherein each of the contributing scores is determined by the decision tree algorithm based on all of the classification results provided as inputs to the decision tree algorithm.
7. The method of claim 1 , wherein the decision tree algorithm is a gradient boosted decision tree algorithm.
8. The method of claim 1 , wherein the one or more predetermined thresholds comprise a first threshold and a second threshold, wherein the first threshold is higher than the second threshold, and further comprising:
automatically approving the evidence package in response to the final score being higher than the first threshold;
marking or tagging the evidence package for further review in response to the final score being between the first threshold and the second threshold; and
automatically rejecting the evidence package in response to the final score being below the second threshold.
9. A system for automatically evaluating evidence of a potential traffic violation, comprising:
an edge device comprising one or more cameras configured to capture videos of a vehicle involved in the potential traffic violation, wherein the edge device comprises one or more processors coupled to a memory, wherein the one or more processors are programmed to generate an evidence package concerning the potential traffic violation, wherein the evidence package comprises one or more event video frames and license plate video frames from the videos captured by the edge device; and
a server communicatively coupled to the edge device, wherein the server comprises one or more server processors programmed to:
receive the evidence package from the edge device,
input the one or more event video frames and license plate video frames into one or more deep learning models running on the server to obtain one or more classification results, wherein each of the classification results is associated with one of a plurality of features,
input one or more of the classification results and their associated features into a decision tree algorithm to obtain a plurality of contributing scores, wherein each of the contributing scores is associated with one of the plurality of features,
calculate a final score based on the contributing scores, and
evaluate the final score against one or more predetermined thresholds to determine whether the evidence package is automatically approved, is automatically rejected, or requires further review.
10. The system of claim 9 , wherein the one or more server processors are further programmed to input the one or more license plate video frames into a license plate classifier running on the server, wherein the classification results comprise confidence scores obtained from the license plate classifier concerning license plate-related features of the vehicle.
11. The system of claim 10 , wherein the license plate classifier comprises a convolutional neural network backbone comprising multiple prediction heads connected to the convolutional neural network backbone.
12. The system of claim 10 , wherein one of the plurality of features is a prediction concerning whether license plate characters on the license plate are arranged in a stacked arrangement, and wherein one of the classification results is a confidence score associated with the prediction concerning whether the license plate characters on the license plate are arranged in the stacked arrangement.
13. The system of claim 9 , wherein the event video frames are captured by an event camera of the edge device coupled to a carrier vehicle while the carrier vehicle is in motion, and wherein the license plate video frames are captured by a license plate recognition (LPR) camera of the edge device coupled to the carrier vehicle while the carrier vehicle is in motion.
14. The system of claim 9 , wherein the one or more server processors are further programmed to calculate the final score by incrementing or decrementing an initial score using the plurality of contributing scores, wherein each of the contributing scores is associated with one of the features, and wherein each of the contributing scores is determined by the decision tree algorithm based on all of the classification results provided as inputs to the decision tree algorithm.
15. The system of claim 9 , wherein the decision tree algorithm is a gradient boosted decision tree algorithm.
16. The system of claim 9 , wherein the one or more predetermined thresholds comprise a first threshold and a second threshold, wherein the first threshold is higher than the second threshold, and wherein the one or more server processors are further programmed to:
automatically approve the evidence package in response to the final score being higher than the first threshold;
mark or tag the evidence package for further review in response to the final score being between the first threshold and the second threshold; and
automatically reject the evidence package in response to the final score being below the second threshold.