Systems and methods for detecting traffic violations using mobile detection devices
Disclosed are systems and methods for detecting traffic violations using one or more mobile detection devices. Videos captured by one or more mobile detection devices can be processed on the mobile detection devices to extract data and information concerning a potential traffic violation involving a vehicle and a restricted road area. The mobile detection devices can transmit such data and information to a server configured to make a determination as to whether a traffic violation has occurred by comparing the data and information received from the mobile detection devices.
1 . One or more non-transitory computer-readable media comprising instructions stored thereon, that when executed by one or more processors, perform steps comprising:
segmenting frames of one or more videos at a pixel-level, the one or more videos captured by one or more video image sensors, the one or more videos acquired by the one or more video image sensors while the one or more video image sensors are in motion;
associating a semantic label to each pixel of the frames of the one or more videos using a semantic segmentation function from a computer vision library;
constructing a virtual three-dimensional (3D) environment representing a real-world environment captured by the one or more video image sensors;
constructing a 3D semantic annotated map using the virtual 3D environment in combination with two-dimensional maps from public or government mapping databases along with the semantic labels associated with pixels of the frames of the one or more videos, the 3D semantic annotated map thereby being a fusion of mapping data and semantic labels obtained from multiple sources;
identifying a vehicle and a restricted road area from frames of one or more videos by applying a plurality of functions from the computer vision library to the frames and passing the frames to one or more deep learning models;
bounding the vehicle in the frames with a vehicular bounding polygon and bounding the restricted road area in the frames with a road bounding polygon;
verifying the identified restricted road area includes a road area subject to prohibition against occupancy or entry based at least in part on the semantic annotated map; and
responsive to verifying the restricted road area is subject to prohibition against occupancy or entry, detecting that a potential traffic violation has occurred based in part on overlap of the vehicular bounding polygon with the road bounding polygon.
2 . The one or more non-transitory computer-readable media of claim 1 , wherein the restricted road area is a restricted lane of a roadway.
3 . The one or more non-transitory computer-readable media of claim 1 , wherein the one or more non-transitory computer-readable media is housed within a device, wherein the device is coupled to a carrier vehicle, and wherein the one or more videos are captured using the one or more video image sensors of the device when the carrier vehicle is in motion.
4 . The one or more non-transitory computer-readable media of claim 1 , wherein the steps further comprise passing the frames of the one or more videos to the deep learning models to bound the vehicle in the frames with the vehicular bounding polygon and bounding the restricted road area in the frames with the road bounding polygon.
5 . The one or more non-transitory computer-readable media of claim 1 , wherein at least one of the deep learning models comprises a convolutional neural network.
6 . A device for detecting a potential traffic violation, comprising:
one or more video image sensors configured to capture one or more videos of a vehicle and a restricted road area; and
one or more processors programmed to executed instructions to:
segment frames of the one or more videos at a pixel-level, the one or more videos acquired by the one or more video image sensors while the one or more video image sensors are in motion;
associate a semantic label to each pixel of the frames of the one or more videos using a semantic segmentation function from a computer vision library;
construct a virtual three-dimensional (3D) environment representing a real-world environment captured by the one or more video image sensors;
construct a 3D semantic annotated map using the virtual 3D environment in combination with two-dimensional maps from public or government mapping databases along with the semantic labels associated with pixels of the frames of the one or more videos, the 3D semantic annotated map thereby being a fusion of mapping data and semantic labels obtained from multiple sources;
identify the vehicle and the restricted road area from frames of the one or more videos by applying a plurality of functions from the computer vision library to the frames and passing the frames to one or more deep learning models running on the device to identify the vehicle and the restricted road area;
bound the vehicle in the frames with a plurality of vehicular bounding polygon and bound the restricted road area in the frames with a plurality of road bounding polygon;
verifying the identified restricted road area includes a road area subject to prohibition against occupancy or entry based at least in part on semantic annotated map; and
responsive to verifying the restricted road area is subject to prohibition against occupancy or entry, detect that a potential traffic violation has occurred based in part on overlap of the plurality of vehicular bounding polygon with the plurality of road bounding polygon.
7 . The device of claim 6 , wherein the restricted road area is a restricted lane of a roadway.
8 . The device of claim 6 , wherein the device is coupled to a carrier vehicle, and wherein the one or more videos are captured using the one or more video image sensors of the device when the carrier vehicle is in motion.
9 . The device of claim 8 , wherein the one or more deep learning models are running on the device when the carrier vehicle is in motion.
10 . The device of claim 6 , wherein the one or more processors are further programmed to pass the frames of the one or more videos to the one or more deep learning models to bound the vehicle in the frames with the plurality of vehicular bounding polygon and bound the restricted road area in the frames with the plurality of road bounding polygon.
11 . The device of claim 6 , wherein the one or more deep learning models comprise a convolutional neural network.
12 . The device of claim 6 , wherein the one or more processors are further programmed to transmit data obtained from the one or more videos to a server.
13 . A method of detecting a traffic violation, comprising:
segmenting frames of one or more videos at a pixel-level, the one or more videos captured by one or more video image sensors, the one or more videos acquired by the one or more video image sensors while the one or more video image sensors are in motion;
associating a semantic label to each pixel of the frames of the one or more videos using a semantic segmentation function from a computer vision library;
constructing a virtual three-dimensional (3D) environment representing a real-world environment captured by the one or more video image sensors;
constructing a 3D semantic annotated map using the virtual 3D environment in combination with two-dimensional maps from public or government mapping databases along with the semantic labels associated with pixels of the frames of the one or more videos, the 3D semantic annotated map thereby being a fusion of mapping data and semantic labels obtained from multiple sources;
identifying a vehicle and a restricted road area from frames of one or more videos by applying a plurality of functions from the computer vision library to the frames and passing the frames to one or more deep learning models;
bounding the vehicle in the frames with a vehicular bounding polygon and bounding the restricted road area in the frames with a road bounding polygon;
verifying the identified restricted road area includes a road area subject to prohibition against occupancy or entry based at least in part on the semantic annotated map; and
responsive to verifying the restricted road area is subject to prohibition against occupancy or entry, detecting that a potential traffic violation has occurred based in part on overlap of the vehicular bounding polygon with the road bounding polygon.
14 . The method of claim 13 , wherein the restricted road area is a restricted lane of a roadway.
15 . The method of claim 13 , wherein the one or more videos are captured using the one or more video image sensors of a device when the carrier vehicle is in motion.
16 . The method of claim 13 further comprising passing the frames of the one or more videos to the one or more deep learning models to bound the vehicle in the frames with the vehicular bounding polygon and bounding the restricted road area in the frames with the road bounding polygon.
17 . The method of claim 13 , wherein at least one of the one or more deep learning models comprises a convolutional neural network.