Video-based vehicle speed estimation from motion vectors in video streams
View Patent ↗Automated low-complexity video-based vehicle speed estimation is described, that operates within the video stream to screen video sequences to identify and eliminate clear non-violators and/or identify and select potential violators within a multi-layer speed enforcement system, in which deeper layers provide enhanced accuracy on selected candidate (speeding) vehicles. Video motion vector clusters corresponding to a vehicle are identified and tracked across multiple frames of captured video. Movement of the motion vector clusters translated from pixels per second to real speed (e.g. miles per hour) to determine whether the vehicle was speeding. Estimated speed data is added to the video stream data is metadata, and video segments of candidate speeding vehicles are stored and/or transmitted for subsequent review (e.g. automated or manual).
1. A computer-implemented method for estimating vehicle speed from video motion vector information, comprising:
receiving video data of a moving vehicle from a camera;
extracting motion vectors in the video data;
identifying and analyzing at least one motion vector cluster representing the moving vehicle; and
tracking at least one feature of the at least one motion vector cluster across a plurality of video frames; and
estimating frame-to-frame speed and mean speed of the vehicle based at least in part on spatio-temporal coordinates of the at least one tracked feature with the aid of calibration data.
2. The method of claim 1 , wherein the motion vectors are at least one of motion vectors used in the compression process, motion vectors extracted from optical flow data, and motion vectors establishing feature correspondences between frames at different time instants.
3. The method of claim 1 , wherein estimating frame-to-frame speed and mean speed of the vehicle is achieved by a combined measure of vector lengths.
4. The method of 3 , wherein the combined measure of vector lengths is one of a trimmed average, a weighted linear average, an order-statistic, and a mode.
5. The method of claim 1 , wherein the motion vector cluster is regularized to eliminate vectors that are determined to not correspond to the expected vehicle trajectory.
6. The method according to claim 1 , wherein the received video data is compressed video data, and wherein identifying the motion vectors comprises extracting the motion vectors from the compressed video data.
7. The method according to claim 1 , wherein the received video data comprises a live uncompressed video stream, and wherein the motion vectors identified as they are generated during video compression.
8. The method according to claim 1 , wherein when the estimated mean speed of the vehicle exceeds a predetermined vehicle speed threshold, the method further comprises transmitting a video segment comprising video of the vehicle, the video segment having the estimated speed information embedded therein, for vehicle speed confirmation and citation issuance.
9. The method according to claim 1 , wherein when the estimated mean speed of the vehicle falls outside a predetermined vehicle speed range, the method further comprises transmitting a video segment comprising video of the vehicle, the video segment having the estimated speed information embedded therein, for vehicle speed confirmation and citation issuance.
10. The method according to claim 1 , further comprising transmitting one or more segments of compressed video data having estimated speed information embedded therein to a remote location for confirmation of vehicle speed, wherein the one or more transmitted segments comprise compressed video of one or more moving vehicles that are candidates for speeding citations.
11. The method according to claim 1 , further comprising, if estimated mean speed of the vehicle exceeds a predetermined vehicle speed threshold, performing speed estimation on a fully decompressed version of the video data.
12. The method according to claim 1 , wherein the camera is mounted to a stationary structure.
13. The method according to claim 1 , wherein the camera is mounted on a vehicle.
14. A processor configured to execute computer-executable instructions for performing the method of claim 1 , the instructions being stored on a non-transitory computer-readable medium.
15. A system that facilitates estimating vehicle speed from video motion vector information, comprising:
a camera configured to capture video data of one or more moving vehicles;
a non-transitory computer-readable medium that stores computer-executable instructions for execution by a processor; and
a processor configured to execute the instructions, the instructions comprising:
receiving video data of a moving vehicle from the camera;
extracting motion vectors in the video data;
identifying and analyzing at least one motion vector cluster representing the moving vehicle;
tracking at least one feature of the at least one motion vector cluster across a plurality of video frames;
estimating frame-to-frame speed and mean speed of the vehicle based at least in part on spatio-temporal coordinates of the at least one tracked feature with the aid of calibration data;
compressing the video data;
embedding estimated speed information in the compressed video stream; and
storing one or more segments of the compressed video stream with embedded estimated speed information for subsequent review.
16. The system of claim 15 , wherein the motion vectors are at least one of motion vectors used in the compression process, motion vectors extracted from optical flow data, and motion vectors establishing feature correspondences between frames at different time instants.
17. The system according to claim 15 , wherein the received video data comprises a live uncompressed video stream, and wherein the instructions further comprise identifying the motion vectors as they are generated during video compression.
18. The system according to claim 15 , wherein when the estimated mean speed of the vehicle exceeds a predetermined vehicle speed threshold, the instructions further comprise transmitting a video segment comprising video of the vehicle, the video segment having the estimated speed information embedded therein, for vehicle speed confirmation and citation issuance.
19. The system according to claim 15 , the instructions further comprising transmitting one or more segments of compressed video data having estimated speed information embedded therein to a remote location for confirmation of vehicle speed, wherein the one or more transmitted segments comprise compressed video of one or more moving vehicles that are candidates for speeding citations.
20. The system according to claim 15 , the instructions further comprising, if estimated mean speed of the vehicle exceeds a predetermined vehicle speed threshold, performing speed estimation on a fully decompressed version of the video data.
21. The system according to claim 15 , wherein the camera is mounted to one of a stationary structure and a vehicle.
22. The system according to claim 15 , wherein the processor and the computer readable medium are integral to the camera.
23. The system according to claim 15 , wherein when the estimated mean speed of the vehicle falls outside a predetermined vehicle speed range, the instructions further comprise transmitting a video segment comprising video of the vehicle, the video segment having the estimated speed information embedded therein, for vehicle speed confirmation and citation issuance.
24. A non-transitory computer-readable medium having stored thereon computer-executable instructions for estimating vehicle speed from video motion vector information, the instructions comprising:
receiving video data of a moving vehicle from a camera;
extracting motion vectors in the video data;
identifying and analyzing at least one motion vector cluster representing the moving vehicle;
tracking at least one feature of the at least one motion vector cluster across a plurality of video frames;
estimating frame-to-frame speed and mean speed of the vehicle based at least in part on spatio-temporal coordinates of the at least one tracked feature with the aid of calibration data
compressing the video data;
embedding estimated speed information in the compressed video stream; and
storing one or more segments of the compressed video stream with embedded estimated speed information for subsequent review.
25. The computer-readable medium of claim 24 , wherein the motion vectors are at least one of motion vectors used in the compression process, motion vectors extracted from optical flow data, and motion vectors establishing feature correspondences between frames at different time instants.
26. The computer-readable medium according to claim 24 , wherein the received video data comprises a live uncompressed video stream, and wherein the instructions further comprise identifying the motion vectors as they are generated during video compression.