Systems and methods for feature-based forward collision detection
Disclosed herein are systems and methods for tracking objects to detect a potential collision with a vehicle. A computing device can receive a first frame and a second frame captured by a capture device mounted to the vehicle. The computing device can then identify first points of interest corresponding to an object identified in the first frame and second points of interest corresponding to the object identified in the second frame. The computing device can then generate a transformation matrix that specifies a transformation of the first points of interest to the second points of interest. The computing device can then determine, based on the transformation matrix, a score corresponding to a potential collision event, and transmit a collision signal to a collision alert device responsive to the score satisfying a potential collision event threshold.
1 . A method for more consistently tracking objects through frames of a capture device to detect a potential collision with a vehicle, the method comprising:
receiving, by one or more processors coupled to memory, a first frame and a second frame captured by a capture device mounted to the vehicle, wherein the second frame is captured after the first frame;
estimating, by the one or more processors and using a neural network regression model that receives a grid of pixels of the first frame and a grid of pixels of the second frame, a first bounding box around a detected object in the first frame and a second bounding box around the detected object in the second frame;
identifying, by the one or more processors, a first plurality of points of interest within the first bounding box corresponding to the detected object identified in the first frame, and a second plurality of points of interest within the second bounding box corresponding to the detected object identified in the second frame;
generating, by the one or more processors, a transformation matrix that specifies a transformation of the first plurality of points of interest to a second subset of the second plurality of points of interest;
determining, by the one or more processors and based on the transformation matrix, a score corresponding to a potential collision event represented by the first frame and the second frame; and
activating, by the one or more processors, a collision signal to a collision alert device for an output on a communication interface to a driver of the vehicle responsive to the score satisfying a potential collision event threshold.
2 . The method of claim 1 , wherein identifying the first plurality of points of interest corresponding to the object identified in the first frame further comprises calculating, by the one or more processors, a plurality of Harris corners for the first frame based on pixels in the first frame.
3 . The method of claim 1 , wherein generating the transformation matrix further comprises:
detecting, by the one or more processors, a first region of the first frame corresponding to the object;
selecting, by the one or more processors, a first subset of the first plurality of points of interest within the first region; and
calculating, by the one or more processors, the transformation matrix based on the first subset.
4 . The method of claim 3 , further comprising:
selecting, by the one or more processors, a second subset of the second plurality of points of interest within a second region of the second frame; and
calculating, by the one or more processors, a change in scale between the first frame and the second frame based on a displacement between the first subset and the second subset.
5 . The method of claim 1 , wherein the score comprises an estimated time-to-collision, and wherein determining the score corresponding to the potential collision event further comprises calculating, by the one or more processors, the estimated time-to-collision based on a scaling factor of the transformation matrix.
6 . The method of claim 5 , further comprising determining, by the one or more processors, that the score satisfies the potential collision event threshold by determining that the estimated time-to-collision is less than the potential collision event threshold.
7 . The method of claim 1 , further comprising:
receiving, by the one or more processors, a third frame captured by the capture device;
generating, by the one or more processors, a second transformation matrix based on the second frame and the third frame; and
updating, by the one or more processors, the score based on the second transformation matrix.
8 . The method of claim 7 , further comprising:
determining, by the one or more processors, a first class of the object as depicted in the second frame and a second class of the object as depicted in the third frame;
comparing, by the one or more processors, the first class of the object as depicted in the second frame to the second class of the object as depicted in the third frame; and
storing, by the one or more processors, an indication of a detection failure event in association with a timestamp corresponding to when the second frame was captured by the capture device.
9 . The method of claim 1 , further comprising:
receiving, by the one or more processors, a third frame captured by the capture device;
determining, by the one or more processors, that calculating a second transformation matrix between the second frame and the third frame has failed; and
storing, by the one or more processors, an indication of a detection failure event in association with a timestamp corresponding to when the second frame was captured by the capture device.
10 . The method of claim 1 , wherein identifying the second plurality of points of interest further comprises executing a Kanade-Lucas-Tomasi (KLT) feature tracking technique based on the first plurality of points of interest.
11 . A system for more consistently tracking objects through frames of a capture device to detect a potential collision with a vehicle, the system comprising one or more processors coupled to a non-transitory memory, the one or more processors configured to:
receive a first frame and a second frame captured by a capture device mounted to the vehicle, wherein the second frame is captured after the first frame;
estimate, using a neural network regression model that receives a grid of pixels of the first frame and a grid of pixels of the second frame, a first bounding box around a detected object in the first frame and a second bounding box around the detected object in the second frame;
identify a first plurality of points of interest within the first bounding box corresponding to the detected object identified in the first frame, and a second plurality of points of interest within the second bounding box corresponding to the detected object identified in the second frame;
generate a transformation matrix that specifies a transformation of the first plurality of points of interest to align with a second subset of the second plurality of points of interest;
determine, based on the transformation matrix, a score corresponding to a potential collision event represented by the first frame and the second frame; and
activate a collision signal to a collision alert device for an output on a communication interface to a driver of the vehicle responsive to the score satisfying a potential collision event threshold.
12 . A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to:
receive a first frame and a second frame captured by a capture device mounted to the vehicle, wherein the second frame is captured after the first frame;
estimate, using a neural network regression model that receives a grid of pixels of the first frame and a grid of pixels of the second frame, a first bounding box around a detected object in the first frame and a second bounding box around the detected object in the second frame;
identify a first plurality of points of interest within the first bounding box corresponding to the detected object identified in the first frame, and a second plurality of points of interest within the second bounding box corresponding to the detected object identified in the second frame;
generate a transformation matrix that specifies a transformation of the first plurality of points of interest to align with a second subset of the second plurality of points of interest;
determine, based on the transformation matrix, a score corresponding to a potential collision event represented by the first frame and the second frame; and
activate a collision signal to a collision alert device for an output on a communication interface to a driver of the vehicle responsive to the score satisfying a potential collision event threshold.