METHODS AND SYSTEMS FOR VEHICLE-BASED TRACKING OF NEARBY EVENTS
The present disclosure generally relates to methods, systems, apparatuses, and non-transitory computer readable media for incident detection using radar sensory data. A vehicular tracking system comprises at least one on-vehicle sensor, such as a radar sensor, that can perceive the environment around the vehicle and capture data related to possible incidents that may be viewed by the sensor. A radar sensor may provide radar data that can be used to calculate velocity vectors, accelerations vectors, azimuth and elevation angles of other vehicles, and this data may be collected and stored for possible incident characterization and accident investigation. The vehicle with the sensor may be configured to behave as a third-party witness to possible incidents. Several sensors may be used in conjunction with one another, where one sensor may trigger another sensor to begin capturing other data that the first sensor may be unable to capture.
1 - 17 . (canceled)
18 . An event recording system, comprising:
a sensor suite configured to collect sensor data about a local environment around a first vehicle, wherein the sensor suite comprises one or more sensors that are each attached to the first vehicle;
a non-transitory memory configured to buffer collected sensor data and storing instructions; and
one or more processors configured to execute the stored instructions to:
analyze the sensor data collected by the sensor suite to detect at least one potential event indicator, wherein the at least one potential event indicator is associated with a first type of event and with a first timestamp; and
responsive to detecting the at least one potential event indicator:
select, from the collected sensor data buffered in the memory, a first portion of sensor data, wherein the first portion of sensor data includes sensor data associated with timestamps within a time window that includes the first timestamp;
generate a potential event detection report, wherein the potential event detection report comprises one or more of: (i) location information indicating a geographic location of the first vehicle, (ii) data identifying the at least one potential event indicator, (iii) timing information indicating a time of occurrence of the at least one potential event indicator, and (iv) environment information comprising data from the first portion of sensor data; and
store the potential event detection report in a portion of the non-transitory memory.
19 . The event recording system of claim 18 , wherein the one or more processors are further configured to analyze the sensor data collected by the sensor suite to detect a plurality of nearby objects and to determine tracking data for the detected plurality of nearby objects, wherein the detected tracing data comprises a plurality of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object.
20 . The event recording system of claim 19 , wherein:
the at least one potential event indicator comprises a first tracking trace associated with a first nearby object and a second tracking trace associated with a second nearby object, wherein:
the position indicated by the first tracking trace is within a first distance of the position indicated by the second tracking trace; and
the position indicated by the first tracking trace is approaching the position indicated by the second tracking with at least a first velocity;
the first type of event is a traffic collision; and
the potential event detection report includes environment information comprising the first tracking trace and the second tracking trace.
21 . The event recording system of claim 19 , wherein:
the sensor suite comprises at least a first multiple-input and multiple-output (MIMO) radar sensor and at least a first image sensor;
the sensor data collected by the sensor suite comprises a first set of sensor data collected by the first MIMO radar sensor and a second set of sensor data collected by the first image sensor; and
analyzing the sensor data to determine tracking data for the detected plurality of nearby objects comprises:
generating a first set of tracking data using the first set of sensor data, the first set of tracking data comprising a first set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object;
generating a second set of tracking data using the second set of sensor data, the second set of tracking data comprising a second set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object; and
fusing the first set of tracking data and the second set of tracking data.
22 . The event recording system of claim 19 , wherein:
the sensor suite comprises at least a first multiple-input and multiple-output (MIMO) radar sensor and at least a first image sensor;
the sensor data collected by the sensor suite comprises a first set of sensor data collected by the first MIMO radar sensor and a second set of sensor data collected by the first image sensor; and
analyzing the sensor data to determine tracking data for the detected plurality of nearby objects comprises generating a first set of tracking data by jointly using the first set of sensor data and the second set of sensor data, the first set of tracking data comprising a first set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object.
23 . The event recording system of claim 18 , wherein the sensor suite comprises:
a first multiple-input and multiple-output (MIMO) radar sensor;
a first lidar sensor;
a first image sensor; and
a first ultrasonic sensor.
24 . The event recording system of claim 19 , wherein the one or more processors are further configured to:
estimate a severity of a potential event associated with the potential event indicator based on the velocity and acceleration indicated by the plurality of tracking traces; and
responsive to the estimated severity being below a predefined threshold, discard the first portion of sensor data.
25 . A method for recording events, comprising:
collecting sensor data about a local environment around a first vehicle, wherein the sensor data is collected by a sensor suite comprised of one or more sensors that are each attached to the first vehicle;
buffering collected sensor data;
analyzing the sensor data collected by the sensor suite to detect at least one potential event indicator, wherein the at least one potential event indicator is associated with a first type of event and with a first timestamp; and
responsive to detecting the at least one potential event indicator:
selecting, from the buffered collected sensor data, a first portion of sensor data, wherein the first portion of sensor data comprises sensor data includes sensor data associated with timestamps within a time window that includes the first timestamp;
generating a potential event detection report, wherein the potential event detection report comprises one or more of: (i) location information indicating a geographic location of the first vehicle, (ii) data identifying the at least one potential event indicator, (iii) timing information indicating a time of occurrence of the at least one potential event indicator, and (iv) environment information comprising data from the first portion of sensor data; and
storing the potential event detection report in a corresponding portion of a data repository.
26 . The method of claim 25 , further comprising analyzing the sensor data collected by the sensor suite to detect a plurality of nearby objects and to determine tracking data for the detected plurality of nearby objects, wherein the detected tracing data comprises a plurality of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object.
27 . The method of claim 26 , wherein:
the at least one potential event indicator comprises a first tracking trace associated with a first nearby object and a second tracking trace associated with a second nearby object, wherein:
the position indicated by the first tracking trace is within a first distance of the position indicated by the second tracking trace; and
the position indicated by the first tracking trace is approaching the position indicated by the second tracking with at least a first velocity;
the first type of event is a traffic collision; and
the potential event detection report includes environment information comprising the first tracking trace and the second tracking trace.
28 . The method of claim 26 , wherein:
the sensor suite comprises at least a first multiple-input and multiple-output (MIMO) radar sensor and at least a first image sensor;
the sensor data collected by the sensor suite comprises a first set of sensor data collected by the first MIMO radar sensor and a second set of sensor data collected by the first image sensor; and
analyzing the sensor data to determine tracking data for the detected plurality of nearby objects comprises:
generating a first set of tracking data using the first set of sensor data, the first set of tracking data comprising a first set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object;
generating a second set of tracking data using the second set of sensor data, the second set of tracking data comprising a second set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object; and
fusing the first set of tracking data and the second set of tracking data.
29 . The method of claim 26 , wherein:
the sensor suite comprises at least a first multiple-input and multiple-output (MIMO) radar sensor and at least a first image sensor;
the sensor data collected by the sensor suite comprises a first set of sensor data collected by the first MIMO radar sensor and a second set of sensor data collected by the first image sensor; and
analyzing the sensor data to determine tracking data for the detected plurality of nearby objects comprises generating a first set of tracking data by jointly using the first set of sensor data and the second set of sensor data, the first set of tracking data comprising a first set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object.
30 . A non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to record nearby events by:
collecting sensor data about a local environment around a first vehicle, wherein the sensor data is collected by a sensor suite comprised of one or more sensors that are each attached to the first vehicle;
buffering collected sensor data;
analyzing the sensor data collected by the sensor suite to detect at least one potential event indicator, wherein the at least one potential event indicator is associated with a first type of event and with a first timestamp; and
responsive to detecting the at least one potential event indicator:
selecting, from the buffered collected sensor data, a first portion of sensor data, wherein the first portion of sensor data includes sensor data associated with timestamps within a time window that includes the first timestamp;
generating a potential event detection report, wherein the potential event detection report comprises one or more of: (i) location information indicating a geographic location of the first vehicle, (ii) data identifying the at least one potential event indicator, (iii) timing information indicating a time of occurrence of the at least one potential event indicator, and (iv) environment information comprising data from the first portion of sensor data; and
storing the potential event detection report in a corresponding portion of a data repository.
31 . The non-transitory computer readable medium of claim 30 , wherein the instructions further cause the processors to record nearby events by further comprising analyzing the sensor data collected by the sensor suite to detect a plurality of nearby objects and to determine tracking data for the detected plurality of nearby objects, wherein the detected tracing data comprises a plurality of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object.
32 . The non-transitory computer readable medium of claim 31 , wherein:
the at least one potential event indicator comprises a first tracking trace associated with a first nearby object and a second tracking trace associated with a second nearby object, wherein:
the position indicated by the first tracking trace is within a first distance of the position indicated by the second tracking trace; and
the position indicated by the first tracking trace is approaching the position indicated by the second tracking with at least a first velocity;
the first type of event is a traffic collision; and
the potential event detection report includes environment information comprising the first tracking trace and the second tracking trace.
33 . The non-transitory computer readable medium of claim 31 , wherein:
the sensor suite comprises at least a first multiple-input and multiple-output (MIMO) radar sensor and at least a first image sensor;
the sensor data collected by the sensor suite comprises a first set of sensor data collected by the first MIMO radar sensor and a second set of sensor data collected by the first image sensor; and
analyzing the sensor data to determine tracking data for the detected plurality of nearby objects comprises:
generating a first set of tracking data using the first set of sensor data, the first set of tracking data comprising a first set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object;
generating a second set of tracking data using the second set of sensor data, the second set of tracking data comprising a second set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object; and
fusing the first set of tracking data and the second set of tracking data.
34 . The non-transitory computer readable medium of claim 31 , wherein:
the sensor suite comprises at least a first multiple-input and multiple-output (MIMO) radar sensor and at least a first image sensor;
the sensor data collected by the sensor suite comprises a first set of sensor data collected by the first MIMO radar sensor and a second set of sensor data collected by the first image sensor; and
analyzing the sensor data to determine tracking data for the detected plurality of nearby objects comprises generating a first set of tracking data by jointly using the first set of sensor data and the second set of sensor data, the first set of tracking data comprising a first set of tracking traces that each indicate, for an associated one of the plurality of nearby objects, a position, velocity, and acceleration of the respective nearby object.
35 . An event tracking system, comprising:
a networking unit configured to receive a plurality of potential event detection reports from a plurality of event recording systems, wherein each of the plurality of potential event detection reports comprises one or more of: (i) location information indicating a geographic location of a first vehicle, (ii) data identifying at least one potential event indicator, (iii) timing information indicating a time of occurrence of the at least one potential event indicator, and (iv) environment information comprising data from a first portion of sensor data, wherein:
the detected potential event indicator is detected by analyzing the sensor data collected by the sensor suite; and
the detected potential event indicator is associated with a first type of event and with a first timestamp;
the first portion of sensor data is selected from a sensor suite configured to collect sensor data about a local environment around the first vehicle; and
the sensor suite comprises one or more sensors that are each attached to the first vehicle; and
the first portion of sensor data comprises sensor data includes sensor data associated with timestamps within a time window that includes the first timestamp;
memory configured to store at least a portion of the received plurality of potential event detection reports; and
one or more processors configured to execute the instructions to:
analyze the received plurality of potential event detection reports to detect an occurrence of at least one tracked event; and
responsive to detecting the occurrence of the at least one tracked event, generating an event record for the at least one tracked event.
36 . The event tracking system of claim 35 , wherein:
the tracked event record comprises location information indicating a geographic location of the at least one tracked event, data identifying a type of at least one tracked event, and scene information comprising data obtained from the environment information of a first set of the received plurality of potential event reports; and
the environment information of each of the first set of potential event detection reports comprises data derived from sensor data collected in a local environment containing the tracked event.
37 . The event tracking system of claim 36 , wherein the one or more processors are further configured to execute the instructions to:
identify, based on the location information of the at least one tracked event, a second vehicle approaching the geographic location of the at least one tracked event; and
transmit a request to the second vehicle to collect supplemental sensor data of the at least one tracked event.