Object movement behavior learning
In various examples, a set of object trajectories may be determined based at least in part on sensor data representative of a field of view of a sensor. The set of object trajectories may be applied to a long short-term memory (LSTM) network to train the LSTM network. An expected object trajectory for an object in the field of view of the sensor may be computed by the LSTM network based at least in part an observed object trajectory. By comparing the observed object trajectory to the expected object trajectory, a determination may be made that the observed object trajectory is indicative of an anomaly.
1 . A method comprising:
determining an object trajectory including at least one series of first positions of one or more objects in an environment;
determining, using one or more portions of the object trajectory, a plurality of predicted object trajectories associated with the object trajectory, the plurality of predicted object trajectories computed using at least one machine learning model;
identifying, for respective trajectories of the plurality of predicted object trajectories, respective deviations from the object trajectory based at least on differences between a series of positions along the object trajectory and at least two other series of positions along the plurality of predicted object trajectories;
determining an anomaly between the object trajectory and the respective trajectories based at least on a value aggregated from individual contributions of the respective deviations for the respective trajectories exceeding a threshold; and
generating one or more notifications indicative of the anomaly.
2 . The method of claim 1 , wherein at least one first predicted trajectory of the plurality of predicted object trajectories corresponds to at least one of a first weather condition or a first lighting condition and at least one second predicted trajectory of the plurality of predicted object trajectories corresponds to at least one of a second weather condition or a second lighting condition.
3 . The method of claim 1 , comprising:
selecting, for the determining of the anomaly, the respective trajectories as a subset of the plurality of predicted object trajectories based at least on magnitudes of deviation scores corresponding to the respective deviations; and
computing the value as a weighted average of the respective deviations for the respective trajectories.
4 . The method of claim 1 , comprising computing the value as a statistical combination of the respective deviations, and comparing the value to the threshold to detect the anomaly.
5 . The method of claim 1 , wherein the determining of the plurality of predicted object trajectories includes providing the one or more portions of the object trajectory as input to the at least one machine learning model, and the at least one machine learning model infers at least one of the plurality of predicted object trajectories from the one or more portions.
6 . The method of claim 1 , wherein the at least one machine learning model includes a long short-term memory (LSTM) network trained using object trajectories observed over time and indications of environment attributes including at least one of weather conditions or lighting conditions associated with the object trajectory to predict data indicating the plurality of predicted object trajectories.
7 . The method of claim 1 , wherein the plurality of predicted object trajectories are determined by:
applying the object trajectory as an input to the at least one machine learning model; and
obtaining, as output from the at least one machine learning model, data indicating at least one predicted object position corresponding to at least one predicted object trajectory of the plurality of predicted object trajectories.
8 . The method of claim 1 , wherein the plurality of predicted object trajectories are determined using a plurality of environmental attributes, and the plurality of predicted object trajectories include trajectory templates corresponding to different respective sets of the plurality of environmental attributes.
9 . The method of claim 1 , wherein the determining the object trajectory includes determining, based at least on sensor data obtained using at least one sensor statically positioned relative to a roadway and representative of a field of view of at least a portion of the roadway, the object trajectory corresponding to at least one object of the one or more objects traveling through the portion of the roadway.
10 . A system comprising:
one or more processing units to perform operations including:
determining an object trajectory including at least one series of first positions of one or more objects in an environment;
determining, using one or more portions of the object trajectory, a plurality of predicted object trajectories associated with the object trajectory, the plurality of predicted object trajectories computed using at least one machine learning model;
comparing the object trajectory to the plurality of predicted object trajectories to identify, for respective trajectories of the plurality of predicted object trajectories, respective deviations between the object trajectory and the respective trajectories;
determining an anomaly between the object trajectory and the respective trajectories based at least on a value aggregated from individual contributions of the respective deviations for the respective trajectories exceeding a threshold; and
generating one or more notifications indicative of the anomaly.
11 . The system of claim 10 , wherein the plurality of predicted object trajectories are computed based at least on applying at least a portion of the object trajectory as input to the at least one machine learning model, and the at least one machine learning model generates output corresponding to one or more predictions of one or more portions of at least one of the plurality of predicted object trajectories.
12 . The system of claim 11 , wherein the plurality of predicted object trajectories are computed based at least on applying, to the at least one machine learning model, data indicative of one or more weather conditions or one or more lighting conditions associated with the object trajectory.
13 . The system of claim 11 , further comprising determining, using the at least one machine learning model, one or more trajectory features including at least one of at least one traffic speed, at least one traffic volume, or at least one traffic flow rate, wherein the determining of the anomaly is further based at least on the one or more trajectory features.
14 . The system of claim 11 , wherein the at least one machine learning model includes a long short-term memory (LSTM) network trained using object trajectories observed over time to predict data indicating the plurality of predicted object trajectories.
15 . At least one processor comprising:
one or more circuits to generate one or more notifications based at least on:
determining, using one or more portions of an object trajectory, a plurality of predicted object trajectories associated with the object trajectory, the plurality of predicted object trajectories computed using at least one machine learning model,
identifying, for respective trajectories of the plurality of predicted object trajectories, respective deviations from the object trajectory based at least on differences between the object trajectory and the respective trajectories, and
determining an anomaly between the object trajectory and the respective trajectories based at least on a value aggregated from individual contributions of the respective deviations for the respective trajectories exceeding a threshold.
16 . The at least one processor of claim 15 , wherein the plurality of predicted object trajectories are computed based at least on at least one of one or more weather conditions or one or more lighting conditions associated with the object trajectory.
17 . The at least one processor of claim 15 , wherein the one or more circuits are further to:
determine, using the at least one machine learning model, one or more trajectory features including at least one of at least one traffic speed, at least one traffic volume, or at least one traffic flow rate; and
determine that at least one object trajectory is indicative of the anomaly based at least on the one or more trajectory features.
18 . The at least one processor of claim 15 , wherein the object trajectory is associated with environmental attributes, and the plurality of predicted object trajectories correspond to respective environmental attributes of the environmental attributes, the environmental attributes including at least one of a weather condition, a lighting condition, or a traffic condition.
19 . The at least one processor of claim 15 , wherein the one or more circuits are further to determine the object trajectory based at least on:
detecting one or more objects in image data generated using one or more cameras; and
determining object positions of at least one series of object positions at a plurality of times based at least on the image data.
20 . The at least one processor of claim 15 , wherein the at least one machine learning model includes a long short-term memory (LSTM) network trained using object trajectories observed over time to predict data indicating the plurality of predicted object trajectories.