Object behavior anomaly detection using neural networks

1. A method comprising:

determining, based at least in part on sensor data generated using a sensor statically positioned relative to a roadway and representative of a field of view of at least a portion of the roadway, one or more object trajectories corresponding to one or more objects traveling through the portion of the roadway;

applying the one or more object trajectories and an indication of environment attributes including at least one of weather conditions or lighting conditions to a long short-term memory (LSTM) network to train the LSTM network;

computing, by the LSTM network and based at least in part on one or more observed object trajectories and corresponding weather conditions or lighting conditions, a plurality of trajectory templates, wherein each trajectory template corresponds with a different weather condition or lighting condition;

selecting a trajectory template, from the plurality of trajectory templates, based on a particular weather condition or lighting condition associated with a particular object; and

determining that an object trajectory of the particular object associated with the particular weather condition or lighting condition is indicative of an anomaly based at least in part on a comparison of the object trajectory to the selected trajectory template associated with the particular weather condition or lighting condition.

2. The method of claim 1 , wherein the sensor is a camera, and the method further comprises determining the object trajectory of the particular object by:

detecting the particular object in image data generated by the camera; and

determining a corresponding real-world position of the particular object at a plurality of times based at least in part on the image data.

3. The method of claim 1 , further comprising determining the object trajectory of the particular object by determining a position of the particular object over a plurality of times, and wherein the position of the particular object is determined using one or more parameters associated with the sensor.

4. The method of claim 1 , further comprising generating a visualization of the object trajectory overlaid on an image, the visualization including an indication of the anomaly when it is determined that the object trajectory is indicative of the anomaly.

5. The method of claim 1 , wherein the particular object includes at least one of: a vehicle, an animal, or a human.

6. The method of claim 1 , wherein the LSTM network is trained over time to adapt to object behavior observed over time.

7. The method of claim 1 , wherein selection of the trajectory template is based on the particular weather condition associated with the particular object, and further comprising:

selecting a second trajectory template based on the lighting condition associated with the particular object, wherein the trajectory template and the second trajectory template are used to determine that the object trajectory of the particular object is indicative of the anomaly.

8. The method of claim 1 , further comprising:

selecting a second trajectory template based on a time period or a traffic condition associated with the particular object;

generating a first deviation score based on a comparison of the object trajectory of the particular object with the trajectory template;

generating a second deviation score based on a comparison of the object trajectory of the particular object with the second trajectory template;

generating a global deviation score for the object trajectory based on the first deviation score and the second deviation score; and

using the global deviation score to determine that the object trajectory of the particular object is indicative of the anomaly.

9. A system comprising:

one or more processors comprising processing circuitry to:

determine, based at least in part on sensor data generated using a sensor statically positioned relative to a roadway and representative of a field of view of at least a portion of the roadway, one or more object trajectories corresponding to one or more objects traveling through the portion of the roadway;

apply the one or more object trajectories and an indication of environment attributes including at least one of weather conditions or lighting conditions to a long short-term memory (LSTM) network to train the LSTM network;

compute, by the LSTM network and based at least in part on one or more observed object trajectories and corresponding weather conditions or lighting conditions, a plurality of trajectory templates, wherein each trajectory template corresponds with a different weather condition or lighting condition;

select a trajectory template, from the plurality of trajectory templates, based on a particular weather condition or lighting condition associated with a particular object; and

determine that an object trajectory of the particular object associated with the particular weather condition or lighting condition is indicative of an anomaly based at least in part on a comparison of the object trajectory to the selected trajectory template associated with the particular weather condition or lighting condition.

10. The system of claim 9 , wherein the sensor is a camera, and the processing circuitry is further to determine the object trajectory of the particular object by:

detecting the particular object in image data generated by the camera; and

determining a corresponding real-world position of the particular object at a plurality of times based at least in part on the image data.

11. The system of claim 9 , wherein the one or more processors comprising processing circuitry are further configured to determine the object trajectory of the particular object by determining a position of the particular object over a plurality of times, and wherein the position of the particular object is determined using one or more parameters associated with the sensor.

12. The system of claim 9 , wherein the one or more processors comprising processing circuitry are further configured to generate a visualization of the object trajectory overlaid on an image, the visualization including an indication of the anomaly when it is determined that the object trajectory is indicative of the anomaly.

13. The system of claim 9 , wherein the particular object includes at least one of: a vehicle, an animal, or a human.

14. The system of claim 9 , wherein the LSTM network is trained over time to adapt to object behavior observed over time.

15. The system of claim 9 , wherein selection of the trajectory template is based on the particular weather condition associated with the particular object, and wherein the one or more processors comprising processing circuitry are further configured to:

select a second trajectory template based on the lighting condition associated with the particular object, wherein the trajectory template and the second trajectory template are used to determine that the object trajectory of the particular object is indicative of the anomaly.

16. The system of claim 9 , wherein the one or more processors comprising processing circuitry are further configured to:

selecting a second trajectory template based on a time period or a traffic condition associated with the particular object;

generating a first deviation score based on a comparison of the object trajectory of the particular object with the trajectory template;

generating a second deviation score based on a comparison of the object trajectory of the particular object with the second trajectory template;

generating a global deviation score for the object trajectory based on the first deviation score and the second deviation score; and

using the global deviation score to determine that the object trajectory of the particular object is indicative of the anomaly.

17. A processor comprising:

processing circuitry to:

determine, based at least in part on sensor data generated using a sensor statically positioned relative to a roadway and representative of a field of view of at least a portion of the roadway, one or more object trajectories corresponding to one or more objects traveling through the portion of the roadway;

apply the one or more object trajectories and an indication of environment attributes including at least one of weather conditions or lighting conditions to a long short-term memory (LSTM) network to train the LSTM network;

compute, by the LSTM network and based at least in part on one or more observed object trajectories and corresponding weather conditions or lighting conditions, a plurality of trajectory templates, wherein each trajectory template corresponds with a different weather condition or lighting condition;

select a trajectory template, from the plurality of trajectory templates, based on a particular weather condition or lighting condition associated with a particular object; and

determine that an object trajectory of the particular object associated with the particular weather condition or lighting condition is indicative of an anomaly based at least in part on a comparison of the object trajectory to the selected trajectory template associated with the particular weather condition or lighting condition.

18. The processor of claim 17 , wherein the sensor is a camera, and the processing circuitry is further configured to determine the object trajectory of the particular object by:

detecting the particular object in image data generated by the camera; and

determining a corresponding real-world position of the particular object at a plurality of times based at least in part on the image data.

19. The processor of claim 17 , wherein the processing circuitry is further configured to determine the object trajectory of the particular object by determining a position of the particular object over a plurality of times, and wherein the position of the particular object is determined using one or more parameters associated with the sensor.

20. The processor of claim 17 , wherein the processing circuitry is further configured to generate a visualization of the object trajectory overlaid on an image, the visualization including an indication of the anomaly when it is determined that the object trajectory is indicative of the anomaly.