Systems and methods for machine learning-based site-specific threat modeling and threat detection

1. A method comprising: with a surveillance system that is communicatively coupled to a plurality of sensor data sources arranged at locations within a plurality of regions of a site under surveillance:

accessing a threat model, represented as a graph structure, that identifies contextual events classified as threats, the graph structure comprising nodes representing detected entities and edges connecting pairs of nodes, each edge identifying an interaction semantic primitive associated with the respective pair of nodes, wherein contextual events are represented as a path comprising at least one edge between connected pairs nodes of the graph structure;

automatically adding new contextual events to the threat model by performing a machine learning process using historical data that identifies frequency of occurrence for at least one contextual event;

identifying at least one contextual event for a site in real-time by processing:

sensor data generated by the sensor data sources, and

co-occurring contextual data for at least one of the regions;

based on probabilistic threat values assigned to at least one of a node and an edge of the path that represents the contextual event, classifying each identified contextual event as one of a threat and a non-threat by using the threat model in real-time;

responsive to classification of an identified contextual event as a threat, controlling a notification system to generate an alert;

automatically identifying a threat response contextual event that identifies dispatching of emergency response services performed for the alert by processing sensor data generated by the sensor data sources; and

based on the identified threat response contextual event, updating a probabilistic threat value of at least one contextual event represented by the threat model.

2. The method of claim 1 , wherein at least one contextual event represented by the threat model identifies at least one semantic primitive and at least one element of contextual data.

3. The method of claim 2 , wherein identifying at least one contextual event comprises:

processing the sensor data in real-time to generate at least one semantic primitive;

accessing the co-occurring contextual data, the co-occurring data relating to the processed sensor data; and

identifying a path of the graph structure that represents each generated semantic primitive and the accessed co-occurring contextual data, the path representing the at least one contextual event.

4. The method of claim 1 ,

wherein at least one contextual event represented by the threat model identifies at least one contextualized primitive.

5. The method of claim 4 , wherein identifying at least one contextual event comprises:

processing the sensor data in real-time to generate at least one semantic primitive;

accessing the co-occurring contextual data, the co-occurring data relating to the processed sensor data;

transforming at least one generated semantic primitive into a contextualized primitive by using the accessed co-occurring contextual data; and

identifying a path of the graph structure that represents each contextualized primitive, the path representing the at least one contextual event.

6. The method of claim 1 ,

further comprising: generating the co-occurring contextual data from sensor data generated by the plurality of sensor data sources, and storing the contextual data in a contextual data storage,

wherein identifying at least one contextual event comprises accessing the stored co-occurring contextual data.

7. The method of claim 1 , wherein a first pair of detected entities is associated with at least one contextual event represented by the threat model, wherein the first pair of detected entities includes a handheld object and a person.

8. The method of claim 1 , wherein at least one contextual event represented by the threat model identifies at least one interaction semantic primitive that identifies an interaction with at least one contextualized primitive.

9. The method of claim 1 , further comprising removing a contextual event from the threat model.

10. The method of claim 1 , wherein identifying at least one contextual event comprises: extracting high-level features from the sensor data by using a high-level feature detection model, and processing the high-level features by using each sub-model of a machine learning ensemble to generate a plurality of semantic primitives.

11. The method of claim 10 , further comprising: training the high-level feature extraction model based on output of each sub-model of the machine learning ensemble.

12. A surveillance system comprising:

a plurality of sensor data sources arranged at locations within a plurality of regions of a site under surveillance;

at least one processor; and

at least one computer-readable storage medium that includes instructions, that when executed by the at least one processor, control the at least one processor to:

access a threat model, represented as a graph structure, that identifies contextual events classified as threats, the graph structure comprising nodes representing detected entities and edges connecting pairs of nodes, each edge identifying an interaction semantic primitive associated with the respective pair of nodes, wherein contextual events are represented as a path comprising at least one edge between connected pairs of nodes of the graph structure,

automatically add new contextual events to the threat model by performing a machine learning process using historical data that identifies frequency of occurrence for at least one contextual event,

generate a plurality of semantic primitives in real-time by processing sensor data generated by the plurality of sensor data sources arranged at locations within the plurality of regions of the site,

identify at least one contextual event for the site in real-time by processing: the semantic primitives and elements of contextual data that are co-occurring with the semantic primitives, and

based on probabilistic threat values assigned to at least one of a node and an edge of the path that represents the contextual event, classify each identified contextual event as one of a threat and a non-threat by using the threat model in real-time,

responsive to classification of an identified contextual event as a threat, control a notification system to generate an alert,

automatically identify a threat response contextual event that identifies dispatching of emergency response services performed for the alert by processing sensor data generated by the sensor data sources, and

based on the identified threat response contextual event, update a probabilistic threat value of at least one contextual event represented by the threat model.

13. The system of claim 12 wherein identifying at least one contextual event comprises:

transforming at least one generated semantic primitive into a contextualized primitive by using the co-occurring contextual data; and

identifying a path of the graph structure that represents each contextualized primitive, the path representing the at least one contextual event.

14. The system of claim 13 , wherein generating a plurality of semantic primitives comprises: extracting high-level features from the sensor data by using the high-level feature detection model, and processing the high-level features by using each sub-model of the machine learning ensemble to generate the plurality of semantic primitives.