Framework for Augmented Machine Decision Making

1 . A method comprising:

receiving sensor data;

classifying the sensor data into a first class by at least requesting processing of a machine computational component, receiving a first result of the machine computation component, requesting processing of an agent computation component, and receiving a first result of the agent computation component, the agent computation component including a platform to query an agent;

classifying the sensor data into a second class by at least requesting processing of the machine computational component, receiving a second result of the machine computation component, requesting processing of the agent computation component, and receiving a second result of the agent computation component,

applying a set of rules to the first class and the second class to enable a determination of a composite classification; and

providing the composite result.

2 . The method of claim 1 , wherein processing of the agent computation component is requested when a confidence of the machine computation component result is below a first threshold.

3 . The method of claim 2 , wherein processing of the agent computational component is requested when the confidence of the machine computation component result is above a second threshold.

4 . The method of claim 1 , wherein the set of rules includes matching sensor data within a predetermined time-window.

5 . The method of claim 1 , wherein the providing includes requesting further processing of the machine computation component result by the agent computation component.

6 . The method of claim 1 , wherein the machine computation component includes a deep learning artificial intelligence classifier.

7 . The method of claim 6 , wherein the machine computation component detects objects and classifies objects in the sensor data, the sensor data including an image.

8 . The method of claim 1 , wherein at least one of the receiving, classifying, and providing is performed by at least one data processor forming part of at least one computing system.

9 . The method of claim 1 , wherein the sensor data includes a first image of a first security system asset and a second image of a second security system asset

10 . A method comprising:

receiving first sensor data of a first security system asset and second sensor data of a second security system asset;

accessing a first predefined modality associated with the first security system asset and a second predefined modality associated with the second security system asset, the first modality defining a first computational task for analyzing the received first sensor data, the second modality defining a second computational task for analyzing the received second sensor data;

instantiating a first solution state machine object and a second solution state machine object, the first solution state machine object having a plurality of states and rules for transitioning between the plurality of state, the plurality of states including an initial state, a first intermediate state, a second intermediate state, and a terminal state;

determining a result of the first task and a result of the second task by executing the first task using the first solution state machine object and the second task using the second solution state machine object, the executing including requesting processing of the first task by a machine computation component and an agent computation component;

determining a composite result by applying a set of rules to the result of the first task and the result of the second task; and

providing the composite result.

11 . The method of claim 10 , wherein the executing of the first task includes:

requesting processing of the first task by, and receiving a result of, the machine computation component when a current state of the first solution state machine object is the first intermediate state, the result received from the machine computation component including a first confidence measure;

requesting processing of the first task by, and receiving a result of, the agent computation component when the current state of the first solution state machine object is the second intermediate state, the result received from the agent computation component including a second confidence measure; and

transitioning the current state of the first solution state machine object according to the transition rules and at least one of: the first confidence measure and the second confidence measure.

12 . The method of claim 10 , wherein the machine computation component executes a machine learning algorithm to perform the task.

13 . The method of claim 10 , wherein the machine computation component includes a deep learning neural network or a convolutional neural network.

14 . The method of claim 10 , wherein the agent computation component includes a platform that queries at least one agent, receives a query result, determines a confidence measure of the agent, and determines the second confidence measure using the confidence measure of the queried agent.

15 . The method of claim 10 , wherein the sensor data includes an image including a single image, a series of images, or a video; and the computational task includes: detecting a pattern in the image; detecting a presence of an object within the image; detecting a presence of a person within the image; detecting intrusion of the object or person within a region of the image; detecting suspicious behavior of the person within the image; detecting an activity of the person within the image; detecting an object carried by the person, detecting a trajectory of the object or the person in the image; a status of the object or person in the image; identifying whether a person who is detected is on a watch list; determining whether a person or object has loitered for a certain amount of time; detecting interaction among person or objects; tracking a person or object; determining status of a scene or environment; determining the sentiment of one or more people; counting the number of objects or people; determining whether a person appears to be lost; determining whether an event is normal or abnormal; and/or

determining whether text matches that in a database.

16 . The method of claim 10 , wherein the security system asset is an imaging device, a video camera, a still camera, a radar imaging device, a microphone, a chemical sensor, an acoustic sensor, a radiation sensor, a thermal sensor, a pressure sensor, a force sensor, or a proximity sensor.

17 . The method of claim 10 , wherein the modality defines: solution state machine object attributes, acceptable confidence for reaching the terminal state, a set of assets that trigger the modality, and agent query structure.

18 . The method of claim 10 , wherein executing the task includes posting, via a messaging queuing protocol, requested processing tasks, and wherein the machine computation component and agent computation component are microservices operating on tasks posted via the messaging queue protocol.

19 . The method of claim 10 , further comprising:

modifying a predictive model of the machine computation component using the result received from the agent computation component as a supervisory signal and the received sensor data as input.

20 . The method of claim 10 , wherein at least one of the receiving, accessing, instantiating, executing, and providing is performed by at least one data processor forming part of at least one computing system.

21 . A non-transitory computer program product which, when executed by at least one data processor forming part of at least one computer, result in operations comprising:

receiving sensor data;

classifying the sensor data into a first class by at least requesting processing of a machine computational component, receiving a first result of the machine computation component, requesting processing of an agent computation component, and receiving a first result of the agent computation component, the agent computation component including a platform to query an agent;

classifying the sensor data into a second class by at least requesting processing of the machine computational component, receiving a second result of the machine computation component, requesting processing of the agent computation component, and receiving a second result of the agent computation component,

applying a set of rules to the first class and the second class to enable a determination of a composite classification; and

providing the composite result.

22 . The computer program product of claim 21 , wherein processing of the agent computation component is requested when a confidence of the machine computation component result is below a first threshold.

23 . The computer program product of claim 22 , wherein processing of the agent computational component is requested when the confidence of the machine computation component result is above a second threshold.

24 . The computer program product of claim 21 , wherein the set of rules includes matching sensor data within a predetermined time-window.

25 . The computer program product of claim 21 , wherein the providing includes requesting further processing of the machine computation component result by the agent computation component.

26 . The computer program product of claim 21 , wherein the machine computation component includes a deep learning artificial intelligence classifier.

27 . A system comprising:

a media processor that receives sensor data;

means for classifying the sensor data into a first class by at least requesting processing of a machine computational component, receiving a first result of the machine computation component, requesting processing of an agent computation component, and receiving a first result of the agent computation component, the agent computation component including a platform to query an agent;

means for classifying the sensor data into a second class by at least requesting processing of the machine computational component, receiving a second result of the machine computation component, requesting processing of the agent computation component, and receiving a second result of the agent computation component; and

means for applying a set of rules to the first class and the second class to enable a determination of a composite classification.

28 . The system of claim 27 , wherein the set of rules includes matching sensor data within a predetermined time-window.

29 . The system of claim 27 , wherein the sensor data includes a first image of a first security system asset and a second image of a second security system asset.