Augmented Machine Decision Making

1 . A method comprising:

receiving sensor data;

classifying the sensor data into one of two or more classes by at least requesting processing of a machine computational component, receiving a result of the machine computation component, requesting processing of an agent computation component, and receiving a result of the agent computation component, the agent computation component including a platform to query an agent; and

providing the result from the agent computation component or the result from the machine computation component.

2 . The method of claim 1 , wherein processing of the agent computation component is requested when a confidence of the machine computation component result is below a first threshold.

3 . The method of claim 2 , wherein processing of the agent computational component is requested when the confidence of the machine computation component result is above a second threshold.

4 . The method of claim 1 , wherein the providing includes requesting further processing of the agent computation component result by the machine computation component.

5 . The method of claim 1 , wherein the providing includes requesting further processing of the machine computation component result by the agent computation component.

6 . The method of claim 1 , wherein the machine computation component includes a deep learning artificial intelligence classifier.

7 . The method of claim 6 , wherein the machine computation component detects objects and classifies objects in the sensor data, the sensor data including an image.

8 . The method of claim 1 , further comprising:

determining a composite result from the machine computation component result and the agent computation component result and using a measure of result confidence.

9 . The method of claim 1 , wherein at least one of the receiving, classifying, and providing is performed by at least one data processor forming part of at least one computing system.

10 . A method comprising:

receiving sensor data of a security system asset;

accessing a predefined modality associated with the security system asset, the modality defining a computational task for analyzing the received sensor data;

instantiating a solution state machine object having a plurality of states and rules for transitioning between the plurality of state, the plurality of states including an initial state, a first intermediate state, a second intermediate state, and a terminal state;

executing the task using the solution state machine object, the executing including:

requesting processing of the task by, and receiving a result of, a machine computation component when a current state of the solution state machine object is the first intermediate state, the result received from the machine computation component including a first confidence measure;

requesting processing of the task by, and receiving a result of, an agent computation component when the current state of the solution state machine object is the second intermediate state, the result received from the agent computation component including a second confidence measure; and

transitioning the current state of the solution state machine object according to the transition rules and at least one of: the first confidence measure and the second confidence measure; and

providing a characterization of the terminal state when the current state of the solution state machine object is the terminal state.

11 . The method of claim 10 , wherein the machine computation component executes a machine learning algorithm to perform the task.

12 . The method of claim 11 , wherein the machine computation component includes a convolutional neural network.

13 . The method of claim 10 , wherein the agent computation component includes a platform that queries at least one agent, receives a query result, determines a confidence measure of the agent, and determines the second confidence measure using the confidence measure of the queried agent.

14 . The method of claim 10 , wherein the sensor data includes an image including a single image, a series of images, or a video; and the computational task includes: detecting a pattern in the image; detecting a presence of an object within the image; detecting a presence of a person within the image; detecting intrusion of the object or person within a region of the image; detecting suspicious behavior of the person within the image; detecting an activity of the person within the image; detecting an object carried by the person, detecting a trajectory of the object or the person in the image; a status of the object or person in the image; identifying whether a person who is detected is on a watch list; determining whether a person or object has loitered for a certain amount of time; detecting interaction among person or objects; tracking a person or object; determining status of a scene or environment; determining the sentiment of one or more people; counting the number of objects or people; determining whether a person appears to be lost; determining whether an event is normal or abnormal; and/or determining whether text matches that in a database.

15 . The method of claim 10 , wherein the security system asset is an imaging device, a video camera, a still camera, a radar imaging device, a microphone, a chemical sensor, an acoustic sensor, a radiation sensor, a thermal sensor, a pressure sensor, a force sensor, or a proximity sensor.

16 . The method of claim 10 , wherein the modality defines: solution state machine object attributes, acceptable confidence for reaching the terminal state, a set of assets that trigger the modality, and agent query structure.

17 . The method of claim 10 , wherein executing the task includes posting, via a messaging queuing protocol, requested processing tasks, and wherein the machine computation component and agent computation component are microservices operating on tasks posted via the messaging queue protocol.

18 . The method of claim 10 , further comprising:

modifying a predictive model of the machine computation component using the result received from the agent computation component as a supervisory signal and the received sensor data as input.

19 . The method of claim 10 , wherein at least one of the receiving, accessing, instantiating, executing, and providing is performed by at least one data processor forming part of at least one computing system.

20 . A non-transitory computer program product which, when executed by at least one data processor forming part of at least one computer, result in operations comprising:

receiving sensor data;

classifying the sensor data into one of two or more classes by at least requesting processing of a machine computational component, receiving a result of the machine computation component, requesting processing of an agent computation component, and receiving a result of the agent computation component, the agent computation component including a platform to query an agent; and

providing the result from the agent computation component or the result from the machine computation component.

21 . The computer program product of claim 20 , wherein processing of the agent computation component is requested when a confidence of the machine computation component result is below a first threshold.

22 . The computer program product of claim 21 , wherein processing of the agent computational component is requested when the confidence of the machine computation component result is above a second threshold.

23 . The computer program product of claim 20 , wherein the providing includes requesting further processing of the agent computation component result by the machine computation component.

24 . The computer program product of claim 20 , wherein the providing includes requesting further processing of the machine computation component result by the agent computation component.

25 . A system comprising:

a media processor that receives sensor data; and

means for classifying the sensor data into one of two or more classes by at least requesting processing of a machine computational component, receiving a result of the machine computation component, requesting processing of an agent computation component, and receiving a result of the agent computation component, the agent computation component including a platform to query an agent.

26 . The system of claim 25 , further comprising:

means for requesting further processing of the agent computation component result by the machine computation component; and

means for requesting further processing of the machine computation component result by the agent computation component.