Graphical Rendition of Multi-Modal Data
Changes and anomalies in multi-modal data are detected, collected and abstracted into understandable and actionable formats utilizing, for example, color, intensity, icons and texture creating a rendition of current and developing situations and events. Changes and anomalies in multi-modal sensor data are detected, aggregated, abstracted and filtered using case-based reasoning providing a tractable data dimensionality. From this collection of data situations are recognized and presented in a means so as to assist a user in accessing an environment and formulate the basis a recommended course of action.
1 . A system for rendition of multi-modal data, comprising:
a data collection module operable to asynchronously collect static and/or dynamic data across a plurality of data modalities
an anomaly detection module operable to detect one or more anomalies for each of a plurality of data modalities against background/historical data;
an anomaly correlation module operable to combine detected anomalies for each of the plurality of modalities into a common representation that is cross-correlated between each modality; and
a rendition module operable to render cross-correlated common representations of detected anomalies.
2 . The system for rendition of multi-modal data according to claim 1 , wherein each data modality is processed asynchronously and in parallel with other data modalities
3 . The system for rendition of multi-modal data according to claim 1 , wherein one or anomalies include unexpected changes in incongruity in collected data.
4 . The system for rendition of multi-modal data according to claim 1 , wherein each data modality is processed asynchronously and in parallel with other data modalities.
5 . The system for rendition of multi-modal data according to claim 1 , wherein the anomaly detection module combines two or more related anomalies within a data modality.
6 . The system for rendition of multi-modal data according to claim 1 , wherein the anomaly detection module prioritizes anomalies based on one or more predetermined criteria.
7 . The system for rendition of multi-modal data according to claim 1 , wherein the anomaly detection module filters detected anomalies.
8 . The system for rendition of multi-modal data according to claim 1 , wherein the rendition module represents clusters of anomalies in visual formats.
9 . A method for rendering multi-modal data; the method comprising:
collecting static and/or dynamic data from one or more modalities;
detecting one or more anomalies for each of a plurality of data modalities against background/historical data;
cross-correlating between each modality detected anomalies into a common frame of reference; and
rendering cross-correlated common representations of detected anomalies.
10 . The method for rendering multi-modal data according to claim 9 , further comprising abstracting collected static and/or dynamic data.
11 . The method for rendering multi-modal data according to claim 10 , further comprising comparing abstracted collected data to pre-identified event characteristics.
12 . The method for rendering multi-modal data according to claim 9 , wherein detecting includes conducting change analysis.
13 . The method for rendering multi-modal data according to claim 12 , wherein change analysis includes motion tracking and occupancy change analysis.
14 . The method for rendering multi-modal data according to claim 12 , wherein change analysis includes radio frequency analysis.
15 . The method for rendering multi-modal data according to claim 12 , wherein change analysis includes acoustical change analysis.
16 . The method for rendering multi-modal data according to claim 12 , wherein change analysis includes visual and thermal imagery analysis.
17 . The method for rendering multi-modal data according to claim 12 , wherein change analysis includes chemical change analysis.
18 . The method for rendering multi-modal data according to claim 9 , wherein detecting includes recognizing data fluctuations within each modality.
19 . The method for rendering multi-modal data according to claim 9 , wherein the common frame of reference possesses a common coordinate system.
20 . The method for rendering multi-modal data according to claim 9 , wherein the common frame or reference is based on environmental features.
21 . The method for rendering multi-modal data according to claim 9 , wherein cross-correlating includes indirect coordination based on evidence of interaction between agents and events that stimulates subsequent actions.
22 . The method for rendering multi-modal data according to claim 9 , wherein cross-correlating includes identifying physical and temporal relationships between and among the detected one or more anomalies.
23 . The method for rendering multi-modal data according to claim 9 , further comprising clustering the detected on or more anomalies based on common spatially salient features.
24 . A computer-readable storage medium tangibly embodying a program of instructions executable by a machine wherein said program of instruction comprises a plurality of program codes for rendering multi-modal data, said program of instruction comprising:
program code for collecting static and/or dynamic data from one or more modalities;
program code for detecting one or more anomalies for each of a plurality of data modalities against background/historical data;
program code for cross-correlating between each modality detected anomalies into a common frame of reference; and
program code for rendering cross-correlated common representations of detected anomalies.
25 . The program of instructions embodied in a computer-readable storage medium of claim 24 , further comprising program code for abstracting collected static and/or dynamic data.
26 . The program of instructions embodied in a computer-readable storage medium of claim 25 , further comprising program code for comparing abstracted collected data to pre-identified event characteristics.
27 . The program of instructions embodied in a computer-readable storage medium of claim 24 , further comprising program code for clustering the detected on or more anomalies based on common spatially salient features.
28 . The program of instructions embodied in a computer-readable storage medium of claim 24 , wherein cross-correlating includes indirect coordination based on evidence of interaction between agents and events that stimulates subsequent actions.
29 . The program of instructions embodied in a computer-readable storage medium of claim 24 , wherein cross-correlating includes identifying physical and temporal relationships between and among the detected one or more anomalies.