Portable apparatus and method for decision support for real time automated multisensor data fusion and analysis

1. A portable sensor fusion apparatus for the simultaneous analysis of multiple and different data inputs from multiple and different sensors, the sensors detecting at least one user defined image, the apparatus comprising a computer-comprising:

a data receiving subsystem capable of receiving streaming inputs from a multiplicity of sensors, and capable of indexing and storing the data and the associated geolocation information for where the data was collected;

a data library subsystem for storing exemplars and signatures for each of the types of sensor data used, including images for objects of interest, spectral radiance from locations of interest, and images of background objects;

a processing subsystem for detecting and matching objects in an image received from a sensor, detecting and matching objects of interest from the data library, the custom software capable of estimating a match probability for individual sensor data or images to the exemplars and signatures stored in the data library, and wherein the processing subsystem is further capable of assigning a new probability of detection by combining the match probability with a quality factor value for specific images and sensor data received from two or more of the streaming inputs, the quality factor value used to determine a new modified probability of detection value for matching specific images or data received, wherein a quality factor is assigned to each data set collected from each sensor and is based upon the reliability and performance of the sensor in comparison to the other sensors and Meta data (capable of high, medium and low detection reliability);

an indexing subsystem to correlate and store detection events with geo spatial information combined with probability of detection, sensor input quality factor and environmental factors to fuse the results of said analysis;

and a graphical user interface to display an image along with the detection events and geolocation associated with it.

2. The apparatus of claim 1 , wherein a high performance digital image camera is used as one sensor and an infrared camera is used as a second sensor.

3. A method of surveilling an area comprising attaching the apparatus of claim 1 to a vehicle.

4. The method of claim 3 , where only fused composite images are transmitted from the apparatus in the vehicle to a display for an operator.

5. A method of creating a composite of input images to form a fused, composite image data set comprising:

providing a sensor fusion apparatus for the simultaneous analysis of multiple and different data inputs from multiple and different sensors, the sensors detecting at least one user defined image, the apparatus containing a library of exemplar data;

selecting a spectral wavelength for detecting an image from one sensor used;

selecting a radar input at a frequency consistent with reflectivity in either light foliage or heavy foliage for another sensor used;

recording an image with visible light, infrared, and radar sensors;

geo-registering the image;

performing an image based pixel level detection of the “first” through “n” optical sensor images and supplementing the image data with spectral and radar data to create a composite image;

determining if an equivalent matched image occurred across the sensors used or in the library of exemplar data;

and displaying the composite image.

6. The method of claim 5 , wherein the composite image includes forestry or agricultural data.

7. The method of claim 5 , wherein the composite image includes border or geographic boundary data.

8. The method of claim 5 , wherein the probability of detection (Pd) and probability of false alarm (Pfa) is determined for each fused composite data set.

9. The method of claim 8 , wherein information on the Pd and Pfa is displayed to an operator in the composite image.

10. The method of claim 5 , wherein the fused composite image is generated by a remote high performance computer in communication with the multiple and different sensors.

11. The method of claim 5 , wherein a probability of detection is calculated for each layer of sensor data, and a composite probability of detection is calculated for the fused image or data.

12. A method of creating a composite of input images to form a fused, composite image data set comprising:

providing a sensor fusion apparatus for the simultaneous analysis of multiple and different data inputs from multiple and different sensors, the sensors detecting at least one user defined image, the apparatus containing a library of exemplar data;

selecting a sensor for detecting coronal discharges;

selecting an infrared sensor consistent with powerline transmission monitoring;

recording an image from the sensors;

geo-registering the image;

performing an image based pixel level detection of the first through “n” optical sensor images and supplementing the image data with spectral and coronal discharge data to create a composite image;

determining if an equivalent matched image occurred across the sensors used or in the library of exemplar data;

and displaying the composite image.

13. The method of claim 12 , wherein the exemplar data includes images of powerline locations in order to monitor the conditions of the powerline.

14. The method of claim 12 , wherein the probability of detection (Pd) and probability of false alarm (Pfa) is determined for each fused composite data set.

15. The method of claim 14 , wherein information on the Pd and Pfa is displayed to an operator in the composite image.

16. The method of claim 12 , wherein the fused composite image is generated by a remote high performance computer in communication with the multiple and different sensors.

17. The method of claim 12 , wherein a probability of detection is calculated for each layer of sensor data, and a composite probability of detection is calculated for the fused image or data.