System and method for spatiotemporal image fusion and integration
View Patent ↗A system and method for creating a spatiotemporal image representation with information content that enables accurate and rapid processing for several applications.
1. A computer implemented method for processing a sequence of images comprising:
using a processor to acquire a sequence of images from at least one of a camera or a memory, the images related to each other by a common parameter and representing a region of interest of a physical location;
extracting multiple 1D signals from at least one image of the of the sequence of images;
stacking the 1D signals to form multiple 1D signal vs common parameter stacks;
applying a joint transform with at least two 1D signal vs common parameter stacks as input; and
outputting at least one transformed signal vs. common parameter stack, the transformed signal vs. common parameter stack representing the sequence of images.
2. The computer implemented method of claim 1 , wherein acquiring a sequence of images includes acquiring video comprising a plurality of temporally related images, wherein the common parameter is time.
3. The computer implemented method of claim 1 , wherein acquiring a sequence of images includes acquiring a plurality of images of a physical location taken from a plurality of vantage points, at least two of the images including common features of the physical location, wherein the common parameter is space.
4. The computer implemented method of claim 1 , wherein applying the joint transform includes using a multiple input, multiple output function, wherein the multiple inputs include multiple 1D signal vs common parameter stacks.
5. The computer implemented method of claim 1 , further comprising redefining at least one of the 1D signal vs common parameter stacks into RGB color channels, transferring the RGB color channels to HSV, and normalizing V to suppress global illumination fluctuation.
6. The computer implemented method of claim 1 , further comprising performing temporal noise reduction on the at least one 1D signal vs common parameter stack.
7. The computer implemented method of claim 6 , further comprising performing spatial noise reduction on the at least one 1D signal vs common parameter stack.
8. The computer implemented method of claim 7 , further comprising performing preprocessing of the at least one 1D signal vs common parameter stacks.
9. The computer implemented method of claim 1 , further comprising generating a plurality of 1D signal vs common parameter stacks, each 1D signal vs common parameter stacks comprised of lines of data extracted from the same location in each image of the sequence of images.
10. A system for processing a sequence of images comprising:
a camera for acquiring a sequence of images, the images related to each other by a common parameter; and
a memory in communication with a processor configured to:
extract multiple 1D signals from at least one image of the of the sequence of images;
stack the 1D signals to form multiple 1D signal vs common parameter stacks;
apply a joint transform with at least two 1D signal vs common parameter stacks as input; and
output at least one transformed signal vs. common parameter stack.
11. The system of claim 10 , wherein the processor is further configured to acquire video comprising a plurality of temporally related images, wherein the common parameter is time.
12. The system of claim 10 , wherein the processor is further configured to acquire a plurality of images of a physical location taken from a plurality of vantage points, at least two of the images including common features of the physical location, wherein the common parameter is space.
13. The system of claim 10 , wherein the processor is further configured to apply the joint transform including using a multiple input, multiple output function, wherein the multiple inputs include multiple 1D signal vs common parameter stacks.
14. The system of claim 10 , wherein the processor is further configured to redefine at least one of the 1D signal vs common parameter stacks into RGB color channels, transfer the RGB color channels to HSV, and normalize V to suppress global illumination fluctuation.
15. The system of claim 10 , wherein the processor is further configured to perform temporal noise reduction on the at least one 1D signal vs common parameter stack.
16. The system of claim 10 , wherein the processor is further configured to perform spatial noise reduction on the at least one 1D signal vs common parameter stack.
17. The system of claim 10 , wherein the processor is further configured to perform preprocessing of the at least one 1D signal vs common parameter stacks.
18. The system of claim 10 , wherein the processor is further configured to generate a plurality of 1D signal vs common parameter stacks, each 1D signal vs common parameter stacks comprised of lines of data extracted from the same location in each image of the sequence of images.