Foveated compressive sensing system
Described is a system for foveated compressive sensing. The system is configured to receive an input image f of a scene and initialize a measurement matrix. Global measurements are then performed, with a lower resolution image of the scene thereafter reconstructed. Task salient regions are extracted from the low resolution image. Thereafter, the system estimates a task-specific operator and detects regions-of-interest (ROI) based on the task salient regions. An ROI-adapted and foveated measurement matrix is then generated. Local measurements are then performed on task-relevant ROIs. A higher resolution image can then be reconstructed of the scene to allow for identification of objects in the ROI.
1. A system for foveated compressive sensing, the system comprising:
one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of:
receiving an input image f of a scene;
initializing a measurement matrix;
performing global measurements of the input image f;
reconstructing a lower resolution image of the scene;
extracting task salient regions from the low resolution image;
estimating a task-specific operator and detecting regions-of-interest (ROI) based on the task salient regions;
generating an ROI-adapted and foveated measurement matrix;
performing local measurements on task-relevant ROIs; and
reconstructing a higher resolution image of the scene to allow for identification of objects in the ROI.
2. The system as set forth in claim 1 , wherein initializing the measurement matrix further includes operations of:
defining a first task;
implementing a task prior; and
initializing the measurement matrix.
3. The system as set forth in claim 2 , wherein in initializing a measurement matrix, the input image f is focused on a spatial light modulator (SLM) that processes image patches in parallel and generates rows of a foveated measurement matrix AP f serially.
4. The system as set forth in claim 3 , wherein at each time step j, a product of the input image f and row j of AP f , is focused on a detector array having a plurality of detectors, such that each detector in the detector array performs a separate patch measurement by spatially integrating AP f f in its local patch area.
5. A computer program product for foveated compressive sensing, the computer program product comprising:
a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of:
receiving an input image f of a scene;
initializing a measurement matrix;
performing global measurements of the input image f,
reconstructing a lower resolution image of the scene;
extracting task salient regions from the low resolution image;
estimating a task-specific operator and detecting regions-of-interest (ROI) based on the task salient regions;
generating an ROI-adapted and foveated measurement matrix;
performing local measurements on task-relevant ROIs; and
reconstructing a higher resolution image of the scene to allow for identification of objects in the ROI.
6. The computer program product as set forth in claim 5 , wherein initializing the measurement matrix further includes operations of:
defining a first task;
implementing a task prior, and
initializing the measurement matrix.
7. The computer program product as set forth in claim 6 , wherein in initializing a measurement matrix, the input image f is focused on a spatial light modulator (SLM) that processes image patches in parallel and generates rows of a foveated measurement matrix AP f serially.
8. The computer program product as set forth in claim 7 , wherein at each time step j, a product of the input image f and row j of AP f , is focused on a detector array having a plurality of detectors, such that each detector in the detector array performs a separate patch measurement by spatially integrating AP f f in its local patch area.
9. A computer implemented method for foveated compressive sensing, the method comprising an act of:
causing one or more processors to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
receiving an input image f of a scene;
initializing a measurement matrix;
performing global measurements of the input image f,
reconstructing a lower resolution image of the scene;
extracting task salient regions from the low resolution image;
estimating a task-specific operator and detecting regions-of-interest (ROI) based on the task salient regions;
generating an ROI-adapted and foveated measurement matrix;
performing local measurements on task-relevant ROIs; and
reconstructing a higher resolution image of the scene to allow for identification of objects in the ROI.
10. The method as set forth in claim 9 , wherein initializing the measurement matrix further includes operations of:
defining a first task;
implementing a task prior; and
initializing the measurement matrix.
11. The method as set forth in claim 10 , wherein in initializing a measurement matrix, the input image f is focused on a spatial light modulator (SLM) that processes image patches in parallel and generates rows of a foveated measurement matrix AP f serially.
12. The method as set forth in claim 11 , wherein at each time step j, a product of the input image f and row j of AP f , is focused on a detector array having a plurality of detectors, such that each detector in the detector array performs a separate patch measurement by spatially integrating AP f f in its local patch area.