Superresolution enhancment of peripheral regions in nonlinear lens geometries
View Patent ↗A technique of enhancing a scene containing one or more off-center peripheral regions within an initial distorted image captured with a large field of view includes determining and extracting an off-center region of interest (hereinafter “ROI”) within the image. Geometric correction is applied to reconstruct the off-center ROI into a rectangular or otherwise undistorted or less distorted frame of reference as a reconstructed ROI. A quality of reconstructed pixels is determined within the reconstructed ROI. One or more additional initially distorted images is/are acquired, and matching additional ROIs are extracted and reconstructed to combine with reduced quality pixels of the first reconstructed ROI using a super-resolution technique to provide one or more enhanced ROIs.
1. Within an image acquisition system comprising a non-linear, wide-angled lens and an imaging sensor, a method of enhancing a scene containing one or more off-center peripheral regions, the method comprising:
acquiring an initial distorted image with a large field of view, including using a non-linear, wide-angled lens and an imaging sensor;
determining and extracting an off-center region of interest (ROI) within said initial distorted image;
applying a geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;
determining, on a pixel-by-pixel basis, a quality of reconstructed pixels within said first reconstructed ROI;
wherein, within the first reconstructed ROI, a first plurality of pixels have quality measures above a threshold, and a second plurality of pixels have quality measures below the threshold; and
acquiring one or more additional initially distorted images, and extracting and reconstructing matching additional ROIs to combine with the second plurality of pixels of said first reconstructed ROI using a super-resolution technique to provide one or more enhanced ROIs;
wherein the method is performed using one or more computing devices.
2. The method of claim 1 , further comprising applying an additional object tracking to the one or more enhanced ROIs to confirm a location of an object below a predetermined size threshold.
3. The method as in claim 1 , further comprising compensating for global motion of the imaging sensor.
4. The method as in claim 1 , further comprising, prior to said applying a geometric correction, applying an initial object detection process to the off-center ROI.
5. The method as in claim 4 , wherein said applying a geometric correction is performed in response to an initial determination that an object exists within the off-center ROI.
6. The method as in claim 2 , wherein said additional object tracking is applied to the first reconstructed ROI to refine or confirm, or both, an initial object detection process.
7. A digital image acquisition device, comprising:
a non-linear, wide-angled lens and an imaging sensor configured to capture digital images of scenes containing one or more off-center peripheral regions, including an initial distorted image with a large field of view;
a processor;
a memory having code embedded therein for programming the processor to perform a method of enhancing a scene containing one or more off-center peripheral regions, wherein the method comprises:
acquiring the initial distorted image with the large field of view, including using the non-linear, wide-angled lens and the imaging sensor;
determining and extracting an off-center region of interest (ROI) within said initial distorted image;
applying a geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;
determining, on a pixel-by-pixel basis, a quality of reconstructed pixels within said first reconstructed ROI;
wherein, within the first reconstructed ROI, a first plurality of pixels have quality measures above a threshold, and a second plurality of pixels have quality measures below the threshold; and
acquiring one or more additional initially distorted images, and extracting and reconstructing matching additional ROIs to combine with pixels of said first reconstructed ROI using a super-resolution technique to provide one or more enhanced ROIs.
8. The device of claim 7 , wherein the method further comprises applying an additional object tracking to the one or more enhanced ROIs to confirm a location of an object below a predetermined size threshold.
9. The device as in claim 7 , wherein the method further comprises compensating for global motion of the imaging sensor.
10. The device as in claim 7 , wherein the method further comprises, prior to said applying a geometric correction, applying an initial object detection process to the off-center ROI.
11. The device as in claim 10 , wherein said applying a geometric correction is performed in response to an initial determination that an object exists within the off-center ROI.
12. The device as in claim 8 , wherein said additional object tracking is applied to the first reconstructed ROI to refine or confirm, or both, an initial object detection process.
13. One or more non-transitory, processor-readable storage media having code embedded therein for programming a processor to perform a method of enhancing a scene captured with a non-linear, wide-angled lens and containing one or more off-center peripheral regions, wherein
the method comprising:
determining and extracting an off-center region of interest (ROI) within an acquired image;
applying a geometric correction to reconstruct the off-center ROI into an approximately rectangular frame of reference as a first reconstructed ROI;
determining, on a pixel-by-pixel basis, a quality of reconstructed pixels within said first reconstructed ROI;
wherein, within the first reconstructed ROI, a first plurality of pixels have quality measures above a threshold, and a second plurality of pixels have quality measures below the threshold; and
acquiring one or more additional initially distorted images, and extracting and reconstructing matching additional ROIs to combine with the second plurality of pixels of said first reconstructed ROI using a super-resolution technique to provide one or more enhanced ROIs.
14. One or more non-transitory processor-readable storage media as in claim 13 , wherein the method further comprises applying an additional object tracking to the one or more enhanced ROIs to confirm a location of an object below a predetermined size threshold.
15. One or more non-transitory processor-readable storage media as in claim 13 , wherein the method further comprises compensating for global motion of an imaging sensor.
16. One or more non-transitory processor-readable storage media as in claim 13 , wherein the method further comprises, prior to said applying a geometric correction, applying an initial object detection process to the off-center ROI.
17. One or more non-transitory processor-readable storage media as in claim 16 , wherein said applying a geometric correction is performed in response to an initial determination that an object exists within the off-center ROI.
18. One or more non-transitory processor-readable storage media as in claim 14 , wherein said additional object tracking is applied to the first reconstructed ROI to refine or confirm, or both, an initial object detection process.