IP Library Granted Patent US 11,368,672
Granted Patent B2
US 11,368,672 · App. 17/020,297 · Granted Jun 21, 2022

Validation of a camera cleaning system

Inventors: Dustin Ryan Yautz (Wexford, PA); Morgan M. Wagner (Pittsburgh, PA); Eric K. Thong (Palo Alto, CA); Christopher N. St. John (Northville, MI)
Assignee: ARGO AI, LLC
H04N17/002G06K9/6215G06V10/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,368,672
App. No.
17/020,297
Granted
Jun 21, 2022
Kind
B2
Abstract

Devices, systems, and methods are provided for testing and validation of a camera. A device may capture a first image of a target using a camera, wherein the camera is in a clean state, and wherein the target is in a line of sight of the camera. The device may apply an obstruction to a portion of a lens of the camera. The device may apply a camera cleaning system to the lens of the camera. The device may capture a post-clean image after applying the camera cleaning system. The device may determine a post-clean SSIM score based on comparing the post clean image to the first image. The device may compare the post-clean SSIM score to a validation threshold. The device may determine a validation state of the camera cleaning system based on the comparison.

Claims (63)

1. A method comprising:

capturing, by one or more processors, a first image of a target using a camera, wherein the camera is in a clean state, and wherein the target is in a line of sight of the camera;

applying an obstruction to a portion of a lens of the camera;

initiating a camera cleaning system to the lens of the camera;

capturing a post-clean image after applying the camera cleaning system;

determining a post-clean structural similarity index measurement (SSIM) score based on comparing the post-clean image to the first image;

comparing the post-clean SSIM score to a validation threshold;

determining a validation state of the camera cleaning system based on the comparison;

adjusting one or more parameters of the camera cleaning system in response to the validation state indicating that the post-clean SSIM score is less than the validation threshold;

determining a first down-sampled image based on down-sampling the post-clean image;

determining a first local SSIM score based on comparing the first image and the post-clean down-sampled image;

determining a second down-sampled image based on down-sampling of the first down-sampled image; and

determining a second local SSIM score based on comparing the first down-sampled image and the second down-sampled image.

2. The method of claim 1 , further comprising determining a global SSIM score, wherein the global SSIM score is an average of the first local SSIM score and the second local SSIM score.

3. The method of claim 1 , further comprising:

determining a first region of the post-clean image having SSIM scores below a validation threshold;

determining a second region of the post-clean image having SSIM scores below the validation threshold; and

determining the first region and the second region are connected together.

4. The method of claim 3 , further comprising displaying a box around the first region and the second region.

5. The method of claim 4 , further comprising flagging the displayed box as a degraded area that has a worse SSIM score than an area without a box around it.

6. The method of claim 5 , further comprising performing one or more adjustments to the camera cleaning system to mitigate the degraded area.

7. A device, the device comprising processing circuitry coupled to storage, the processing circuitry configured to:

capture a first image of a target using a camera, wherein the camera is in a clean state, and wherein the target is in a line of sight of the camera;

apply an obstruction to a portion of a lens of the camera;

initiate a camera cleaning system to the lens of the camera;

capture a post-clean image after applying the camera cleaning system;

determine a post-clean structural similarity index measurement (SSIM) score based on comparing the post-clean image to the first image;

compare the post-clean SSIM score to a validation threshold;

determine a validation state of the camera cleaning system based on the comparison;

adjust one or more parameters of the camera cleaning system in response to the validation state indicating that the post-clean SSIM score is less than the validation threshold;

determine a first down-sampled image based on down-sampling the post-clean image;

determine a first local SSIM score based on comparing the first image and the post-clean down-sampled image;

determine a second down-sampled image based on down-sampling of the first down-sampled image; and

determine a second local SSIM score based on comparing the first down-sampled image and the second down-sampled image.

8. The device of claim 7 , wherein the processing circuitry is further configured to:

determine a global SSIM score, wherein the global SSIM score is an average of the first local SSIM score and the second local SSIM score.

9. The device of claim 7 , wherein the processing circuitry is further configured to:

determine a first region of the post-clean image having SSIM scores below a validation threshold;

determine a second region of the post-clean image having SSIM scores below the validation threshold; and

determine the first region and the second region are connected together.

10. The device of claim 9 , wherein the processing circuitry is further configured to display a box around the first region and the second region.

11. The device of claim 10 , wherein the processing circuitry is further configured to flag the displayed box as a degraded area that has a worse SSIM score than an area without a box around it.

12. The device of claim 11 , wherein the processing circuitry is further configured to perform one or more adjustments to the camera cleaning system to mitigate the degraded area.

13. A non-transitory computer-readable medium storing computer-executable instructions which when executed by one or more processors result in performing operations comprising:

capturing a first image of a target using a camera, wherein the camera is in a clean state, and wherein the target is in a line of sight of the camera;

applying an obstruction to a portion of a lens of the camera;

initiating a camera cleaning system to the lens of the camera;

capturing a post-clean image after applying the camera cleaning system;

determining a post-clean structural similarity index measurement (SSIM) score based on comparing the post-clean image to the first image;

comparing the post-clean SSIM score to a validation threshold;

determining a validation state of the camera cleaning system based on the comparison;

adjusting one or more parameters of the camera cleaning system in response to the validation state indicating that the post-clean SSIM score is less than the validation threshold;

determining a first down-sampled image based on down-sampling the post-clean image;

determining a first local SSIM score based on comparing the first image and the post-clean down-sampled image;

determining a second down-sampled image based on down-sampling of the first down-sampled image; and

determining a second local SSIM score based on comparing the first down-sampled image and the second down-sampled image.

14. The non-transitory computer-readable medium of claim 13 , further comprising determining a global SSIM score, wherein the global SSIM score is an average of the first local SSIM score and the second local SSIM score.

15. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise:

determining a first region of the post-clean image having SSIM scores below a validation threshold;

determining a second region of the post-clean image having SSIM scores below the validation threshold; and

determining the first region and the second region are connected together.

16. The non-transitory computer-readable medium of claim 15 , wherein the operations further comprise displaying a box around the first region and the second region.

17. The non-transitory computer-readable medium of claim 16 , wherein the operations further comprise flagging the displayed box as a degraded area that has a worse SSIM score than an area without a box around it.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2024
From: ARGO AI, LLC
To: VOLKSWAGEN GROUP OF AMERICA INVESTMENTS, LLC
Reel/Frame 069177/0099 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: YAUTZ, DUSTIN RYAN; WAGNER, MORGAN M.; THONG, ERIC K.; ST. JOHN, CHRISTOPHER N.
To: ARGO AI, LLC
Reel/Frame 054164/0019 →
Continuity (1)
Related Publication 20220086423A1 · Mar 17, 2022