IP Library Granted Patent US 11,997,252
Granted Patent B2
US 11,997,252 · App. 18/348,492 · Granted May 28, 2024

Validation of a camera cleaning system

Inventors: Dustin Ryan Yautz (Wexford, PA); Morgan M. Wagner (Pittsburgh, PA); Eric K. Thong (Palo Alto, CA); Christopher N. St. John (Northville, MI)
Assignee: Argo AI, LLC
H04N17/002G06F18/22G06V10/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,997,252
App. No.
18/348,492
Granted
May 28, 2024
Kind
B2
Abstract

Devices, systems, and methods are provided for testing and validation of a cleaning system for a sensor such as a camera. A device may capture a first image of a target using the sensor, wherein the sensor is in a clean state, and wherein the target is in a line of sight of the sensor. The device may apply an obstruction to a portion of a lens of the sensor. The device may apply a cleaning system to the lens. The device may capture a post-clean image after applying the cleaning system. The device may determine a post-clean image quality score based on comparing the post clean image to the first image. The device may compare the post-clean image quality score to a validation threshold. The device may determine a validation state of the cleaning system based on the comparison.

Claims (60)

1. A sensor cleaning validation system comprising:

a sensor with a lens;

a processor; and

a storage device containing programming instructions that are configured to cause the processor to:

receive a baseline image captured by the sensor when the lens is a clean state,

receive, from the sensor after a cleaning system has performed a cleaning operation to remove a detected obstruction from the lens, a post clean image,

generate a post-clean image quality score for the post clean image by:

performing a first down-sampling operation of the post clean image to generate a first down-sampled image, and

determining a first image quality score for the first down-sampled image, and

determine an operational state of the cleaning system in response to the post-clean image quality score being less than a validation threshold.

2. The sensor cleaning validation system of claim 1 , wherein the sensor comprises a camera.

3. The sensor cleaning validation system of claim 1 , further comprising additional programming instructions to adjust at least one parameter of the cleaning system in response to the operational state corresponding to a non-performing state.

4. The sensor cleaning validation system of claim 1 , wherein the-image quality score comprises a structural similarity index measurement (SSIM) score.

5. The sensor cleaning validation system of claim 1 , further comprising additional programming instructions to:

perform a second down-sampling operation of the first down-sampled image to generate a second down-sampled image; and

determine a second image quality score for the second down sampled image, the second image quality score being a second local image quality score.

6. The sensor cleaning validation system of claim 5 , further comprising additional programming instructions to:

generate a global image quality score representing an average of the first local image quality score and the second local image quality score.

7. The sensor cleaning validation system of claim 6 , further comprising additional programming instructions to identify, based on the global image quality score, regions of interest in the post clean image, wherein the regions of interest correspond to locations on the lens that have not been properly cleaned.

8. The sensor cleaning validation system of claim 1 , further comprising additional programming instructions to:

identify a first region of the post clean image having image quality scores below the validation threshold;

identify a second region of the post clean image having image quality scores below the validation threshold; and

determine that the first region and the second region are connected together.

9. The sensor cleaning validation system of claim 8 , further comprising additional programming instructions to:

cause a display screen to display a bounding box around the first region, around the second region, or around the first region and the second region in response to the first region, the second region, or a combined size of the first region and second region spanning an area greater than a predetermined pixel size.

10. The sensor cleaning validation system of claim 9 , further comprising additional programming instructions to:

classify the displayed bounding box as a degraded area that has a worse image quality score than an area without a box around it; and

adjust at least one parameter of the cleaning system to mitigate the degraded area.

11. The sensor cleaning validation system of claim 10 , wherein the at least one parameter of the cleaning system comprises an airflow speed of the cleaning system.

12. A sensor cleaning validation method comprising, by a processor:

receiving a baseline image captured by a sensor when a lens of the sensor is a clean state;

receiving, from the sensor after a cleaning system has performed a cleaning operation to remove a detected obstruction from the lens, a post clean image;

generating a post-clean image quality score for the post clean image by:

performing a first down-sampling operation of the post clean image to generate a first down-sampled image, and

determining a first image quality score for the first down-sampled image; and

determining an operational state of the cleaning system in response to the post-clean image quality score being less than a validation threshold.

13. The method of claim 12 further comprising, by the processor, adjusting at least one parameter of the cleaning system in response to the operational state corresponding to a non-performing state.

14. The method of claim 12 further comprising, by the processor:

performing a second down-sampling operation of the first down-sampled image to generate a second down-sampled image; and

determining a second image quality score for the second down sampled image, the second image quality score being a second local image quality score.

15. The method of claim 14 further comprising, by the processor:

generating a global image quality score representing an average of the first local image quality score and the second local image quality score.

16. The method of claim 15 further comprising, by the processor:

identifying, based on the global image quality score, regions of interest in the post clean image, wherein the regions of interest correspond to locations on the lens that have not been properly cleaned.

17. The method of claim 12 further comprising, by the processor:

identifying a first region of the post clean image having image quality scores below the validation threshold;

identifying a second region of the post clean image having image quality scores below the validation threshold; and

determining that the first region and the second region are connected together.

18. The method of claim 17 further comprising, by the processor:

causing a display screen to display a bounding box around the first region, around the second region, or around the first region and the second region in response to the first region, the second region, or a combined size of the first region and second region spanning an area greater than a predetermined pixel size.

19. The method of claim 18 further comprising, by the processor:

classifying the displayed bounding box as a degraded area that has a worse image quality score than an area without a box around it; and

adjusting at least one parameter of the cleaning system to mitigate the degraded area.

20. A non-transitory, machine-readable medium containing programming instructions that are configured to cause a processor to:

receive a baseline image captured by a sensor when a lens of the sensor is a clean state;

receive, from the sensor after a cleaning system has performed a cleaning operation to remove a detected obstruction from the lens, a post clean image;

generate a post-clean image quality score for the post clean image by:

performing a first down-sampling operation of the post clean image to generate a first down-sampled image, and

determining a first image quality score for the first down-sampled image; and

determine an operational state of the cleaning system in response to the post-clean image quality score being less than a validation threshold.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2024
From: ARGO AI, LLC
To: VOLKSWAGEN GROUP OF AMERICA INVESTMENTS, LLC
Reel/Frame 069177/0099 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 7, 2023
From: YAUTZ, DUSTIN RYAN; WAGNER, MORGAN M.; THONG, ERIC K.; ST. JOHN, CHRISTOPHER N.
To: ARGO AI, LLC
Reel/Frame 064181/0966 →
Continuity (3)
Continuation 17844588 · Jun 20, 2022
Continuation 17020297 · Sep 14, 2020
Related Publication 20230353722A1 · Nov 2, 2023