IP Library › Granted Patent US 11,820,338
Granted Patent B2
US 11,820,338 · App. 17/172,436 · Granted Nov 21, 2023

Autonomous vehicle cleaning and feedback system using adjustable ground truth

Inventors: Paul D. Schmalenberg (Pittsburgh, PA); Sean P. Rodrigues (Ann Arbor, MI); Atsushi Iwai (Pittsburgh, PA)
Assignee: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
B60S1/64B60Q3/80
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,820,338
App. No.
17/172,436
Granted
Nov 21, 2023
Kind
B2
Abstract

A vehicle includes a cleanable surface disposed within its interior. The vehicle also includes a processor, a light source able to illuminate the surface, and a sensor configured to capture an image of the surface while the surface is illuminated by the light source. The vehicle also includes a passenger survey system for receiving input from a passenger regarding cleanliness of the surface. The processor is configured to adjust a detection threshold based on the input, and to detect foreign material on the surface based on the detection threshold and a comparison between the captured image and a stored image. After detecting the foreign material on the surface, the processor indicates that the vehicle needs to be cleaned, and may initiate an automatic cleaning procedure.

Claims (74)

1. A vehicle comprising:

an interior and an exterior;

a surface disposed within the interior;

a processor;

a non-lidar light source configured to illuminate the surface, wherein the non-lidar light source comprises a blue light source, a violet light source, a near-infrared light source, or a near-ultraviolet light source;

a sensor configured to, under control of the processor, capture an image of the surface while the surface is illuminated by the non-lidar light source; and

a stored image of the surface;

a detection threshold; and

a passenger survey system configured to receive an input from a passenger regarding cleanliness of the surface,

wherein the processor is configured to adjust the detection threshold based on the input;

wherein the processor is configured to detect foreign material on the surface based on the detection threshold and a comparison between the captured image and the stored image,

wherein the processor is configured to, after detecting the foreign material on the surface, indicate that the vehicle needs to be cleaned.

2. The vehicle of claim 1 , wherein the passenger survey system is disposed within the interior of the vehicle, and wherein the vehicle further comprises a cleaning element configured to, when activated by the processor indicating that the vehicle needs to be cleaned, clean the surface.

3. The vehicle of claim 2 , wherein the passenger survey system comprises a touchscreen, a voice interface, a gesture interface, or a housing comprising electromechanical buttons.

4. The vehicle of claim 2 , wherein the processor is configured to activate the cleaning element for:

a fixed period of time,

a period of time based on the input, or

a period of time based on a property of the detected foreign material.

5. The vehicle of claim 2 , wherein the cleaning element is a UV lamp or a cleaning fluid delivery system.

6. The vehicle of claim 1 , further comprising:

a movable seat; and

a seat control system configured to, under control of the processor, move the movable seat such that the surface is exposed.

7. The vehicle of claim 1 , further comprising:

an external light control system configured to adjust an amount of external light entering the interior of the vehicle prior to activation of the non-lidar light source.

8. The vehicle of claim 7 , wherein the external light control system comprises automatic curtains, automatic shades, automatic louvres, automatic visors, electrochromic elements, an automatic garage door controller, or an autonomous driving system configured to drive the vehicle to a location with a desirable lighting condition.

9. The vehicle of claim 1 , wherein the passenger survey system is further configured to receive a second input from the passenger regarding a wear defect of the surface, and wherein the processor is configured, via the non-lidar light source and sensor, to update the stored image of the surface to include the wear defect.

10. The vehicle of claim 1 , wherein:

the non-lidar light source comprises an ultraviolet (UV) light source, and the sensor comprises a UV, visible light, or infrared (IR) sensor;

the non-lidar light source comprises a visible light source, and the sensor comprises a visible light camera; or

the non-lidar light source comprises an infrared light source, and the sensor comprises an infrared sensor.

11. A method for cleaning an interior surface of a vehicle, the method comprising:

in a vehicle comprising an interior and an exterior, providing a surface disposed within the interior; and

under control of a processor:

illuminate the surface with a non-lidar light source, wherein the non-lidar light source comprises a blue light source, a violet light source, a near-infrared light source, or a near-ultraviolet light source;

capture a baseline image of the surface with a sensor, while the surface is illuminated by the non-lidar light source;

store the baseline image;

define a detection threshold;

with a passenger survey system, receive an input from a passenger regarding cleanliness of the surface,

adjust the detection threshold based on the input;

capture a current image of the surface with a sensor, while the surface is illuminated by the non-lidar light source;

detect foreign material on the surface based on the detection threshold and a comparison between the current image and the stored image; and

after detecting the foreign material on the surface, clean the surface by activating a cleaning element.

12. The method of claim 11 , wherein the passenger survey system is disposed within the interior of the vehicle, and wherein the passenger survey system comprises a touchscreen, a voice interface, a gesture interface, or a housing comprising electromechanical buttons.

13. The method of claim 11 , further comprising:

under control of the processor, moving a movable seat such that the surface is exposed.

14. The method of claim 11 , further comprising adjusting an amount of external light entering the interior of the vehicle prior to activation of the non-lidar light source.

15. The method of claim 14 , wherein adjusting the amount of external light entering the interior comprises activating automatic curtains, automatic shades, automatic louvres, automatic visors, electrochromic elements, an automatic garage door controller, or an autonomous driving system configured to drive the vehicle to a location with a desirable lighting condition.

16. The method of claim 11 , further comprising:

under control of the processor:

with the passenger survey system, receiving a second input from the passenger regarding a wear defect of the surface; and

with the non-lidar light source and sensor, updating the stored image of the surface to include the wear defect.

17. The method of claim 11 , activating the cleaning element comprises turning on the cleaning element for:

a fixed period of time,

a period of time based on the input, or

a period of time based on a property of the detected foreign material.

18. The method of claim 11 , wherein the cleaning element is a UV lamp or a cleaning fluid delivery system.

19. The method of claim 11 , wherein:

the non-lidar light source comprises an ultraviolet (UV) light source, and the sensor comprises a UV, visible light, or infrared (IR) sensor;

the non-lidar light source comprises a visible light source, and the sensor comprises a visible light camera; or

the non-lidar light source comprises an infrared light source, and the sensor comprises an infrared sensor.

20. A system for cleaning a surface of an interior of a vehicle, the system comprising:

the vehicle, wherein the vehicle comprises an interior and an exterior;

a surface disposed within the interior;

a processor;

a non-lidar light source configured to illuminate the surface, wherein the non-lidar light source comprises a blue light source, a violet light source, a near-infrared light source, or a near-ultraviolet light source;

a sensor configured to, under control of the processor, capture an image of the surface while the surface is illuminated by the non-lidar light source; and

a stored image of the surface;

a cleaning element configured to, when activated by the processor, clean the surface, a detection threshold;

a movable seat;

a seat control system configured to, under control of the processor, move the movable seat such that the surface is exposed;

an external light control system configured to, under control of the processor, adjust an amount of external light entering the interior of the vehicle prior to activation of the non-lidar light source; and

a passenger survey system disposed within the interior of the vehicle and configured to receive an input from a passenger regarding cleanliness of the surface, wherein the processor is configured to adjust the detection threshold based on the input;

wherein the processor is configured to detect foreign material on the surface based on the detection threshold and a comparison between the captured image and the stored image,

wherein the processor is configured to, after detecting the foreign material on the surface, activate the cleaning element.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 4, 2024
From: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 066022/0018 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 10, 2021
From: SCHMALENBERG, PAUL D; RODRIGUES, SEAN P.; IWAI, ATSUSHI
To: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
Reel/Frame 055219/0808 →
Continuity (1)
Related Publication 20220250589A1 · Aug 11, 2022
Cited By (1)
US 12,243,251