IP Library Granted Patent US 11,106,927
Granted Patent B2
US 11,106,927 · App. 16/234,307 · Granted Aug 31, 2021

Method for monitoring an interior state of an autonomous vehicle

Inventors: Carol Reiley (Mountain View, CA); Fabien Blanc-Paques (Mountain View, CA); Vineet Jain (Mountain View, CA); Tory Smith (Mountain View, CA); Gahl Levy (Mountain View, CA); Caitlin A. Surakitbanharn (Mountain View, CA); Chip J. Alexander (Mountain View, CA)
G06K9/00832G05D1/0088G08B3/10G08B5/36G08B13/19602G08B21/24G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,106,927
App. No.
16/234,307
Granted
Aug 31, 2021
Kind
B2
Abstract

Systems and methods disclosed herein involve monitoring an interior state of a vehicle. A pre-ride image of an interior is recorded prior to initiating a ride A post-ride image of the interior is recorded following completion of the ride. A region of the post-ride image differing from the pre-ride image is detected, with the region of the post-ride image representing a change in the interior. A type of an object depicted in the region of the post-ride image is classified based on features extracted from the region of the post-ride image. In response to classifying the object depicted in the region of the post-ride image as a personal item, a prompt is served to the user to retrieve the object.

Claims (30)

1. A method for monitoring an interior state of an autonomous vehicle, the method comprising:

recording a pre-ride image of an interior of the autonomous vehicle prior to unlocking doors of the autonomous vehicle for a user at a pickup location in connection with initiating a ride corresponding to a ride request;

autonomously navigating to a destination location specified in the ride request following completion of the ride, recording a post-ride image of the interior of the autonomous vehicle in response to detecting a departure of the user from the autonomous vehicle following arrival at the destination location;

detecting a region of the post-ride image differing from the pre-ride image, the region of the post-ride image representing a change in the interior of the autonomous vehicle following occupancy of the autonomous vehicle by the user;

classifying a type of an object depicted in the region of the post-ride image based on features extracted from the region of the post-ride image; and

in response to classifying the object depicted in the region of the post-ride image as a personal item, serving a prompt to the user to retrieve the object from the autonomous vehicle.

2. The method of claim 1 , wherein serving the prompt includes at least one of sending a communication to a mobile device associated with the user or a visual notification displayed using a display arranged on the autonomous vehicle.

3. The method of claim 1 , wherein serving the prompt includes issuing an audible alarm at the autonomous vehicle.

4. The method of claim 1 , wherein the prompt is served automatically to the user to retrieve the object from the autonomous vehicle in response to a confidence score for classification of the object as the personal item exceeding a threshold score.

5. The method of claim 4 , wherein the prompt is served following manual review when the confidence score for classification of the object is below the threshold score.

6. The method of claim 1 , wherein a video of a section of the interior of the autonomous vehicle is captured via a camera arranged in the autonomous vehicle.

7. The method of claim 1 , wherein the departure of the user is detected based on a closure of a last door of the autonomous vehicle following arrival of the autonomous vehicle at the destination location.

8. The method of claim 1 , wherein in response to classifying the object as a liquid, the autonomous vehicle navigates autonomously to a facility for cleanup.

9. The method of claim 1 , wherein in response to classifying the object as trash, prompting removal of the object from the autonomous vehicle.

10. The method of claim 9 , wherein in response to classifying the object as trash or a liquid, the autonomous vehicle navigates autonomously to a facility for cleanup and a cleaning surcharge is appended to a fare for the ride.

11. The method of claim 1 , wherein in response to detecting damage to the interior of the autonomous vehicle in the post-ride image, the autonomous vehicle navigates autonomously to a facility for repair.

12. The method of claim 1 , wherein the prompt includes the region of the post-ride image.

13. The method of claim 12 , wherein autonomous navigation away from the destination location is delayed until an earlier of retrieval of the object from the autonomous vehicle and an expiration of a timer.

14. The method of claim 1 , wherein a field around an exterior of the autonomous vehicle is scanned for the user following the prompt being served to the user.

15. The method of claim 14 , wherein the user is detected in the field around the exterior of the autonomous vehicle using facial recognition.

16. The method of claim 1 , wherein the autonomous vehicle autonomously changes the destination location to a hospital when detected movements and posture of the user deviate from a human rider model, the autonomous vehicle autonomously navigating to the hospital.

17. One or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising:

obtaining a pre-ride image of an interior of an autonomous vehicle prior to unlocking doors of the autonomous vehicle for entry of a user into the autonomous vehicle at a pickup location corresponding to a ride request, the autonomous vehicle autonomously navigating to a destination location specified in the ride request;

in response to detecting a departure of the user from the autonomous vehicle following arrival at the destination, recording a post-ride image of the interior of the autonomous vehicle;

comparing the post-ride image to the pre-ride image to identify a region of the post-ride image representing a change in the interior of the autonomous vehicle following occupancy of the user in the autonomous vehicle;

implementing an object model to classify an object depicted in the region of the post-ride image; and

in response to classifying the object depicted in the region of the post-ride image as a personal item, generating a prompt to the user to retrieve the object from the autonomous vehicle.

18. The one or more tangible non-transitory computer-readable storage media of claim 17 , wherein serving the prompt includes at least one of sending a communication to a mobile device associated with the user or a visual notification displayed using a display arranged on the autonomous vehicle.

19. The one or more tangible non-transitory computer-readable storage media of claim 17 , wherein the autonomous vehicle navigates to a facility for repair in response to damage to the interior of the autonomous vehicle being detected in the post-ride image.

20. The one or more tangible non-transitory computer-readable storage media of claim, 17 , wherein the autonomous vehicle navigates to a facility for cleanup in response to classification of the object as a liquid.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2024
From: DRIVE.AI, INC.
To: DIRECT CURRENT CAPITAL LLC
Reel/Frame 070510/0930 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 24, 2019
From: REILEY, CAROL; BLANC-PAQUES, FABIEN; JAIN, VINEET; SMITH, TORY; LEVY, GAHL; ALEXANDER, CHIP J.; SURAKITBANHARN, CAITLIN A.
To: DRIVE.AI
Reel/Frame 048987/0908 →
Continuity (2)
Provisional Application 62610696 · Dec 27, 2017
Related Publication 20190197325A1 · Jun 27, 2019
Cited By (3)
US 12,269,371 US 12,319,206 US 12,418,947