IP Library Granted Patent US 12,106,586
Granted Patent B2
US 12,106,586 · App. 17/705,972 · Granted Oct 1, 2024

Lost object tracking system

Inventors: Nestor Grace (San Francisco, CA); Katelyn Mary Wolfenberger (San Francisco, CA); Hooman Barekatain (Los Angeles, CA); Danyan Qian (Santa Clara, CA); Weichao Kong (Sunnyvale, CA)
Assignee: GM Cruise Holdings LLC
G06V20/59G06T7/20G06T7/70G06V10/803G06T2207/30268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,106,586
App. No.
17/705,972
Granted
Oct 1, 2024
Kind
B2
Abstract

A method includes determining that an object has fallen to a surface of interest in an interior cabin of a vehicle during a transportation service provided using the vehicle; tracking a location of the object on the surface of interest using at least one interior sensor; updating tracking information for the object, the tracking information including the tracked location of the object and a time at which the location was tracked; inferring a continued existence of the object after the object has moved out of a view of the at least one interior sensor and tagging the object as obscured from view in the tracking information for the object; identifying a user associated with the transportation service; and alerting the identified user to retrieve the object.

Claims (45)

1. A method comprising:

determining that an object has fallen to a surface of interest in an interior cabin of a vehicle during a transportation service provided using the vehicle;

determining a location of the object on the surface of interest using at least one interior sensor;

updating tracking information for the object, the tracking information including the determined location of the object and a time at which the location was determined;

updating, in response to the object moving out of a view of the at least one interior sensor, the tracking information with the location of the object at a time at which the object moved out of the view of the at least one interior sensor;

inferring a continued existence of the object when the object is out of the view of the at least one interior sensor, tagging the object as obscured from view in the tracking information for the object, and inferring the location of the object based on a tilt and a speed of the vehicle;

identifying a user associated with the transportation service; and

alerting the identified user to retrieve the object, the alert including the location of the object.

2. The method of claim 1 , further comprising determining an identity of the object using sensory data from the at least one interior sensor, wherein the tracking information for the object includes the identity of the object.

3. The method of claim 1 , further comprising:

determining whether the user has exited the cabin of the vehicle; and

if the user has exited the cabin of the vehicle, contacting the user to arrange for a return of the object to the user.

4. The method of claim 3 , wherein the contacting the user comprises at least one of telephoning the user, texting the user, emailing the user, and messaging the user via a user application associated with the transportation service.

5. The method of claim 1 , further comprising:

determining whether the user wants the object returned;

if the user wants the object returned, facilitating a return of the object to the user; and

if the user does not want the object returned, removing the object from the vehicle during servicing of the vehicle.

6. The method of claim 1 , further comprising, when the object returns to the view of the at least one interior sensor subsequent to being out of the view of the at least one interior sensor, removing an out of sight (OOS) tag from the tracking information for the object.

7. The method of claim 1 , wherein the at least one interior sensor comprises at least one camera.

8. The method of claim 1 , further comprising at least one auxiliary sensor for tracking the location of the object when the object is out of the view of the at least one interior sensor, wherein the at least one auxiliary sensor comprises at least one of a weight sensor, a touch sensor, an ultrasonic sensor, and a laser sensor and wherein the at least one auxiliary sensor is arranged such that sensory data from the at least one auxiliary sensor is used to track the object when the object is out of the view of the at least one interior sensor.

9. A method comprising:

detecting, using sensory data from at least one interior sensor provided in a cabin of a vehicle, that an object is in an occluded area of the vehicle cabin during a transportation service provided using the vehicle;

inferring, based on a tilt and a speed of the AV, a location of the object when the object is in the occluded area of the vehicle cabin;

identifying a user associated with the transportation service;

determining an identity of the object using the sensory data;

if the identity of the object is not trash, contacting the user to arrange for return of the object to the user; and

if the identity of the object is trash, returning the vehicle to a service station associated with a provider of the transportation service to enable removal of the object from the vehicle.

10. The method of claim 9 , further comprising, if the owner does not want the object returned, returning the vehicle to the service station to enable removal of the object from the vehicle.

11. The method of claim 9 , further comprising, if the user is a passenger in the vehicle, alerting the user to the location of the object before the transportation service ends and the user exits the vehicle.

12. The method of claim 11 , wherein the user is alerted using at least one of a mobile device associated with the user, a display provided in the vehicle, and a speaker provided in the vehicle.

13. The method of claim 11 , further comprising:

determining whether the object has been returned to the user; and

after the object has been returned to the owner, updating a lost object management database to indicate the object has been returned to the owner.

14. The method of claim 13 , wherein the determining is performed using the sensory data from the at least one interior sensor.

15. The method of claim 9 , further comprising:

if the user is not a passenger in the vehicle, retaining the object at a central location and inviting the user to retrieve the object from the central location; and

after the owner retrieves the object from the central location, updating a lost object management database to indicate that the object has been retrieved.

16. A lost object tracking (LOT) system for an autonomous vehicle (AV) used to provide a transportation service, the LOT system comprising:

an interior sensor provided in a cabin of the AV;

a LOT control module that tracks a location of an object on a surface of interest within the cabin during the transportation service using sensory data from the interior sensor when the object is not occluded from a view of the interior sensor and infers, based on a tilt and a speed of the AV, the location of the object when the object is occluded from the view of the interior sensor; and

a LOT communications module that identifies an owner of the object based on the transportation service and alerts the owner to retrieve the object, wherein the alert includes the inferred location of the object.

17. The LOT system of claim 16 , further comprising at least one auxiliary sensor that provides secondary sensory data used in the inferring the location of the object when the object is occluded from the view of the at least one interior sensor.

18. The LOT system of claim 17 , wherein the at least one auxiliary sensor comprises at least one of a weight sensor, a touch sensor, an ultrasonic sensor, and a laser sensor.

19. The LOT system of claim 17 , wherein the at least one auxiliary sensor is arranged such that sensory data from the at least one auxiliary sensor can be used to track the object when the object is occluded from the view of the at least one interior sensor.

20. The LOT system of claim 16 , wherein the at least one interior sensor comprises a camera.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 28, 2022
From: GRACE, NESTOR; WOLFENBERGER, KATELYN MARY; BAREKATAIN, HOOMAN; QIAN, DANYAN; KONG, WEICHAO
To: GM CRUISE HOLDINGS LLC
Reel/Frame 059414/0427 →
Continuity (1)
Related Publication 20230306754A1 · Sep 28, 2023
Cited By (1)
US 12,566,444