IP Library › Granted Patent US 10,210,387
Granted Patent B2
US 10,210,387 · App. 15/585,764 · Granted Feb 19, 2019

Method and apparatus for detecting and classifying objects associated with vehicle

Inventors: David Clifford (Royal Oak, MI); Orhan Demirovic (Sterling Heights, MI); Orhan Bulan (Troy, MI); Eric Hatfield (Clinton Township, MI); David Hiemenga (Grand Rapids, MI); Mohannad Murad (Troy, MI)
G06K9/00536B60R1/00G06K9/6202G06K9/6215G06T7/11G06T7/70H04N7/183B60R2300/30B60R2300/8006G06T2207/10016G06T2207/30268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,210,387
App. No.
15/585,764
Granted
Feb 19, 2019
Kind
B2
Abstract

A method and apparatus for detecting and classifying objects associated with a vehicle are provided. The method includes capturing a first image and a second image of a cabin of a vehicle, dividing the first image and the second image into a plurality of regions, comparing a first region of the first image to a second region of the second image, in response to detecting a difference between the second region and the first region, classifying an object present in the difference between the second region and the first region and labeling the classified object, and providing a notification of the classified object to at least one from among an occupant of the vehicle and an operator of the vehicle.

Claims (43)

1. A method for detecting and classifying objects associated with a vehicle, the method comprising:

prior to an occupant entering the vehicle, capturing a first image of a cabin of the vehicle;

after the occupant exits the vehicle, capturing a second image of the cabin of the vehicle;

dividing the first image and the second image into a plurality of regions;

comparing a first region of the first image to a second region of the second image, the second region of the second image corresponding to the first region in the first image;

in response to detecting a difference between the second region and the first region, classifying an object present in the difference between the second region and the first region and labeling the classified object; and

providing a notification of the classified object to at least one from among an occupant of the vehicle and an operator of the vehicle.

2. The method of claim 1 , further comprising:

receiving an input to re-label the classified object from the operator of the vehicle; and

re-classifying the object present in the difference between the second region and the first region based on the received input.

3. The method of claim 1 , wherein the plurality of regions comprise two or more from among a front passenger seat, a front driver side, a rear left seat, a rear right seat, a rear seat area, a front passenger floor, a front driver floor, a rear left floor, a rear right floor, a rear floor, a dashboard, a cup holder, a center console, a trunk area and a surface adjacent to rear window.

4. The method of claim 1 , wherein the providing the notification of the detected difference comprises transmitting information on the second image or the classified object to an operator.

5. The method of claim 1 , wherein the capturing the first image and the second image of the cabin of the vehicle is performed by a camera.

6. The method of claim 1 , wherein the comparing the first region of the first image to the second region of the second image comprises detecting a change in pixel values in between the second region and the first region.

7. The method of claim 1 , wherein the classifying the object present in the difference between the second region and the first region comprises comparing an image of the object to images of objects stored in a database and classifying the object according to a classification of an image stored in the database that is closest to the image of the object.

8. The method of claim 1 , wherein the classifying the object in present in the difference between the second region and the first region comprises identifying the object by performing at least one from among edge matching, greyscale matching and gradient matching.

9. A non-transitory computer readable medium comprising computer instructions executable by a computer to perform the method of claim 1 .

10. An apparatus that detects and classifies objects associated with a vehicle, the apparatus comprising:

at least one memory comprising computer executable instructions; and

at least one processor configured to read and execute the computer executable instructions, the computer executable instructions causing the at least one processor to:

prior to an occupant entering a vehicle, capture a first image of a cabin of the vehicle;

after the occupant exits the vehicle, capture a second image of the cabin of the vehicle;

divide the first image and the second image into a plurality of regions;

compare a first region of the first image to a second region of the second image, the second region of the second image corresponding to the first region in the first image;

in response to detecting a difference between the second region and the first region, classify an object present in the difference between the second region and the first region and labeling the classified object; and

provide a notification of the classified object to at least one from among an occupant of the vehicle and an operator of the vehicle.

11. The apparatus of claim 10 , wherein the computer executable instructions cause the at least one processor to:

receive an input to re-label the classified object from the operator of the vehicle; and

reclassify the object present in the difference between the second region and the first region based on the received input.

12. The apparatus of claim 10 , wherein the plurality of regions comprise two or more from among a front passenger seat, a front driver side, a rear left seat, a rear right seat, a rear seat area, a front passenger floor, a front driver floor, a rear left floor, a rear right floor, a rear floor, a dashboard, a cup holder, a center console, a trunk area and a surface adjacent to rear window.

13. The apparatus of claim 10 , wherein the computer executable instructions further cause the at least one processor to provide the notification of the detected difference by transmitting information on the second image or the classified object to an operator.

14. The apparatus of claim 10 , further comprising a camera mounted in the vehicle,

wherein the computer executable instructions further cause the at least one processor to control the camera to capture the first image and the second image.

15. The apparatus of claim 14 , wherein the computer executable instructions further cause the at least one processor to compare the first region of the first image to the second region of the second image by detecting a change in pixel values in between the second region and the first region.

16. The apparatus of claim 10 , wherein the computer executable instructions further cause the at least one processor to classify the object present in the difference between the second region and the first region by comparing an image of the object to images of objects stored in a database and classifying the object according to a classification of an image stored in the database that is closest to the image of the object.

17. The apparatus of claim 10 , wherein the computer executable instructions further cause the at least one processor to classify the object in present in the difference between the second region and the first region by identifying the object by performing at least one from among edge matching, greyscale matching and gradient matching.

18. A non-transitory computer readable medium comprising computer readable instructions, the computer readable instructions executable by a computer to perform a method comprising:

receiving a first image of a cabin of a vehicle, the first image captured prior to an occupant entering the vehicle;

receiving a second image of the cabin of the vehicle, the second image captured after the occupant exits the vehicle;

dividing the first image and the second image into a plurality of regions;

comparing a first region of the first image to a second region of the second image, the second region of the second image corresponding to the first region in the first image;

in response to detecting a difference between the second region and the first region, classifying an object present in the difference between the second region and the first region and labeling the classified object; and

providing a notification of the classified object to at least one from among an occupant of the vehicle and an operator of the vehicle.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 19, 2017
From: MURAD, MOHANNAD
To: GM GLOBAL TECHNOLOGY OPERATIONS LLC
Reel/Frame 042747/0052 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 3, 2017
From: CLIFFORD, DAVID; DEMIROVIC, ORHAN; BULAN, ORHAN; HATFIELD, ERIC; HIEMENGA, DAVID
To: GM GLOBAL TECHNOLOGY OPERATIONS LLC
Reel/Frame 042227/0990 →
Continuity (1)
Related Publication 20180322342A1 · Nov 8, 2018
Cited By (1)
US 12,602,769