IP Library › Granted Patent US 11,861,917
Granted Patent B2
US 11,861,917 · App. 17/876,645 · Granted Jan 2, 2024

Apparatuses, systems, and methods for detecting vehicle occupant actions

Inventors: Aaron Scott Chan (Lisle, IL); Kenneth J. Sanchez (San Francisco, CA)
Assignee: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANIES
G06V20/597B60W40/08G06F16/50G06F18/2415G06N7/01G06T7/20G06V10/751G06V20/59G06V40/20G06Q40/08H04N7/181H04N23/90
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,861,917
App. No.
17/876,645
Granted
Jan 2, 2024
Kind
B2
Abstract

Method and system for detecting vehicle occupant actions are disclosed. For example, the method includes receiving, by one or more processors, previously classified image data from a previously classified image database, the previously classified image data representing driver postures that are rotated and scaled to be standardized for a range of different drivers and different locations of a vehicle interior sensor within a given vehicle, wherein the previously classified image data is representative of previously classified vehicle occupant actions; receiving, by the one or more processors, current image data from the vehicle interior sensor subsequent to the vehicle interior sensor being registered within the given vehicle, wherein the current image data is representative of current vehicle occupant actions; and determining, by the one or more processors, a vehicle occupant action based at least in part upon a comparison of the current image data and the previously classified image data.

Claims (49)

1. A system for detecting vehicle occupant actions, the system comprising:

one or more processors; and

a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to:

receive previously classified image data from a previously classified image database, the previously classified image data representing driver postures that are rotated and scaled to be standardized for a range of different drivers and different locations of a vehicle interior sensor within a vehicle, wherein the previously classified image data is representative of previously classified vehicle occupant actions;

receive current image data from the vehicle interior sensor subsequent to the vehicle interior sensor being registered within the vehicle, wherein the current image data is representative of current vehicle occupant actions; and

determine a vehicle occupant action based at least in part upon a comparison of the current image data and the previously classified image data.

2. The system of claim 1 , wherein:

the current image data is representative of a two-dimensional representation of at least one occupant within the vehicle;

the vehicle occupant action is representative of an associated behavior; and

the associated behavior with a higher severity includes a degree of motion by the at least one occupant that is larger than a degree of motion included in the associated behavior with a lower severity.

3. The system of claim 1 , wherein the previously classified image data is representative of a two-dimensional representation of at least one occupant within the vehicle.

4. The system of claim 1 , wherein the previously classified image data is representative of a three-dimensional representation of at least one occupant within the vehicle.

5. The system of claim 1 , wherein:

the current image data is representative of a three-dimensional representation of at least one occupant within the vehicle,

the vehicle occupant action is representative of an associated behavior; and

the associated behavior with a higher severity includes a degree of motion by the at least one occupant that is larger than a degree of motion included in the associated behavior with a lower severity.

6. The system of claim 1 , wherein the current image data includes images that are representative of a vehicle occupant using a cellular telephone, a vehicle occupant looking out a vehicle side window, a vehicle occupant adjusting a vehicle radio, a vehicle occupant adjusting a vehicle heating, ventilation and air conditioning system, two vehicle occupants talking with one another, a vehicle occupant reading a book or magazine, a vehicle occupant putting on makeup, or a vehicle occupant looking at themselves in a mirror.

7. The system of claim 1 , wherein the previously classified image data includes images that have previously been classified as being representative of a vehicle occupant using a cellular telephone, a vehicle occupant looking out a vehicle side window, a vehicle occupant adjusting a vehicle radio, a vehicle occupant adjusting a vehicle heating, ventilation and air conditioning system, two vehicle occupants talking with one another, a vehicle occupant reading a book or magazine, a vehicle occupant putting on makeup, or a vehicle occupant looking at themselves in a mirror.

8. A computer-implemented method for detecting vehicle occupant actions, the method comprising:

receiving, by one or more processors, previously classified image data from a previously classified image database, the previously classified image data representing driver postures that are rotated and scaled to be standardized for a range of different drivers and different locations of a vehicle interior sensor within a vehicle, wherein the previously classified image data is representative of previously classified vehicle occupant actions;

receiving, by the one or more processors, current image data from the vehicle interior sensor subsequent to the vehicle interior sensor being registered within the vehicle, wherein the current image data is representative of current vehicle occupant actions; and

determining, by the one or more processors, a vehicle occupant action based at least in part upon a comparison of the current image data and the previously classified image data.

9. The method of claim 8 , wherein:

the current image data is representative of a two-dimensional representation of at least one occupant within the vehicle;

wherein the vehicle occupant action is representative of an associated behavior; and

the associated behavior with a higher severity includes a degree of motion by the at least one occupant that is larger than a degree of motion included in the associated behavior with a lower severity.

10. The method of claim 8 , wherein:

the current image data is representative of a three-dimensional representation of at least one occupant within the vehicle;

wherein the vehicle occupant action is representative of an associated behavior; and

the associated behavior with a higher severity includes a degree of motion by the at least one occupant that is larger than a degree of motion included in the associated behavior with a lower severity.

11. The method of claim 8 , wherein the previously classified image data is representative of a three-dimensional representation of at least one occupant within the vehicle.

12. The method of claim 8 , wherein the previously classified image data is representative of a two-dimensional representation of at least one occupant within the vehicle.

13. The method of claim 8 , wherein the current image data includes images that are representative of vehicle occupant locations/orientations, cellular telephone locations/orientations, vehicle occupant eye locations/orientations, vehicle occupant head location/orientation, vehicle occupant hand location/orientation, a vehicle occupant torso location/orientation, a seat belt location, or a vehicle seat location/orientation.

14. The method of claim 8 , wherein the previously classified image data includes images that have previously been classified as being representative of known vehicle occupant locations/orientations, known cellular telephone locations/orientations, known vehicle occupant eye locations/orientations, known vehicle occupant head location/orientation, known vehicle occupant hand location/orientation, a known vehicle occupant torso location/orientation, a known seat belt location, or a known vehicle seat location/orientation.

15. A non-transitory computer-readable media storing instructions for detecting vehicle occupant actions, the instructions when executed by one or more processors of a computing device, cause the computing device to:

receive previously classified image data from a previously classified image database, the previously classified image data representing driver postures that are rotated and scaled to be standardized for a range of different drivers and different locations of a vehicle interior sensor within a vehicle, wherein the previously classified image data is representative of previously classified vehicle occupant actions;

receive current image data from the vehicle interior sensor subsequent to the vehicle interior sensor being registered within the vehicle, wherein the current image data is representative of current vehicle occupant actions; and

determine a vehicle occupant action based at least in part upon a comparison of the current image data and the previously classified image data.

16. The non-transitory computer-readable media of claim 15 , wherein:

the previously classified image data is representative of a two-dimensional representation of at least one occupant within the vehicle;

wherein the vehicle occupant action is representative of an associated behavior; and

the associated behavior with a higher severity includes a degree of motion by the at least one occupant that is larger than a degree of motion included in the associated behavior with a lower severity.

17. The non-transitory computer-readable media of claim 15 , wherein the current image data is representative of a three-dimensional representation of at least one occupant within the vehicle.

18. The non-transitory computer-readable media of claim 15 , wherein the current image data includes images that are representative of a vehicle occupant using a cellular telephone, a vehicle occupant looking out a vehicle side window, a vehicle occupant adjusting a vehicle radio, a vehicle occupant adjusting a vehicle heating, ventilation and air conditioning system, two vehicle occupants talking with one another, a vehicle occupant reading a book or magazine, a vehicle occupant putting on makeup, or a vehicle occupant looking at themselves in a mirror, vehicle occupant locations/orientations, cellular telephone locations/orientations, vehicle occupant eye locations/orientations, vehicle occupant head location/orientation, vehicle occupant hand location/orientation, a vehicle occupant torso location/orientation, a seat belt location, or a vehicle seat location/orientation.

19. The non-transitory computer-readable media of claim 15 , wherein the previously classified image data includes images that have previously been classified as being representative of a vehicle occupant using a cellular telephone, a vehicle occupant looking out a vehicle side window, a vehicle occupant adjusting a vehicle radio, a vehicle occupant adjusting a vehicle heating, ventilation and air conditioning system, two vehicle occupants talking with one another, a vehicle occupant reading a book or magazine, a vehicle occupant putting on makeup, a vehicle occupant looking at themselves in a mirror, known vehicle occupant locations/orientations, known cellular telephone locations/orientations, known vehicle occupant eye locations/orientations, known vehicle occupant head location/orientation, known vehicle occupant hand location/orientation, a known vehicle occupant torso location/orientation, a known seat belt location, or a known vehicle seat location/orientation.

20. The non-transitory computer-readable media of claim 15 , wherein:

the previously classified image data is representative of a three-dimensional representation of at least one occupant within the vehicle;

wherein the vehicle occupant action is representative of an associated behavior; and

the associated behavior with a higher severity includes a degree of motion by the at least one occupant that is larger than a degree of motion included in the associated behavior with a lower severity.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2023
From: CHAN, AARON SCOTT; SANCHEZ, KENNETH J.
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 063343/0025 →
Continuity (2)
Continuation 15181673 · Jun 14, 2016
Related Publication 20220375265A1 · Nov 24, 2022
Cited By (2)
US 12,583,405 US 12,662,150