IP Library Granted Patent US 11,361,574
Granted Patent B2
US 11,361,574 · App. 16/661,930 · Granted Jun 14, 2022

System and method for monitoring for driver presence and position using a driver facing camera

Inventors: Cathy L. Boon (Irvine, CA); Zheng Li (Irvine, CA); Hans M. Molin (Irvine, CA)
Assignee: Bendix Commercial Vehicle Systems LLC
G06V40/103G06V10/25G06V10/44G06V20/593G06V20/597B60K28/02B60R11/04B60R2011/0003
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,361,574
App. No.
16/661,930
Granted
Jun 14, 2022
Kind
B2
Abstract

A method for determining a presence of an occupant within a vehicle includes capturing, via a driver facing camera, video image data, including a plurality of image frames of a field-of-view of the driver facing camera. At least one area-of-interest within the image frames is defined, and a foreground pixel count of each image frame of each area-of-interest is determined. At each Nth image frame, an occupancy for each area of interest is determined based at least in part on the foreground pixel count. The occupancy indicates whether the occupant is present or absent in the respective area-of-interest. A history of the occupancy determination for each area-of-interest is maintained, and a final status for each area-of-interest is determined based on the respective history of occupancy. External indicators are initiated based on the final status determination.

Claims (42)

1. A method for determining a presence of an occupant within a vehicle, comprising:

capturing, via a driver facing camera, video image data, including a plurality of image frames of a field-of-view of the driver facing camera;

defining at least one area-of-interest within the image frames;

determining a foreground pixel count of each image frame of each area-of-interest by counting the number of pixels in the foreground of the image frame;

determining, at each Nth image frame, an occupancy for each area-of-interest based at least in part on the foreground pixel count, the occupancy indicating whether the occupant is present or absent in the respective area-of-interest;

maintaining a history of the occupancy determination for each area-of-interest; and

determining a final status for each area-of-interest based on the respective history of occupancy.

2. The method of claim 1 , further comprising:

determining an edge pixel count of each image frame of each area-of-interest,

wherein the determination of the occupancy is also based on the edge pixel count.

3. The method of claim 2 , wherein determining the edge pixel count is via application of a Sobel operator to the image frame.

4. The method of claim 1 , wherein determining the foreground pixel count is via frame-to-frame differencing.

5. The method of claim 1 , wherein every Nth image frame is from every 5 image frames to every 20 image frames.

6. The method of claim 1 , wherein defining the area-of-interest includes generating an area-of-interest mask, and wherein the foreground pixel count is determined based on a logical AND operation applied to the area-of-interest mask and a foreground pixel identification.

7. The method of claim 1 , wherein defining the area-of-interest includes generating an area-of-interest mask, and wherein the edge pixel count is determined based on a logical AND operation applied to the area-of-interest mask and an edge pixel identification.

8. The method of claim 1 , wherein determining the foreground pixel count of each area-of-interest includes tracking an observed characteristic value of each pixel within each area-of-interest for significant changes over a predetermined period.

9. The method of claim 1 , further comprising:

initiating external indicators and/or vehicle functions based on the final status determination.

10. The method of claim 9 , wherein at least one of the vehicle functions initiated is an anti-rollaway function.

11. The method of claim 9 , wherein the final status of each area-of-interest is determined independently, but wherein the external indicators and/or vehicle functions are initiated based on a combined consideration of the respective final statuses of a plurality of areas-of-interest.

12. The method of claim 11 , wherein at least one of the vehicle functions initiated is an anti-rollaway function.

13. A system for determining a presence of an occupant within a vehicle, comprising:

a driver facing camera configured to capture video image data, including a plurality of image frames of a field-of-view;

a mask module configured to define at least one area-of-interest within the image frames;

a difference module configured to determine a foreground pixel count of each image frame of each area-of-interest by counting the number of pixels in the foreground of the image frame; and

a status module configured to:

determine, at each Nth image frame, an occupancy for each area-of-interest based at least in part on the foreground pixel count, the occupancy indicating whether the occupant is present or absent in the respective area-of-interest,

maintain a history of the occupancy determination for each area-of-interest, and

determine a final status for each area-of-interest based on the respective history of occupancy.

14. The system of claim 13 , further comprising:

an edge extraction module configured to determine an edge pixel count of each image frame of each area-of-interest, wherein the determination of the occupancy is also based on the edge pixel count.

15. The system of claim 14 , wherein determining the edge pixel count is via application of a Sobel operator to the image frame.

16. The system of claim 13 , wherein determining the foreground pixel count is via frame-to-frame differencing.

17. The system of claim 13 , wherein every Nth image frame is from every 5 image frames to every 20 image frames.

18. The system of claim 13 , wherein defining the area-of-interest includes generating an area-of-interest mask, and wherein the foreground pixel count is determined based on a logical AND operation applied to the area-of-interest mask and a foreground pixel identification.

19. The system of claim 13 , wherein defining the area-of-interest includes generating an area-of-interest mask, and wherein the edge pixel count is determined based on a logical AND operation applied to the area-of-interest mask and an edge pixel identification.

20. The system of claim 13 , wherein determining the foreground pixel count of each area-of-interest includes tracking an observed characteristic value of each pixel within each area-of-interest for significant changes over a predetermined period.

21. The system of claim 13 , further comprising:

a controller configured to initiate external indicators and/or vehicle functions based on the final status determination.

22. The system of claim 21 , wherein at least one of the vehicle functions initiated is an anti-rollaway function.

23. The system of claim 21 , wherein the final status of each area-of-interest is determined independently, but wherein the external indicators and/or vehicle functions are initiated based on a combined consideration of the respective final statuses of a plurality of areas-of-interest.

24. The system of claim 23 , wherein at least one of the vehicle functions initiated is an anti-rollaway function.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2024
From: BENDIX COMMERCIAL VEHICLE SYSTEMS LLC
To: RM ACQUISITION, LLC
Reel/Frame 067824/0004 →
CHANGE OF ADDRESS Recorded May 9, 2022
From: BOON, CATHY L.; LI, ZHENG; MOLIN, HANS M.
To: BENDIX COMMERCIAL VEHICLE SYSTEMS LLC
Reel/Frame 059911/0365 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 26, 2021
From: BOON, CATHY L.; LI, ZHENG; MOLIN, HANS M.
To: BENDIX COMMERCIAL VEHICLE SYSTEMS LLC
Reel/Frame 056360/0601 →
Continuity (1)
Related Publication 20210124916A1 · Apr 29, 2021