IP Library Granted Patent US 11,861,867
Granted Patent B2
US 11,861,867 · App. 17/625,572 · Granted Jan 2, 2024

Systems, devices and methods for measuring the mass of objects in a vehicle

Inventors: Yaniv Friedman (Tel-Aviv-Yafo, IL); Guy Trugman (Rishon Letsiyon, IL); Noam Nisenholz (Jerusalem, IL)
Assignee: GENTEX CORPORATION
G06T7/75B60R21/01512G01S7/4915G01S17/894G06T7/521G06T7/60G06V10/80G06V10/82G06V20/59G06V20/593G06V20/647G06V40/103H04N13/25H04N13/254G06T2200/04G06T2207/10021G06T2207/10028G06T2207/20212G06T2207/30268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,861,867
App. No.
17/625,572
Granted
Jan 2, 2024
Kind
B2
Abstract

There are provided methods and systems for estimating the mass of one or more occupants in a vehicle cabin comprising obtaining multiple images of the one or more occupants, comprising a sequence of 2D (two dimensional) images and 3D (three dimensional) images of the vehicle cabin captured by an image sensor, applying a pose detection algorithm on each of the obtained sequences of 2D images to yield one or more skeleton representations of said one or more occupants and combining the one or more 3D images of said sequence of 3D images with said one or more skeleton representations of said one or more occupants to yield skeleton models and analyze the skeleton models to extract one or more features of each of the one or more occupants and further process the one or more extracted features of the skeleton models to estimate the mass of the one or more occupants.

Claims (37)

1. A method for estimating mass of one or more occupants in a vehicle cabin, said method comprising:

providing a processor configured to:

obtain multiple images of said one or more occupants, wherein the multiple images comprising a sequence of two-dimensional (2D) images and three-dimensional (3D) images of said vehicle cabin captured by an image sensor;

apply a pose detection algorithm on each of the obtained sequences of the 2D images to yield one or more skeleton representations of said one or more occupants;

combine one or more of the 3D images of the sequence of the 3D images with the one or more skeleton representations of said one or more occupants to yield at least one skeleton model for each one of said one or more occupants wherein the at least one skeleton model comprises information relating to a distance of one or more key-points of the at least one skeleton model from a viewpoint;

analyze the at least one skeleton model to extract one or more features of each of said one or more occupants; and

process the one or more extracted features of the at least one skeleton model to estimate said mass of each said one or more occupants.

2. The method of claim 1 , wherein the processor is configured and enabled to filter out the at least one skeleton model based on predefined filtering criteria to yield valid skeleton models.

3. The method of claim 2 , wherein the predefined filtering criteria includes specific selection rules which define valid poses or orientations of said one or more occupants.

4. The method of claim 2 , wherein the predefined filtering criteria is based on one of:

a high density model; and

measured confidence grades of one or more key-points in the skeleton representations from the 2D images, wherein the confidence grades are based on a measured probability heat map of the one or more key-points.

5. The method of claim 1 wherein the processor is configured to generate one or more output signals comprising said estimated mass of each one of said one or more occupants, wherein the output signals are associated with an operation of one or more units of said vehicle cabin, and wherein the units of said vehicle cabin are selected from the group consisting of an airbag, an Electronic Stabilization Control (ESC) Unit, and a safety belt.

6. The method of claim 1 , wherein the sequence of 2D images are visual images of said vehicle cabin.

7. The method of claim 1 , wherein the sequence of 3D images are one or more of reflected light pattern images and stereoscopic images.

8. The method of claim 1 , wherein the image sensor is selected from a group consisting of Time of Flight (ToF) camera and a stereoscopic camera.

9. The method of claim 1 , wherein the pose detection algorithm is configured to identify a pose or orientation of said one or more occupants in the obtained 2D images.

10. The method of claim 1 , wherein the pose detection algorithm is configured to:

identify multiple key-points of one or more occupant body portions in at least one 2D image of the sequence of 2D images; and

link pairs of the detected multiple key points to generate the one or more skeleton representations of said one or more occupant in the 2D images, wherein the multiple key points are joints of a body of said one or more occupant.

11. The method of claim 1 , wherein the one or more extracted features are features of said one or more occupants including said one or more occupant's shoulder length, torso length, knee length, pelvis location, and hip width.

12. A system for estimating mass of one or more occupants in a vehicle cabin, said system comprising:

a sensing device comprising:

an illumination module comprising one or more illumination sources configured to illuminate said vehicle cabin;

at least one imaging sensor configured to capture a sequence of two-dimensional (2D) images and three-dimensional (3D) images of said vehicle cabin; and

at least one processor configured to:

apply a pose detection algorithm on each of the captured sequences of 2D images to yield one or more skeleton representations of said one or more occupants;

combine one or more of the 3D images of the sequence of 3D images with the one or more skeleton representations of said one or more occupants to yield at least one skeleton model for each one or more occupants, wherein the at least one skeleton model comprises information relating to a distance of one or more key-points in the at least one skeleton model from a viewpoint;

analyze the at least one skeleton model to extract one or more features of each of the one or more occupants; and

process the one or more extracted features of the at least one skeleton model to estimate said mass of each of said one or more occupants.

13. The system of claim 12 , wherein the processor is configured to filter out the at least one skeleton model based on predefined filtering criteria to yield valid skeleton models.

14. The system of claim 13 , wherein the predefined filtering criteria includes specific selection rules which define valid poses or orientations of said one or more occupants.

15. The system of claim 13 , wherein the predefined filtering criteria is based on measured confidence grades of one or more key points in the one or more skeleton representations, wherein the measured confidence grades are based on a measured probability heat map of the one or more key points.

16. The system of claim 13 , wherein the predefined filtering criteria is based on a high-density model.

17. The system of claim 12 , wherein the sensing device is selected from a group consisting of a Time of Flight (ToF) sensing device and a stereoscopic sensing device.

18. The system of claim 12 , wherein the sensing device is a structured light pattern sensing device and the one or more illumination sources is configured to project modulated light in a predefined structured light pattern on said vehicle cabin, wherein the predefined structured light pattern is constructed of a plurality of diffused light elements, and wherein a shape of the diffused light elements is one or more of a dot, a line, a stripe, or a combination thereof.

19. The system of claim 12 , wherein the processor is configured to generate one or more output signals comprising said estimated mass or body mass classification of each of said one or more occupants, wherein the output signals correspond to an operation of one or more units of said vehicle cabin, and wherein the units of said vehicle cabin are selected from a group consisting of an airbag, an Electronic Stabilization Control (ESC) Unit, and a safety belt.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2022
From: FRIEDMAN, YANIV; TRUGMAN, GUY; NISENHOLZ, NOAM
To: GUARDIAN OPTICAL TECHNOLOGIES, LTD.
Reel/Frame 058595/0606 →
Continuity (2)
Provisional Application 62871787 · Jul 9, 2019
Related Publication 20220292705A1 · Sep 15, 2022