IP Library Granted Patent US 10,140,532
Granted Patent B2
US 10,140,532 · App. 14/111,440 · Granted Nov 27, 2018

Method and device for monitoring at least one vehicle occupant, and method for operating at least one assistance device

Inventors: Galia Weidl (Boeblingen, DE); Ralf Herrtwich (Kleinmachnow, DE); Michael Schrauf (Esslingen, DE)
Assignee: Daimler AG
G06K9/00845A61B5/02416A61B5/18G06K9/00832B60W2040/0818B60W2040/0872
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,140,532
App. No.
14/111,440
Granted
Nov 27, 2018
Kind
B2
Abstract

A method and device for monitoring at least one vehicle passenger in a vehicle involves capturing images of the vehicle passenger using an image capturing unit and analyzing the captured images using an image processing unit. At least one vital sign of the vehicle passenger is determined by the image analysis of the captured images, which can be used to operate at least one assistance device of a vehicle.

Claims (37)

1. A method for monitoring at least one vehicle passenger in a vehicle, the method comprising:

capturing images of the vehicle passenger using at least one camera directed toward a first bodily region;

analyzing the captured images of the first bodily region using at least one processor;

determining a first vital sign and a second vital sign different from the first vital sign of the vehicle passenger, the determining of each of the first and second vital signs being based on the image analysis of the captured images;

determining, based on the determined first vital sign and the second vital sign, one or more diagnostic values for: (i) tachycardia or bradycardia, (ii) heart rate variability, and (iii) tendency related to viral infection; and

recommending a suitable countermeasure based at least on the determined one or more diagnostic values,

wherein the determined first vital sign is a pulse rate and the second vital sign is a breathing rate, and

wherein the first vital sign and the second vital sign are both determined from the same captured images of the first bodily region so that it is unnecessary to move or rotate the at least one camera toward a second bodily region to determine the second vital sign.

2. The method according to claim 1 , wherein one or more of the first and second vital signs is determined by

detecting an optical absorption value or an intensity value of reflected light of a skin region of the vehicle passenger, or

calculating a change of the optical absorption value or of the intensity value.

3. The method according to claim 1 , wherein during image processing for analyzing the captured images, at least one region suited to image analysis is looked for in each of the captured images and only the at least one region suited to image analysis is analyzed.

4. The method according to claim 3 , wherein the at least one region suited to image analysis contains a face of the vehicle passenger.

5. The method according to claim 3 , wherein the captured images of the vehicle passenger are color, wherein the region of the images suited to image analysis is split into individual color channels and at least one of the individual color channels is analyzed to determine one or more of the first and second vital signs.

6. The method according to claim 5 , wherein a spectral power density of the at least one of the individual color channels of the region suited to image analysis is determined using a Fast-Fourier Transform.

7. A device for monitoring at least one vehicle passenger in a vehicle, the device comprising:

at least one camera directed toward a first bodily region and configured to capture images of the vehicle passenger; and

at least one processor executing stored instructions to:

analyze the captured images of the first bodily region,

determine a first vital sign and a second vital sign different from the first vital sign of the vehicle passenger, the determination of each of the first and second vital signs being based on image analysis of the captured images,

determine, based on the determined first vital sign and the second vital sign, one or more diagnostic values for: (i) tachycardia or bradycardia, (ii) heart rate variability, and (iii) tendency related to viral infection, and

recommend a suitable countermeasure based at least on the determined one or more diagnostic values,

wherein the determined first vital sign is a pulse rate and the second vital sign is a breathing rate, and

wherein the first vital sign and the second vital sign are both determined from the same captured images of the first bodily region so that it is unnecessary to move or rotate the at least one camera toward a second bodily region to determine the second vital sign.

8. A method, comprising:

capturing images of a vehicle passenger in a vehicle using at least one camera directed toward a first bodily region;

analyzing the captured images of the first bodily region using at least one processor;

determining a first vital sign and a second vital sign different from the first vital sign of the vehicle passenger, the determining of each of the first and second vital signs being based on the image analysis of the captured images;

determining, based on the determined first vital sign and the second vital sign, one or more diagnostic values for: (i) tachycardia or bradycardia, (ii) heart rate variability, and (iii) tendency related to viral infection; and

activating at least one function of an assistance device if a determined value of one or more of the first and second vital signs deviates from a predetermined value or value region and recommending a suitable countermeasure based at least on the determined one or more diagnostic values,

wherein the determined first vital sign is a pulse rate and the second vital sign is a breathing rate, and

wherein the first vital sign and the second vital sign are both determined from the same captured images of the first bodily region so that it is unnecessary to move or rotate the at least one camera toward a second bodily region to determine the second vital sign.

9. The method according to claim 8 , wherein an intervention in a steering device, braking device, or in a drive train of the vehicle, an optical, audible, or haptic warning device, or at least one comfort function or an emergency function is activated as a function of the assistance device.

10. The method according to claim 1 , further comprising the step of distinguishing whether the first vital sign indicates calm, stress, or elation based on the one or more diagnostic values related to the heart rate variability.

11. The method according to claim 1 , wherein the suitable countermeasure includes one or more of the following: (i) taking a refreshment break, (ii) moving regularly, (iii) taking sufficient fluid, and (iv) taking required medication.

12. The method according to claim 1 , further comprising activating at least one function of one or more devices of the vehicle based on the determined first vital sign and the second vital sign.

13. The method according to claim 1 , further comprising providing the recommendation of the suitable countermeasure to the at least one vehicle passenger of the vehicle via one or more devices of the vehicle and the at least one vehicle passenger acting on the recommendation based on the recommendation provided by the one or more devices of the vehicle.

Assignments (2)
CHANGE OF NAME Recorded Jul 29, 2025
From: DAIMLER AG
To: MERCEDES BENZ GROUP AG
Reel/Frame 072886/0467 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 24, 2013
From: WEIDL, GALIA; HERRTWICH, RALF; SCHRAUF, MICHAEL
To: DAIMLER AG
Reel/Frame 031845/0900 →
Priority Claims (1)
DE 10 2011 016 772 · Apr 12, 2011 · national
Continuity (1)
Related Publication 20140104405A1 · Apr 17, 2014