IP Library Granted Patent US 12,307,792
Granted Patent B2
US 12,307,792 · App. 17/781,586 · Granted May 20, 2025

Abnormal physical condition determination system, abnormal physical condition determination method, and computer program

Inventor: Yuta Shimizu (Tokyo, JP)
Assignee: NEC CORPORATION
G06V20/597B60W40/08F16C32/0465F25B31/026G06T7/73G06V10/62G06V10/74G06V40/171B60W60/0016B60W2420/403B60W2540/223G06T2207/30201G06T2207/30268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,307,792
App. No.
17/781,586
Granted
May 20, 2025
Kind
B2
Abstract

An abnormal physical condition determination system includes: an extraction unit that extracts a plurality of feature quantities indicating a condition of a target person from an image of the target person; an accumulation unit that accumulates the plurality of feature quantities as time series data; a calculation unit that calculates a relationship between each feature quantity from the plurality of feature quantities accumulated in the accumulation unit; and a determination unit that determines an abnormal physical condition of the target person on the basis of the relationship. According to such an abnormal physical condition determination system, it is possible to appropriately determine the abnormal physical condition of the target person.

Claims (35)

1. An abnormal physical condition determination system comprising:

at least one memory storing instructions; and

at least one processor configured to execute instructions to:

extract a plurality of feature quantities indicating a condition of a target person from an image of the target person;

accumulate the plurality of feature quantities as time series data in the memory;

calculate a relationship between each feature quantity from the plurality of feature quantities accumulated in the memory; and

determine an abnormal physical condition of the target person on the basis of the relationship, wherein

the relationship includes at least one of correlations between (i) a face position and a position of a pupil, (ii) a face pan direction and a gaze horizontal direction, and (iii) a face tilt direction and a gaze vertical direction, and

the abnormal physical condition of the target person is determined when at least one or more of a plurality of conditions are satisfied, including

(a) when the face position and the pupil position do not change in the same manner, (b) when the face pan direction and the gaze horizontal direction do not change in the same manner, and (c) when the face tilt direction and the gaze vertical direction do not change in the same manner.

2. The abnormal physical condition determination system according to claim 1 , wherein the processor calculates the relationship from a degree of a change per unit time in the plurality of feature quantities.

3. The abnormal physical condition determination system according to claim 1 , wherein the processor calculates a correlation or a ratio between different feature quantities as the relationship.

4. The abnormal physical condition determination system according to claim 1 , wherein the plurality of feature quantities includes feature quantities related to a position of a pupil of the target person, a direction of a face, or a gaze direction.

5. The abnormal physical condition determination system according to claim 1 , wherein the processor determines and outputs a degree of the abnormal physical condition on the basis of the relationship.

6. The system for determining abnormal physical condition according to claim 1 , wherein the processor is further configured to present information about the abnormal physical condition of the target person corresponding to a determination result of the processor, to at least one of the target person and a person who is around the target person.

7. The abnormal physical condition determination system according to claim 1 , wherein

the target person is an occupant of a vehicle, and

the processor is further configured to output a control information for controlling the vehicle in accordance with the abnormal physical condition, on the basis of a determination result of the processor.

8. The abnormal physical condition determination system according to claim 1 , wherein the abnormal physical condition is a motion sickness.

9. An abnormal physical condition determination method performed by a computer and comprising:

extracting a plurality of feature quantities indicating a condition of a target person from an image of the target person;

accumulating the plurality of feature quantities as time series data;

calculating a relationship between each feature quantity from the plurality of feature quantities accumulated in the accumulation unit; and

determining an abnormal physical condition of the target person on the basis of the relationship, wherein

the relationship includes at least one of correlations between (i) a face position and a position of a pupil, (ii) a face pan direction and a gaze horizontal direction, and (iii) a face tilt direction and a gaze vertical direction, and

the abnormal physical condition of the target person is determined when at least one or more of a plurality of conditions are satisfied, including

(a) when the face position and the pupil position do not change in the same manner, (b) when the face pan direction and the gaze horizontal direction do not change in the same manner, and (c) when the face tilt direction and the gaze vertical direction do not change in the same manner.

10. A non-transitory recording medium storing a computer program executable by a computer to perform an abnormal physical condition determination method comprising:

extracting a plurality of feature quantities indicating a condition of a target person from an image of the target person;

accumulating the plurality of feature quantities as time series data;

calculating a relationship between each feature quantity from the plurality of feature quantities accumulated in the accumulation unit; and

determining an abnormal physical condition of the target person on the basis of the relationship, wherein

the relationship includes at least one of correlations between (i) a face position and a position of a pupil, (ii) a face pan direction and a gaze horizontal direction, and (iii) a face tilt direction and a gaze vertical direction, and

the abnormal physical condition of the target person is determined when at least one or more of a plurality of conditions are satisfied, including

(a) when the face position and the pupil position do not change in the same manner, (b) when the face pan direction and the gaze horizontal direction do not change in the same manner, and (c) when the face tilt direction and the gaze vertical direction do not change in the same manner.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2022
From: SHIMIZU, YUTA
To: NEC CORPORATION
Reel/Frame 060071/0488 →
Continuity (1)
Related Publication 20230005279A1 · Jan 5, 2023
References Cited (31)
US 11404075B1 · Lakhani · 2022 [cited by examiner]
US 11919522B2 · Yamauchi · 2024 [cited by examiner]
US 20140121927A1 · Hanita · 2014 [cited by examiner]
US 20170124383A1 · Ohbitsu · 2017 [cited by applicant]
US 20190110729A1 · Yamataka · 2019 [cited by examiner]
US 20190135325A1 · Lisseman · 2019 [cited by examiner]
US 20200249764A1 · Page · 2020 [cited by examiner]
US 20200334477A1 · Aoi · 2020 [cited by examiner]
US 20200337623A1 · Bulut · 2020 [cited by examiner]
US 20200385025A1 · Nishimura et al. · 2020 [cited by applicant]
US 20210031789A1 · Moriura et al. · 2021 [cited by applicant]
US 20220076039A1 · Li · 2022 [cited by examiner]
US 20220313132A1 · Tsujikawa · 2022 [cited by examiner]
EP 3484874A1 · 2019 [cited by applicant]
EP 3494874A1 · 2019 [cited by examiner]
JP 2004301692A · 2004 [cited by applicant]
JP 2006034576A · 2006 [cited by applicant]
JP 2015141099A · 2015 [cited by applicant]
JP 2006040055A · 2016 [cited by applicant]
JP 2018160136A · 2018 [cited by applicant]
WO 2013008300A1 · 2013 [cited by applicant]
WO 2016013090A1 · 2016 [cited by applicant]
WO 2017208529A1 · 2017 [cited by applicant]
WO 2017209225A1 · 2017 [cited by applicant]
WO 2019177002A1 · 2019 [cited by applicant]
JP Office Communication for JP Application No. 2023-092541, mailed on Apr. 30, 2024 with English Translation. [cited by applicant]
International Search Report for PCT Application No. PCT/JP2019/047514, mailed on Feb. 10, 2020. [cited by applicant]
Nakai, Rena, “Imaging technology used in automobiles, Hight accuracy and low-cost driver and cabin monitoring realized by the latest deep learning”, Eizojoho Industrial, Aug. 1, 2019, vol. 51. No. 4, pp. 21-23. [cited by applicant]
Wada, T. et al., “Analysis of driver's head tilt using a mathematical model of motion sickness”, International Journal of Industrial Ergonomics, Jan. 2018, vol. 63, pp. 89-97, in particular, p. 96, left column lines 53-… [cited by applicant]
JP Office Action for JP Application No. 2024-087375, mailed on Jan. 21, 2025 with English Translation. [cited by applicant]
US Office Communication for U.S. Appl. No. 18/500,436, mailed on Mar. 13, 2025. [cited by applicant]