IP Library › Granted Patent US 11,308,325
Granted Patent B2
US 11,308,325 · App. 16/654,868 · Granted Apr 19, 2022

Systems and methods for predicting real-time behavioral risks using everyday images

Inventors: Francis McClernon (Durham, NC); Matthew Engelhard (Durham, NC); Lawrence Carin (Durham, NC); Jason Oliver (Durham, NC)
Assignee: Duke University
G06K9/00671G06K9/6282G06K9/6286G06N3/04G06N3/08G08B7/06G09B19/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,308,325
App. No.
16/654,868
Granted
Apr 19, 2022
Kind
B2
Abstract

A system includes a camera configured to generate image data and a computing device in electronic communication with the camera. The computing device includes at least one processor and is configured to receive, from the camera, one or more images representative of a location. The computing device is further configured to apply a trained classifier to the one or more images to classify the location into one of at least two risk categories, wherein the classification is based on a likelihood of a subject performing a target behavior based on presence of the subject in the location. The computing device is additionally configured to issue a risk alert responsive to the trained classifier classifying the location into a high-risk category.

Claims (36)

1. A system comprising:

a camera configured to generate image data; and

a computing device in electronic communication with the camera, the computing device comprising at least one processor, wherein the computing device is configured to:

receive, from the camera, one or more images representative of a location;

apply a trained classifier to the one or more images to classify the location into one of at least two risk categories, wherein the classification is based on a likelihood of a subject performing a target behavior based on presence of the subject in the location, wherein the target behavior comprises smoking and wherein the trained classifier comprises a convolutional neural network trained to identify one or more objects in the one or more images, and wherein the trained classifier differentiates between a first risk category indicative of a smoking environment and a second risk category indicative of a nonsmoking environment with an accuracy greater than 70%; and

issue a risk alert responsive to the trained classifier classifying the location into a high-risk category, wherein the risk alert comprises one or more of an audible alert, a visual alert, or a tactile alert.

2. The system according to claim 1 , wherein the target behavior comprises one or more of anxiety, eating disorders, falling, insomnia, ADHD, obsessive compulsive disorder, sensory integration disorder, eating behavior, physical activity, or alcoholism.

3. The system according to claim 1 , wherein the trained classifier further comprises an interpretable linear classifier trained to generate a risk index based on the one or more identified objects in the one or more images, wherein the location is classified into a risk category based on the risk index.

4. The system according to claim 1 , wherein the system comprises a display interface configured to display an image of the location with at least one identified object highlighted to indicate an association between the at least one identified object and the target behavior.

5. The system according to claim 1 , wherein the trained classifier comprises a decision tree.

6. The system according to claim 1 , wherein the trained classifier is further trained to distinguish between a first category of high-risk locations in which the subject is at risk of performing the target behavior while present in one of the first category of high-risk locations and a second group of high-risk locations in which the subject is at risk of performing the target behavior subsequent to being present in one of the second category of high-risk locations.

7. The system according to claim 1 , wherein the computing device is a wearable computing device and wherein the camera is physically coupled to the wearable computing device.

8. The system according to claim 1 , wherein the at least one processor is configured to issue the risk alert responsive to the trained classifier classifying the location into the high-risk category based on the trained classifier classifying the location into the high-risk category for each image of a predetermined threshold number of images.

9. The system according to claim 1 , wherein the computing device is further configured to:

determine whether the subject performed the target behavior in the location; and

further train the classifier based on the determination whether the subject performed the target behavior in the location.

10. The system according to claim 9 , further comprising one or more sensors in electronic communication with the computing device, wherein the computing device is configured to determine whether the subject performed the target behavior in the location based on sensor data from the one or more sensors.

11. The system according to claim 1 , wherein the computing device is further configured to provide at least one behavioral recommendation to reduce risk of the subject performing the target behavior.

12. The system according to claim 11 in which the at least one behavioral recommendation to reduce the risk of the subject performing the target behavior comprises one or more of: utilizing a behavioral coping strategy, engaging in an alternate behavior, reconfiguring the location to reduce risk, leaving the location, utilizing a therapy aid, or contacting a support individual.

13. The system according to claim 1 , wherein the trained classifier is further trained to classify the location based on a likelihood of a subject having a physiological event based on presence of the subject in the location.

14. The system according to claim 13 , wherein the physiological event comprises one or more of an asthma attack, seizures, an allergic reaction, high blood pressure, or a stroke.

15. A method comprising:

receiving, from a camera, one or more images representative of a location;

applying a trained classifier to the one or more images to classify the location into one of at least two risk categories,

wherein the classification is based on a likelihood of a subject performing a target behavior based on presence of the subject in the location,

wherein the target behavior comprises smoking and wherein the trained classifier comprises a convolutional neural network trained to identify one or more objects in the one or more images, and

wherein the trained classifier differentiates between a first risk category indicative of a smoking environment and a second risk category indicative of a nonsmoking environment with an accuracy greater than 70%; and

issuing a risk alert responsive to the trained classifier classifying the location into a high-risk category, wherein the risk alert comprises one or more of an audible alert, a visual alert, or a tactile alert.

16. A mobile computing device comprising:

a camera configured to generate one or more images representative of a location of the mobile computing device; and

a non-transitory computer-readable medium comprising program instructions executable by at least one processor to cause the at least one processor to:

apply a trained classifier to the one or more images to classify the location into one of at least two risk categories,

wherein the classification is based on a likelihood of a subject performing a target behavior based on presence of the subject in the location,

wherein the target behavior comprises smoking and wherein the trained classifier comprises a convolutional neural network trained to identify one or more objects in the one or more images, and

wherein the trained classifier differentiates between a first risk category indicative of a smoking environment and a second risk category indicative of a nonsmoking environment with an accuracy greater than 70%; and

issue a risk alert responsive to the trained classifier classifying the location into a high-risk category, wherein the risk alert comprises one or more of an audible alert, a visual alert, or a tactile alert.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 13, 2021
From: MCCLERNON, FRANCIS; ENGELHARD, MATTHEW; CARIN, LAWRENCE; OLIVER, JASON
To: DUKE UNIVERSITY
Reel/Frame 055908/0632 →
Continuity (3)
Provisional Application 62867952 · Jun 28, 2019
Provisional Application 62746077 · Oct 16, 2018
Related Publication 20200117901A1 · Apr 16, 2020
Cited By (1)
US 12,333,806