IP Library Granted Patent US 11,710,075
Granted Patent B2
US 11,710,075 · App. 17/396,499 · Granted Jul 25, 2023

Hazard recognition

Inventors: Manish Shah (San Francisco, CA); Jeffrey Greenberg (San Francisco, CA); Evan Minamoto (Emeryville, CA); Navin Gupta (San Francisco, CA); Paolo Tagliani (Gavardo, IT); Francisco Jose Montiel Navarro (Malaga, ES)
Assignee: Bardavon Health Digitial, Inc.
G06N20/00G06F3/0482G06F18/214G06F18/24G06F18/40G06T7/0002G06T7/70G06V20/20G06V20/41G06V20/52G06T2207/10016G06T2207/20081G06V2201/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,710,075
App. No.
17/396,499
Granted
Jul 25, 2023
Kind
B2
Abstract

Methods, systems, and devices are provided for identifying hazards. According to one aspect, a computer-implemented method can include receiving a plurality of sensor data including one or more image files from a mobile device. The method can include generating one or more position and label pairs based on the plurality of sensor data. The method can include assigning a hazard recognition to each of the position and label pairs. The method can include assigning a score associated to each of the hazard recognitions. The method can include displaying a result including one or more image results based on the one or more image files, one or more hazard recognitions, the one or more hazard recognitions associated with at least one of the one or more image results, and one or more scores associated to each of the hazard recognitions.

Claims (44)

1. A system comprising one or more processors, and a non-transitory computer-readable medium including one or more sequences of instructions that, when executed by the one or more processors, cause the system to perform operations comprising:

generating one or more position and label pairs based on a plurality of sensor data, the sensor data including one or more image files received from a mobile device;

generating, via a trained artificial intelligence (AI) model, a hazard recognition to be assigned to each of the position and label pairs, the AI model trained based on a training data set based on a plurality of images of hazards labeled according to a plurality of hazard types;

generating, via the trained AI model, a score to be associated with each of the hazard recognitions; and

displaying a result comprising:

one or more image results based on the one or more image files;

one or more hazard recognitions, the one or more hazard recognitions associated with at least one of the one or more image results and a plurality of different hazard types; and

one or more scores associated to each of the different hazard types of the hazard recognitions.

2. The system of claim 1 , wherein the sensor data is comprised of video capture data from a video or imaging capture device, imaging angle from an accelerometer of the mobile device, user selected data, or a combination thereof.

3. The system of claim 2 , wherein the user selected data are derived from selectable themes or categories on a mobile application.

4. The system of claim 3 , wherein the mobile application comprises a user interface configured to receive selections by the user and give recommendations, or recommended selections, or a combination thereof.

5. The system of claim 4 , wherein the recommendations are based on a detection of a minimum or maximum threshold based on a camera angle of the mobile device when the user is using the mobile application.

6. The system of claim 1 , wherein the label identifies a characteristic of an item, scene, path, condition, potential hazards, or a combination thereof.

7. The system of claim 1 , wherein the position identifies a local position of an item, scene, path, condition, potential hazards, or a combination thereof.

8. The system of claim 6 , wherein the local position is identified based on an entire frame of an image generated by the sensor data.

9. The system of claim 6 , wherein the local position is identified based on a portion of an image associated to a bounding box or a segmented list of points within the image.

10. The system of claim 1 , further comprising the operations of:

applying a recognition filter to each pair of the one or more position and label pairs; and

generating one or more filtered label and position pairs.

11. The system of claim 9 , wherein applying the recognition filter to each pair of the one or more position and label pairs comprises applying a hysteresis filtering.

12. The system of claim 9 , wherein applying the recognition filter to each pair of the one or more position and label pairs comprises context recognition filtering of a user based on a selected context provided by the user.

13. The system of claim 9 , further comprising: generating a virtual hazard recognition based on the generating of the one or more filtered label and position pairs.

14. The system of claim 1 , wherein training the AI model comprises:

receiving a plurality of video files from a crowdsourcing platform;

processing each video file into a set of image files;

sending each image file of the set of image files into an image queue;

processing the image queue comprising labelling each image file according to a hazard characterization each image file represents; and

tagging each of the image files with one or more of the hazard characterizations.

15. The system of claim 1 , wherein the hazard recognition is assigned based on a user's personal data including age, health, vision, physical mobility, personal medical information and conditions, or a combination thereof.

16. The system of claim 1 , further comprising the operations of: associating one or more hazard recognitions and score of the hazard recognitions with a scene or space.

17. The system of claim 16 , further comprising the operations of: generating a hazard score based on an aggregate of each score of the hazard recognitions associated with the scene or space.

18. The system of claim 1 , further comprising:

generating one or more corrective measures based on at least part of the one or more scores associated with the hazard recognitions, and

displaying the one or more corrective measures on the mobile device.

19. The system of claim 1 , further comprising: receiving a selection input from a human agent to determine whether a hazard classification based on one or more of the assigned hazard recognitions has been addressed.

20. The system of claim 1 , further comprising:

receiving, via a server, one or more images from the mobile device;

detecting one or more hazards by processing the one or more images via a trained server-based AI hazard detection model; and

transmitting to the mobile device an indication of the detected one or more hazards.

21. The system of claim 1 , further comprising:

labeling the detected one or more hazards; and

segmenting the one or more images according to the labeled hazard.

22. The system of claim 21 , wherein the segmentation of the one or more images generates a graphical indication identifying a respective detected hazard.

23. The system of claim 21 , further comprising training the AI model with the one or more images received from the mobile device and the labeled hazards.

Assignments (2)
MERGER Recorded Mar 13, 2023
From: PEERWELL, INC.
To: BARDAVON HEALTH DIGITIAL, INC.
Reel/Frame 062965/0150 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 13, 2021
From: SHAH, MANISH; GREENBERG, JEFFREY; MINAMOTO, EVAN; GUPTA, NAVIN; TAGLIANI, PAOLO; JOSE MONTIEL NAVARRO, FRANCISCO
To: PEERWELL, INC.
Reel/Frame 057167/0175 →
Continuity (3)
Continuation 16385559 · Apr 16, 2019
Provisional Application 62658299 · Apr 16, 2018
Related Publication 20210365685A1 · Nov 25, 2021