Granted Patent B2

US 12,361,714 · App. 17/818,706 · Granted Jul 15, 2025

Inventors: Nageswara Sastry Renduchintala (Kolkata, IN); Hamid Majdabadi (Ottawa, CA); Su Liu (Austin, TX); Yang Liang (BeiJing, CN)

Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION

G06V20/52G06T7/70G06V20/41G08B21/02G16Y10/60G06T2207/10016

Abstract

Claims (70)

1. A computer-based method of detecting the danger of an object in close proximity of a subject, the method comprising:

receiving real-time data from one or more IoT devices in a surrounding environment, wherein the real-time data includes a video feed of one or more subjects engaged in one or more activities;

detecting and classifying the one or more subjects and one or more objects in an image from the one or more IoT devices;

identifying one or more risk factors associated with each object based on the classification of the one or more objects;

correlating the one or more risk factors associated with each object with the one or more subjects in the image based on the one or more activities and the classification of the one or more subjects, wherein correlating the one or more risk factors includes correlating a first risk factor of the one or more risk factors with a first subject of the one or more subjects and correlating a different risk factor of the one or more risk factors with a different subject of the one or more subjects based on one or more characteristics associated with the first subject and the different subject;

identifying relative positions of the one or more subjects and the one or more objects in the image;

determining whether a current position of at least one subject is dangerous based on the relative positions of the one or more subjects and the one or more objects and the one or more correlated risk factors; and

in response to determining the current position of the at least one subject is dangerous based on a current distance between the at least one subject and at least one object of the one or more objects being less than a minimum risk distance, notifying a user of the danger to the at least one subject, wherein notifying the user of the danger to the at least one subject further comprises:

sending a simultaneous notification to a robotic device to perform a preventive action, wherein the simultaneous notification causes the robotic device to at least remove the at least one object to a location where a distance between the at least one subject and the at least one object is greater than the minimum risk distance.

2. The computer-based method of claim 1 , further comprising a training phase, wherein the training phase comprises:

receiving a historical video feed previously captured from an online repository;

identifying and classifying one or more historical subjects and one or more historical objects in the historical video feed;

identifying one or more pre-defined risk factors associated with each historical object from the online repository;

correlating the one or more pre-defined risk factors associated with each historical object with the one or more historical subjects based on the one or more historical activities and the classification of the one or more historical subjects;

identifying historical relative positions of the one or more historical subjects and the one or more historical objects in the historical video feed;

obtaining a pre-defined minimum risk distance between the one or more historical subjects and the one or more historical objects from the online repository;

determining whether an injury resulted to at least one historical subject; and

in response to determining the injury resulted to the at least one historical subject, updating the pre-defined minimum risk distance in the online repository for the at least one historical subject.

3. The computer-based method of claim 2 , wherein the online repository contains a table including the pre-defined minimum risk distance for each historical subject, and wherein the pre-defined minimum risk distance is updated in the table for the at least one historical subject who was injured.

4. The computer-based method of claim 1 , wherein receiving the real-time data further comprises:

receiving pre-defined configuration criteria from the user, wherein the pre-defined configuration criteria includes an area ID for each IOT device and one or more service preferences.

5. The computer-based method of claim 4 , wherein the notification includes the area ID where the current position of the at least one subject is dangerous.

6. The computer-based method of claim 1 , wherein the preventive action further includes blocking the at least one subject from reaching the one or more objects.

7. A computer system, the computer system comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more computer-readable tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, wherein the computer system is capable of performing a method comprising:

receiving real-time data from one or more IoT devices in a surrounding environment, wherein the real-time data includes a video feed of one or more subjects engaged in one or more activities;

detecting and classifying the one or more subjects and one or more objects in an image from the one or more IoT devices;

identifying one or more risk factors associated with each object based on the classification of the one or more objects;

correlating the one or more risk factors associated with each object with the one or more subjects in the image based on the one or more activities and the classification of the one or more subjects, wherein correlating the one or more risk factors includes correlating a first risk factor of the one or more risk factors with a first subject of the one or more subjects and correlating a different risk factor of the one or more risk factors with a different subject of the one or more subjects based on one or more characteristics associated with the first subject and the different subject;

identifying relative positions of the one or more subjects and the one or more objects in the image;

determining whether a current position of at least one subject is dangerous based on the relative positions of the one or more subjects and the one or more objects and the one or more correlated risk factors; and

in response to determining the current position of the at least one subject is dangerous based on a current distance between the at least one subject and at least one object of the one or more objects being less than a minimum risk distance, notifying a user of the danger to the at least one subject, wherein notifying the user of the danger to the at least one subject further comprises:

sending a simultaneous notification to a robotic device to perform a preventive action, wherein the simultaneous notification causes the robotic device to at least remove the at least one object to a location where a distance between the at least one subject and the at least one object is greater than the minimum risk distance.

8. The computer system of claim 7 , further comprising a training phase, wherein the training phase comprises:

receiving a historical video feed previously captured from an online repository;

identifying and classifying one or more historical subjects and one or more historical objects in the historical video feed;

identifying one or more pre-defined risk factors associated with each historical object from the online repository;

correlating the one or more pre-defined risk factors associated with each historical object with the one or more historical subjects based on the one or more historical activities and the classification of the one or more historical subjects;

identifying historical relative positions of the one or more historical subjects and the one or more historical objects in the historical video feed;

obtaining a pre-defined minimum risk distance between the one or more historical subjects and the one or more historical objects from the online repository;

determining whether an injury resulted to at least one historical subject; and

in response to determining the injury resulted to the at least one historical subject, updating the pre-defined minimum risk distance in the online repository for the at least one historical subject.

9. The computer system of claim 8 , wherein the online repository contains a table including the pre-defined minimum risk distance for each historical subject, and wherein the pre-defined minimum risk distance is updated in the table for the at least one historical subject who was injured.

10. The computer system of claim 7 , wherein receiving the real-time data further comprises:

receiving pre-defined configuration criteria from the user, wherein the pre-defined configuration criteria includes an area ID for each IOT device and one or more service preferences.

11. The computer system of claim 10 , wherein the notification includes the area ID where the current position of the at least one subject is dangerous.

12. The computer system of claim 7 , wherein the preventive action further includes blocking the at least one subject from reaching the one or more objects.

13. A computer program product, the computer program product comprising:

one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more computer-readable tangible storage medium, the program instructions executable by a processor capable of performing a method, the method comprising:

receiving real-time data from one or more IoT devices in a surrounding environment, wherein the real-time data includes a video feed of one or more subjects engaged in one or more activities;

detecting and classifying the one or more subjects and one or more objects in an image from the one or more IoT devices;

identifying one or more risk factors associated with each object based on the classification of the one or more objects;

correlating the one or more risk factors associated with each object with the one or more subjects in the image based on the one or more activities and the classification of the one or more subjects, wherein correlating the one or more risk factors includes correlating a first risk factor of the one or more risk factors with a first subject of the one or more subjects and correlating a different risk factor of the one or more risk factors with a different subject of the one or more subjects based on one or more characteristics associated with the first subject and the different subject;

identifying relative positions of the one or more subjects and the one or more objects in the image;

determining whether a current position of at least one subject is dangerous based on the relative positions of the one or more subjects and the one or more objects and the one or more correlated risk factors; and

in response to determining the current position of the at least one subject is dangerous based on a current distance between the at least one subject and at least one object of the one or more objects being less than a minimum risk distance, notifying a user of the danger to the at least one subject, wherein notifying the user of the danger to the at least one subject further comprises:

sending a simultaneous notification to a robotic device to perform a preventive action, wherein the simultaneous notification causes the robotic device to at least remove the at least one object to a location where a distance between the at least one subject and the at least one object is greater than the minimum risk distance.

14. The computer program product of claim 13 , further comprising a training phase, wherein the training phase comprises:

receiving a historical video feed previously captured from an online repository;

identifying and classifying one or more historical subjects and one or more historical objects in the historical video feed;

identifying one or more pre-defined risk factors associated with each historical object from the online repository;

correlating the one or more pre-defined risk factors associated with each historical object with the one or more historical subjects based on the one or more historical activities and the classification of the one or more historical subjects;

identifying historical relative positions of the one or more historical subjects and the one or more historical objects in the historical video feed;

obtaining a pre-defined minimum risk distance between the one or more historical subjects and the one or more historical objects from the online repository;

determining whether an injury resulted to at least one historical subject; and

in response to determining the injury resulted to the at least one historical subject, updating the pre-defined minimum risk distance in the online repository for the at least one historical subject.

15. The computer program product of claim 14 , wherein the online repository contains a table including the pre-defined minimum risk distance for each historical subject, and wherein the pre-defined minimum risk distance is updated in the table for the at least one historical subject who was injured.

16. The computer program product of claim 13 , wherein receiving the real-time data further comprises:

receiving pre-defined configuration criteria from the user, wherein the pre-defined configuration criteria includes an area ID for each IOT device and one or more service preferences.

17. The computer program product of claim 16 , wherein the notification includes the area ID where the current position of the at least one subject is dangerous.

Assignments (1)

Continuity (1)

References Cited (28)