Augmented reality risk vulnerability analysis
View Patent ↗A system, method and program product for identifying a risk hazard vulnerability at a target environment and presenting a detected risk hazard. The method configures an evaluator mobile computing device to obtain, via the mobile device, a real-world view or image data of items/objects in a target real-world environment and, optionally, or in addition, obtain other context data associated with the target environment. The method invokes operations of a risk vulnerability model to detect a risk associated with the target object within the real-world view and based on any additional associated context data. The method causes the mobile device to further generate a visual, audible or haptics indication of the identified risk vulnerability and/or a calculated risk score of the target object for display on the mobile computing device and display the risk vulnerability indication on the real-world view displayed at the mobile device.
1. A method comprising:
obtaining, using an image sensor associated with a mobile computing device, a real-world view of a target location;
detecting, in real-time, using a processor at said mobile computing device, a target object within said real-world view;
obtaining, in real-time, using the processor, additional context data associated with said target object, said additional context data comprising a real-time weather condition at the target location produced from one or more remotely located Internet of Things (IoT) devices and received over a wireless network connection;
obtaining an image capture of said target object;
extracting text or indicia information located on said target object directly from said image capture of said target object;
correlating said text or indicia information against relevant data that would indicate a risk vulnerability exists with respect to the target object;
detecting, using said processor, a risk hazard associated with said target object based on said additional context data and said correlated information indicating a risk vulnerability exists with respect to the target object, wherein said risk hazard detecting comprises:
running, at the processor, a model trained to recognize said target object of said target location within said real-world view display; and correlating, using said model, said recognized target object with a known risk hazard;
generating, using said processor, an indication of said risk vulnerability of said target object for display on said mobile computing device;
capturing, using the mobile computing device, a corresponding real-world audio data recording of the target location;
evaluating the captured real-world audio data;
recognizing audio patterns associated with items known to present a risk hazard;
classifying the objects/items captured in the audio data according to a risk assessment category;
generating, using said processor, a risk vulnerability indication at said mobile computing device corresponding to said risk assessment category; and
displaying, using said processor, said risk vulnerability indication on said real-world view displayed at said mobile computing device.
2. The method according to claim 1 , further comprising:
communicating, using said processor, said detected risk hazard over a network to a user computing device, to apprise a user at said user computing device of said risk vulnerability.
3. The method according to claim 1 , wherein said displaying said risk vulnerability indication comprises:
rendering an object, a data, or both an object and data for display on said mobile computing device as an overlay on the real-world view display.
4. The method according to claim 3 , wherein to correlate said recognized target object with a known risk hazard, said model is further trained to:
compare, using a processor device, a recognized target object with one or more historical images of said target object detected as being associated with a known risk hazard.
5. The method according to claim 3 , further comprising:
determining, using said processor, a risk assessment category associated with one or more target objects in said view;
computing, using said processor, a risk vulnerability score based on said context data and said associated target object;
comparing said score against a threshold corresponding to a risk hazard; and
communicating, using said processor, said target score as said risk vulnerability indication associated with said target object.
6. The method according to claim 1 , further comprising: generating, using said processor, said risk vulnerability indication as a sensation for said user via a haptic interface at said mobile computing device.
7. A system comprising:
an image sensor associated with a mobile computing device;
a memory storage device storing program instructions;
a hardware processor coupled to said memory storage device and configured to run said program instructions to perform a method to:
obtain, using the image sensor, a real-world view of a target location;
detect, in real-time, at said mobile computing device, a target object within said real-world view;
obtain, in real-time, additional context data associated with said target object, said additional context data comprising a real-time weather condition at the target location produced from one or more remotely located Internet of Things (IoT) devices and received over a wireless network connection;
obtain an image capture of said target object;
extract text or indicia information located on said target object directly from said image capture of said target object;
correlate said text or indicia information against relevant data that would indicate a risk vulnerability exists with respect to the target object;
detect a risk hazard associated with said target object based on said additional context data and said correlated information indicating a risk vulnerability exists with respect to the target objects wherein to detect a risk hazard, the hardware processor is further configured to:
run a model trained to recognize said target object of said target location within said real-world view display; and correlate, using said model, said recognized target object with a known risk hazard; generate an indication of said risk vulnerability of said target object for display on said mobile computing device; and
wherein said hardware processor is further configured to: capture a corresponding real-world audio data recording of the target location; evaluate the captured real-world audio data; recognize audio patterns associated with items known to present a risk hazard; classify the objects/items captured in the audio data according to a risk assessment category; and generate a risk vulnerability indication at said mobile computing device corresponding to said risk assessment category;
display said risk vulnerability indication on said real-world view displayed at said mobile computing device.
8. The system according to claim 7 , wherein said hardware processor is further configured to:
communicate said detected risk hazard over a network to a user computing device, to apprise a user at said user computing device of said risk vulnerability.
9. The system according to claim 8 , wherein to display said risk vulnerability indication, said hardware processor is further configured to:
render an object, a data, or both an object and data, for display on said mobile computing device as an overlay on the real-world view display.
10. The system according to claim 9 , wherein to correlate said recognized target object with a known risk hazard, said model is further trained to:
compare a recognized target object with one or more historical images of said target object detected as being associated with a known risk hazard.
11. The system according to claim 9 , wherein said hardware processor is further configured to:
determine a risk assessment category associated with one or more target objects in said view;
compute a risk vulnerability score based on said context data and said associated target object;
compare said score against a threshold corresponding to a risk hazard; and
communicate said target score as said risk vulnerability indication associated with said target object.
12. The system according to claim 7 , said hardware processor is further configured to:
generate said risk vulnerability indication as a sensation for said user via a haptic interface at said mobile computing device.
13. A computer program product comprising a non-transitory computer-readable medium comprising instructions that, when executed by at least one processor comprising hardware, configure the at least one processor to:
obtain, using an image sensor associated with a mobile computing device, a real-world view of a target location;
detect, in real-time, at said mobile computing device, a target object within said real-world view;
obtain, in real-time, additional context data associated with said target object, said additional context data comprising a real-time weather condition at the target location produced from one or more remotely located Internet of Things (IoT) devices and received over a wireless network connection;
obtain an image capture of said target object;
extract text or indicia information located on said target object directly from said image capture of said target object;
correlate said text or indicia information against relevant data that would indicate a risk vulnerability exists with respect to the target object;
detect a risk hazard associated with said target object based on said additional context data and said correlated information indicating a risk vulnerability exists with respect to the target object,
wherein to detect said risk hazard, said instructions further configure the at least one processor to:
run a model trained to recognize said target object of said target location within said real-world view display; and correlate, using said model, said recognized target object with a known risk hazard; generate an indication of said risk vulnerability of said target object for display on said mobile computing device;
wherein said instructions further configure the at least one processor to: capture a corresponding real-world audio data recording of the target location; evaluate the captured real-world audio data; recognize audio patterns associated with items known to present a risk hazard; classify the objects/items captured in the audio data according to a risk assessment category; and generate a risk vulnerability indication at said mobile computing device corresponding to said risk assessment category; and
display said risk vulnerability indication on said real-world view displayed at said mobile computing device.
14. The computer program product according to claim 13 , wherein said instructions further configure the at least one processor to:
communicate said detected risk hazard over a network to a user computing device, to apprise a user at said user computing device of said risk vulnerability.
15. The computer program product according to claim 13 , wherein to display said risk vulnerability indication, said instructions further configure the at least one processor to:
render an object, a data, or both an object and data for display on said mobile computing device as an overlay on the real-world view display.
16. The computer program product according to claim 15 , wherein to correlate said recognized target object with a known risk hazard, said model is further trained to:
compare, using a processor device, a recognized target object with one or more historical images of said target object detected as being associated with a known risk hazard.
17. The computer program product according to claim 15 , wherein said instructions further configure the at least one processor to:
determine a risk assessment category associated with one or more target objects in said view;
compute a risk vulnerability score based on said context data and said associated target object;
compare said score against a threshold corresponding to a risk hazard; and
communicate said target score as said risk vulnerability indication associated with said target object.