Sanitized surface monitoring and smart user alerts
A method, computer program product, and system include a processor(s) that map a physical environment by utilizing one or more image capture devices to scan aspects of the physical environment. The mapping identifies contamination levels and features associated with objects in the environment. The processor(s) utilizes unsupervised learning and supervised learning to identify activities engaged in by a user in the environment. The processor(s) determines that a trigger event has occurred. The processor(s) identify an activity engaged in by the user and determine if a user interface utilized by the program code to display results is visible to the user. If the processor(s) determines that the interface is not visible, the processor(s) selects an alert mechanism to alert the user to the trigger event and alerts the user to the trigger event with the selected alert mechanism.
1. A computer-implemented method, comprising:
mapping, by one or more processors, a physical environment based on utilizing one or more image capture devices to scan aspects of the physical environment, wherein the mapping comprises identifying contamination levels and features associated with objects in the physical environment;
utilizing, by the one or more processors, unsupervised learning and supervised learning to identify activities engaged in by a user in the physical environment;
based on the mapping, determining, by the one or more processors, that a trigger event has occurred, wherein the trigger event comprises the one or more processors identifying a feature pre-configured to be the trigger event;
identifying, by the one or more processors, an activity engaged in by the user, wherein a convolutional neural network (CNN) automatically extracts high-level features by processing raw sensory data to extract discriminative features, wherein the CNN associates the discriminative features with a set of activity elements to predict the activity engaged in by the user, wherein the feature pre-configured to be the trigger event is switching from an initial activity in which the user is attentive to the physical environment to the activity predicted by the CNN, and wherein, based on the trigger event, the one or more processors determine whether the physical environment includes a contaminated or sanitized surface and select an alert mechanism to notify the user of the contamination based on the activity predicted;
based on the identifying, determining, by the one or more processors, if a user interface utilized by the program code to display results is visible to the user in light of the activity;
in response to determining that the interface utilized by the program code to display results is not visible to the user due to the activity, selecting, by the one or more processors, the alert mechanism to alert the user to the trigger event; and
alerting, by the one or more processors, the user to the trigger event, utilizing the alert mechanism.
2. The computer-implemented method of claim 1 , wherein the determining if the user interface utilized by the program code to display results is visible to the user comprises:
referencing, by the one or more processors, a pre-configured compendium.
3. The computer-implemented method of claim 1 , further comprising:
monitoring, by the one or more processors, a response of the user to the alert mechanism, wherein the monitoring comprises:
obtaining, by the one or more processors, for each alert and for each type of type of alert mechanism to the user, a timing value reflecting time between the alert and the user viewing data associated with the trigger event.
4. The computer-implemented of claim 3 , further comprising:
obtaining, by the one or more processors, over time, timing values for various alerts to the user;
generating, by the one or more processors, based on the timing values, a user profile, wherein the user profile ranks the various alerts by the alert mechanism from shortest timing value to longest timing value of the timing values to determine efficacy levels for each alert mechanism for the user; and
updating, by the one or more processors, the user profile, based on the efficacy levels and any change in efficacy levels, over the time.
5. The computer-implemented method of claim 4 , wherein the selecting the alert mechanism to alert the user to the trigger event comprises:
obtaining, by the one or more processors, the user profile; and
selecting, by the one or more processors, the alert mechanism, wherein the alert mechanism is associated with a highest efficacy level.
6. The computer-implemented method of claim 1 , wherein the mapping further comprises controlling a smart camera embedded with a mixed reality device to scan the aspects of the physical environment.
7. The computer-implemented method of claim 1 , wherein the mapping comprises utilizing a scanning mechanism to identify the objects in the physical environment.
8. The computer-implemented method of claim 7 , wherein identifying the objects in the physical environment comprises obtaining data related to the anti-bacterial properties of materials that comprise the objects.
9. The computer implemented method of claim 7 , wherein the scanning mechanism is a one-stage detection model selected from the group consisting of: You Only Look Once and a Single-Shot Detector.
10. The computer-implemented method of claim 1 , wherein utilizing the unsupervised learning and the supervised learning comprises:
obtaining, by the one or more processors, data from one or more sensors proximate to the user; and
executing, by the one or more processors, a Deep Auto-Encoder-Set Network for Activity Recognition to identify the activities engaged in by the user in the physical environment.
11. The computer-implemented method of claim 1 , wherein the unsupervised learning comprises:
obtaining, by the one or more processors, unlabeled sensor measurements from sensors proximate to the user; and
learning, from the unlabeled sensor measurements, feature representations for activity sets to train an activity recognition model.
12. The computer-implemented method of claim 11 , wherein the supervised learning comprises:
leveraging, by the one or more processors, label information from a data source, to extract more discriminative features than the feature representations from the unlabeled sensor data.
13. The computer-implemented method of claim 12 , further comprising:
training, by the one or more processors, the activity recognition model, with the feature representations and the more discriminative features to generate the activity sets for sensory data from the sensors.
14. The computer-implemented method of claim 1 , wherein the alert mechanism comprises one or more of: audio interleaving, visual injection into an ongoing visual session, haptic feedback, and auditory feedback.
15. The computer-implemented method of claim 1 , further comprising:
providing, by the one or more processors, the user interface to enable the user to view the contamination levels and the features associated with the objects in the physical environment, the providing comprising:
displaying, by the one or more processors, in a graphical user interface on a computing device, a digital representation of the physical environment; and
overlaying, by the one or more processors, on the digital representation of the physical environment, indicators on a portion of the objects in the digital representation, wherein the indicators correlate with the contamination levels.
16. The computer-implemented method of claim 1 , further comprising:
providing, by the one or more processors, the user interface to enable the user to view the contamination levels and the features associated with the objects in the physical environment, the providing comprising:
projecting, by the one or more processors, utilizing a projection system, on the physical environment, indicators on a portion of the objects, wherein the indicators correlate with the contamination levels.
17. The computer-implemented method of claim 1 , wherein the one or more image capture devices are selected from the group consisting of: one or more video cameras, one or more spectroscopy cameras, and one or more infrared cameras.
18. The computer-implemented method of claim 1 , further comprising:
transmitting, by the one or more processors, a command to a robotic cleaning device to mitigate the trigger event.
19. A computer program product comprising:
a computer readable storage medium readable by one or more processors of a shared computing environment comprising a computing system and storing instructions for execution by the one or more processors for performing a method comprising:
mapping, by the one or more processors, a physical environment based on utilizing one or more image capture devices to scan aspects of the physical environment, wherein the mapping comprises identifying contamination levels and features associated with objects in the physical environment;
utilizing, by the one or more processors, unsupervised learning and supervised learning to identify activities engaged in by a user in the physical environment;
based on the mapping, determining, by the one or more processors, that a trigger event has occurred, wherein the trigger event comprises the one or more processors identifying a feature pre-configured to be the trigger event;
identifying, by the one or more processors, an activity engaged in by the user, wherein a convolutional neural network (CNN) automatically extracts high-level features by processing raw sensory data to extract discriminative features, and wherein the CNN associates the discriminative features with a set of activity elements to predict the activity engaged in by the user, wherein, based on the trigger event, the one or more processors determine whether the physical environment includes a contaminated or sanitized surface and select an alert mechanism to notify the user of the contamination based on the activity predicted;
based on the identifying, determining, by the one or more processors, if a user interface utilized by the program code to display results is visible to the user in light of the activity;
in response to determining that the interface utilized by the program code to display results is not visible to the user due to the activity, selecting, by the one or more processors, the alert mechanism to alert the user to the trigger event; and
alerting, by the one or more processors, the user to the trigger event, utilizing the alert mechanism.
20. A computer system comprising:
a memory;
the one or more processors in communication with the memory;
program instructions executable by the one or more processors to perform a method, the method comprising:
mapping, by the one or more processors, a physical environment based on utilizing one or more image capture devices to scan aspects of the physical environment, wherein the mapping comprises identifying contamination levels and features associated with objects in the physical environment;
utilizing, by the one or more processors, unsupervised learning and supervised learning to identify activities engaged in by a user in the physical environment;
based on the mapping, determining, by the one or more processors, that a trigger event has occurred, wherein the trigger event comprises the one or more processors identifying a feature pre-configured to be the trigger event;
identifying, by the one or more processors, an activity engaged in by the user, wherein a convolutional neural network (CNN) automatically extracts high-level features by processing raw sensory data to extract discriminative features, wherein the CNN associates the discriminative features with a set of activity elements to predict the activity engaged in by the user, wherein, based on the trigger event, the one or more processors determine whether the physical environment includes a contaminated or sanitized surface and select an alert mechanism to notify the user of the contamination based on activity predicted;
based on the identifying, determining, by the one or more processors, if a user interface utilized by the program code to display results is visible to the user in light of the activity;
in response to determining that the interface utilized by the program code to display results is not visible to the user due to the activity, selecting, by the one or more processors, the alert mechanism to alert the user to the trigger event; and
alerting, by the one or more processors, the user to the trigger event, utilizing the alert mechanism.