System and method for visual feedback
View Patent ↗Systems, methods, and computer program products are provided for providing feedback. An image of an individual is received. At least one item associated with the individual in the image of the individual is detected using a trained machine learning model. The at least one item is one of a plurality of items of interest. The trained machine learning model takes as an input the image of the individual and generates as an output an indicator of the at least one item of interest detected. A recommendation is generated responsive to detecting the item. The divestiture recommendation is then presented to the individual.
1 . A system for providing visual feedback to individuals at a security checkpoint, the system comprising:
a camera;
a display; and
a processing system comprising:
a memory storing computer readable instructions and a trained machine learning model; and
a processing device for executing the computer readable instructions, the computer readable instructions controlling the processing device to perform operations comprising:
receiving an image of an individual from the camera;
detecting at least one non-threat item associated with the individual in the image of the individual using the trained machine learning model, the at least one non-threat item being one of a plurality of non-threat items of interest, the non-threat items of interest are items to be divested prior to an initial screening, the trained machine learning model taking as an input the image of the individual and generating as an output an indicator of the at least one non-threat item of interest detected;
generating a divestiture recommendation for the individual responsive to detecting the at least one of the non-threat items in the image of the individual; and
presenting to the individual, on the display, the divestiture recommendation as the visual feedback.
2 . The system of claim 1 , wherein
the trained machine learning model is trained for a non-threat divestiture use case.
3 . The system of claim 1 , wherein the indicator is a bounding box around the item overlaid on the image or a video of the individual.
4 . The system of claim 1 , wherein the indicator is a mask of the item overlaid on the image or a video of the individual.
5 . The system of claim 1 , wherein the indicator is an outline around the item overlaid on the image or a video of the individual.
6 . The system of claim 1 , wherein the camera, the display, and the processing system are integrated into a housing.
7 . The system of claim 1 , further comprising a microphone and a speaker.
8 . The system of claim 1 , wherein the trained machine learning model is based on a region based convolutional neural network (R-CNN) architecture.
9 . The system of claim 1 , wherein the divestiture recommendation is presented as text.
10 . The system of claim 1 , wherein the divestiture recommendation is presented as an animation.
11 . The system of claim 1 , wherein the indicator is overlaid on the image or a video of the individual.
12 . The system of claim 1 , wherein the indicator is overlaid on a digital representation.
13 . The system of claim 12 , wherein the digital representation is an avatar.
14 . The system of claim 1 , wherein the trained machine learning model outputs a label for the item.
15 . The system of claim 1 , wherein the image is a frame extracted from a video of the individual.
16 . A system for providing auditory feedback to individuals at a security checkpoint, the system comprising:
a camera;
a speaker; and
a processing system comprising:
a memory storing computer readable instructions and a trained machine learning model; and
a processing device for executing the computer readable instructions, the computer readable instructions controlling the processing device to perform operations comprising:
receiving an image of an individual from the camera;
detecting at least one non-threat item associated with the individual in the image of the individual using the trained machine learning model, the at least one non-threat item being one of a plurality of non-threat items of interest, the non-threat items of interest are items to be divested prior to an initial screening, the trained machine learning model taking as an input the image of the individual and generating as an output an indicator of the at least one non-threat item of interest detected;
generating a divestiture recommendation for the individual responsive to detecting the at least one of the non-threat items in the image of the individual; and
presenting to the individual, using the speaker, the divestiture recommendation as the auditory feedback.
17 . The system of claim 16 , wherein the auditory feedback is a spoken instruction.
18 . The system of claim 16 , wherein the system further comprises a display, wherein the operations further comprise presenting to the individual, using the display, the divestiture recommendation as visual feedback.
19 . A computer-implemented method for providing visual feedback to individuals at a security checkpoint, the computer-implemented method comprising:
receiving an image of an individual;
detecting at least one non-threat item associated with the individual in the image of the individual using a trained machine learning model, the at least one non-threat item being one of a plurality of non-threat items of interest, the non-threat items of interest are items to be divested prior to an initial screening, the trained machine learning model taking as an input the image of the individual and generating as an output an indicator of the at least one non-threat item of interest detected;
generating a divestiture recommendation for the individual responsive to detecting the at least one of the non-threat items in the image of the individual; and
presenting to the individual, on a display, the divestiture recommendation as the visual feedback.
20 . The computer-implemented method of claim 19 , further comprising presenting, using a speaker, the divestiture recommendation to the individual as auditory feedback.
21 . A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations for providing visual feedback to individuals at a security checkpoint, the operations comprising:
receiving an image of an individual from a camera;
detecting at least one non-threat item associated with the individual in the image of the individual using a trained machine learning model, the at least one non-threat item being one of a plurality of non-threat items of interest, the non-threat items of interest are items to be divested prior to an initial screening, the trained machine learning model taking as an input the image of the individual and generating as an output an indicator of the at least one non-threat item of interest detected;
generating a divestiture recommendation for the individual responsive to detecting the at least one non-threat items in the image of the individual; and
presenting to the individual, on the display, the divestiture recommendation as the visual feedback.