Computer-readable recording medium, information processing method, and information processing device
A non-transitory computer-readable recording medium stores therein an information processing program that causes a computer to execute a process including, acquiring a video of a person who grasps an item to be registered in an accounting machine, by analyzing the acquired video, calculating a score indicating a level of reliability of the item that is contained in the video with respect to each of a plurality of possible items that are set previously, acquiring information on the item that is registered in the accounting machine by the person by operating the accounting machine, based on the calculated score, selecting a possible item from the possible items, and based on the selected possible item and the acquired information on the item, generating an alert indicating abnormality of the item that is registered in the accounting machine.
1 . A non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process comprising:
inputting a video of respective possible items available in a store to a machine learning model trained to specify an item from the video, and calculating and storing scores indicating a level of reliability of the respective possible items;
acquiring a video of a person who grasps an item to be registered in an accounting machine;
inputting the acquired video, and calculating a first score that is highest among the scores of the respective possible items;
acquiring, from among the scores of the respective possible items, a second score corresponding to the item that is registered in the accounting machine by the person by operating the accounting machine;
when a difference between the first score and the second score is equal to or greater than a threshold, determining that the item grasped by the person is different from the item registered in the accounting machine; and
generating an alert indicating abnormality of the item that is registered in the accounting machine.
2 . The non-transitory computer-readable recording medium according to claim 1 , wherein the accounting machine is a self-checkout terminal device.
3 . The non-transitory computer-readable recording medium according to claim 1 , wherein the process further includes:
the machine learning model is a contrastive language-image pre-training (CLIP) model that refers to reference original data in which item attributes are associated with each of a plurality of layers, and
by inputting the acquired video to the CLIP model, specifying a first item attribute contained in the video from the item attributes of a first layer;
based on the specified first item attribute, specifying second item attributes from the item attributes of a second layer under the first layer; and
by inputting the acquired video to the CLIP model, selecting the item attribute contained in the video as the possible item from the second item attributes.
4 . The non-transitory computer-readable recording medium according to claim 1 , wherein the process further includes, when generating the alert, causing the accounting machine to display a message that is previously set.
5 . The non-transitory computer-readable recording medium according to claim 1 , wherein the process further includes causing an information processing terminal device that a store staff uses to output the alert containing an identifier of the accounting machine in which the item with abnormality is registered.
6 . An information processing method that causes a computer to execute a process comprising:
inputting a video of respective possible items available in a store to a machine learning model trained to specify an item from the video, and calculating and storing scores indicating a level of reliability of the respective possible items;
acquiring a video of a person who grasps an item to be registered in an accounting machine;
inputting the acquired video, and calculating a first score that is highest among the scores of the respective possible items;
acquiring, from among the scores of the respective possible items, a second score corresponding to the item that is registered in the accounting machine by the person by operating the accounting machine;
when a difference between the first score and the second score is equal to or greater than a threshold, determining that the item grasped by the person is different from the item registered in the accounting machine; and
generating an alert indicating abnormality of the item that is registered in the accounting machine.
7 . The information processing method according to claim 6 , wherein the accounting machine is a self-checkout terminal device.
8 . The information processing method according to claim 6 , wherein the process further includes:
the machine learning model is a contrastive language-image pre-training (CLIP) model that refers to reference original data in which item attributes are associated with each of a plurality of layers, and
by inputting the acquired video to the CLIP model, specifying a first item attribute contained in the video from the item attributes of a first layer;
based on the specified first item attribute, specifying second item attributes from the item attributes of a second layer under the first layer; and
by inputting the acquired video to the CLIP model, selecting the item attribute contained in the video as the possible item from the second item attributes.
9 . The information processing method according to claim 6 , wherein the process further includes, when generating the alert, causing the accounting machine to display a message that is previously set.
10 . The information processing method according to claim 6 , wherein the process further includes causing an information processing terminal device that a store staff uses to output the alert containing an identifier of the accounting machine in which the item with abnormality is registered.
11 . An information processing device comprising:
a memory; and
a processor coupled to the memory and configured to:
input a video of respective possible items available in a store to a machine learning model trained to specify an item from the video, and calculate and store scores indicating a level of reliability of the respective possible items;
acquire a video of a person who grasps an item to be registered in an accounting machine;
input the acquired video, and calculating a first score that is highest among the scores of the respective possible items;
acquire, from among the scores of the respective possible items, a second score corresponding to the item that is registered in the accounting machine by the person by operating the accounting machine;
when a difference between the first score and the second score is equal to or greater than a threshold, determining that the item grasped by the person is different from the item registered in the accounting machine; and
generate an alert indicating abnormality of the item that is registered in the accounting machine.
12 . The information processing device according to claim 11 , wherein the accounting machine is a self-checkout terminal device.
13 . The information processing device according to claim 11 , wherein the processor is further configured to:
the machine learning model is a contrastive language-image pre-training (CLIP) model that refers to reference original data in which item attributes are associated with each of a plurality of layers, and
by inputting the acquired video to the CLIP model, specify a first item attribute contained in the video from the item attributes of a first layer;
based on the specified first item attribute, specifying second item attributes from the item attributes of a second layer under the first layer; and
by inputting the acquired video to the CLIP model, selecting the item attribute contained in the video as the possible item from the second item attributes.