Recording medium, alert generation method, and information processing apparatus
A non-transitory computer-readable recording medium stores therein an alert generation program that causes a computer to execute a process including acquiring video image of a person who is scanning a code of a commodity product to an accounting machine, specifying, by analyzing the acquired video image, from among a plurality of commodity product candidates that are set in advance, a commodity product candidate that corresponds to the commodity product that is included in the video image, acquiring an item of the commodity product that has been registered to the accounting machine by scanning the code of the commodity product to the accounting machine, and generating, based on an item of the specified commodity product candidate and the item of the commodity product acquired from the accounting machine, an alert that indicates an abnormality of the commodity product that has been registered to the accounting machine.
1 . A non-transitory computer-readable recording medium having stored therein an alert generation program that causes a computer to execute a process comprising:
acquiring video image of a person who is scanning a code of a commodity product to an accounting machine;
inputting the acquired video image to a zero-shot image classifier as a machine learning model to specify, from among a plurality of commodity product candidates arranged in a store in which the accounting machine is installed, a commodity product candidate that corresponds to the commodity product that is included in the video image;
acquiring an item of the commodity product that has been registered to the accounting machine by scanning the code of the commodity product to the accounting machine; and
generating, when an item of the commodity product candidate specified by the machine learning model and the item of the commodity product acquired from the accounting machine do not match, an alert that indicates an abnormality of the commodity product that has been registered to the accounting machine.
2 . The non-transitory computer-readable recording medium according to claim 1 , wherein
The machine learning model specifies,
from among a plurality of commodity product candidates that are set in advance, a commodity product candidate that corresponds to the commodity product that is included in the video image.
3 . The non-transitory computer-readable recording medium according to claim 2 , wherein
the inputting includes
inputting the video image to an image encoder that is included in the machine learning model,
inputting a plurality of texts that correspond to the plurality of commodity product candidates to a text encoder that is included in the machine learning model, and
specifying, based on a degree of similarity between a vector of the video image that is output by the image encoder and a vector of the text that is output by the text encoder, the commodity product candidate that corresponds to the commodity product that is included in the video image from among the plurality of commodity product candidates.
4 . The non-transitory computer-readable recording medium according to claim 3 , wherein
the machine learning model refers to reference source data in which an attribute of the commodity product is associated with each of a plurality of hierarchies, and
the inputting includes specifying the commodity product candidate by
inputting the video image to the image encoder,
inputting a text to the text encoder for each attribute of the commodity product that is included in a first hierarchy,
narrowing down, based on the degree of similarity between the vector of the video image that is output by the image encoder and the vector of the text that is output by the text encoder, the attribute that corresponds to the commodity product that is included in the video image from among the attributes of the commodity products that are included in the first hierarchy,
inputting the video image to the image encoder,
inputting a text to the text encoder for each attribute of the commodity product that is included in a second hierarchy obtained by narrowing down the attributes of the commodity products that are included in the first hierarchy, and
specifying, based on the degree of similarity between the vector of the video image that is output by the image encoder and the vector of the text that is output by the text encoder, the attribute that corresponds to the commodity product that is included in the video image from among the attributes of the commodity products that are included in the second hierarchy.
5 . The non-transitory computer-readable recording medium according to claim 1 , wherein the generating includes generating an alert that warns of a mismatch between the item of the specified commodity product candidate and the item of the commodity product acquired from the accounting machine.
6 . The non-transitory computer-readable recording medium according to claim 1 , wherein the generating includes generating an alert that includes either a price difference of an amount of money for a purchase between the item of the specified commodity product candidate and the item of the commodity product acquired from the accounting machine or identification information on the accounting machine.
7 . The non-transitory computer-readable recording medium according to claim 6 , wherein the process further includes transmitting the alert to a terminal that is used by a store clerk.
8 . The non-transitory computer-readable recording medium according to claim 1 , wherein the accounting machine is a self-service checkout terminal.
9 . The non-transitory computer-readable recording medium according to claim 2 , wherein
the inputting includes referring to, by the machine learning model, reference source data in which an attribute of the commodity product is associated with each of a plurality of hierarchies and specifying the plurality of commodity product candidates, and
the reference source data is the plurality of commodity product candidates that are arranged in a store at a present moment from among candidates for a large number of types of commodity products that are targeted for a replacement.
10 . An alert generation method comprising:
acquiring video image of a person who is scanning a code of a commodity product to an accounting machine;
inputting the acquired video image to a zero-shot image classifier as a machine learning model to specify, from among a plurality of commodity product candidates arranged in a store in which the accounting machine is installed, a commodity product candidate that corresponds to the commodity product that is included in the video image;
acquiring an item of the commodity product that has been registered to the accounting machine by scanning the code of the commodity product to the accounting machine; and
generating, when an item of the commodity product candidate specified by the machine learning model and the item of the commodity product acquired from the accounting machine do not match, an alert that indicates an abnormality of the commodity product that has been registered to the accounting machine.
11 . An information processing apparatus comprising:
a memory; and
a processor coupled to the memory and the processor configured to:
acquire video image of a person who is scanning a code of a commodity product to an accounting machine;
input the acquired video image to a zero-shot image classifier as a machine learning model to specify, from among a plurality of commodity product candidates arranged in a store in which the accounting machine is installed, a commodity product candidate that corresponds to the commodity product that is included in the video image;
acquire an item of the commodity product that has been registered to the accounting machine by scanning the code of the commodity product to the accounting machine; and
generate, when an item of the commodity product candidate specified by the machine learning model and the item of the commodity product acquired from the accounting machine do not match, an alert that indicates an abnormality of the commodity product that has been registered to the accounting machine.