Processing apparatus, processing method, and non-transitory storage medium
The present invention provides a processing apparatus ( 10 ) including: an acquisition unit ( 11 ) that acquires a plurality of images, captured by a plurality of cameras, of a product held by a customer with a hand; a detection unit ( 12 ) that detects an object from each of the plurality of images; a positional information generation unit ( 13 ) that generates positional information indicating a position in the image, for the each detected object; a product kind-related information generation unit ( 14 ) that generates product kind-related information that determines a product kind, for the each detected object; an extraction unit ( 15 ) that extracts a group of a plurality of objects which satisfies conditions with each other, the conditions being related to the positional information and the product kind-related information; and a product recognition result output unit ( 16 ) that outputs a product recognition result for the each extracted group.
1. A processing apparatus comprising:
at least one memory storing one or more instructions; and
at least one processor configured to execute the one or more instructions to:
acquire, by the at least one processor executing the instructions and from a first camera, a first image captured by the first camera from a first direction, the first image including a product held by a customer with a hand;
acquire, by the at least one processor executing the instructions and from a second camera, a second image captured by the second camera from a second direction different from the first direction, the second image including the product held by the customer with the hand;
detect, by the at least one processor executing the instructions, a first object from the first image;
detect, by the at least one processor executing the instructions, a second object from the second image;
generate, by the at least one processor executing the instructions, positional information indicating a position of the first object in the first image and a position of the second object in the second image;
generate, by the at least one processor executing the instructions and based on the first image and the second image, product kind-related information indicating a product kind of the first object and a product kind of the second object;
extract, by the at least one processor executing the instructions, a group of the first object and the second object that satisfies a positional condition and a product kind condition, the positional condition defined with respect to the position of the first object in the first image and the position of the second object in the second image, the product kind condition being defined with respect to the product kind of the first object and the product kind of the second object; and
output, by the at least one processor executing the instructions, a product recognition result of the first object and the second object of the extracted group.
2. The processing apparatus according to claim 1 , wherein
the positional condition is that a position of the first object in the first image satisfies a positional relationship when the second object in the second image are a same subject.
3. The processing apparatus according to claim 2 , wherein
the positional condition is that a position in which the first object may be present in a three-dimensional space estimated from setting information about a first camera and the position of the first object in the first image, and a position in which the second object may be present in the third-dimensional space estimated from setting information about the second camera, the position of the second object in the second image, and a relative relationship between the first camera and the another camera satisfy the positional relationship when the first object and the second object are the same subject.
4. The processing apparatus according to claim 1 , wherein
the product kind-related information is a feature value of an appearance of each of the first object and the second object, and
the product kind condition is that a similarity degree of the feature value is equal to or more than a reference value.
5. The processing apparatus according to claim 1 , wherein
the product kind-related information is product kind identification information determined based on a feature value of an appearance of each of the first object and the second object, and
the product kind condition is that the product kind identification information coincides.
6. The processing apparatus according to claim 5 , wherein
the product kind condition is that
the product kind coincides, and
a relationship between a feature portion of a first product determined based on the feature value of the appearance of the first object, and a feature portion of a second product determined based on the feature value of the appearance of the second object satisfies an orientation condition.
7. A processing method comprising:
acquiring, by at least one processor executing instructions stored on at least one memory, and from a first camera, a first image captured by the first camera from a first direction, the first image including a product held by a customer with a hand;
acquiring, by the at least one processor executing the instructions and from a second camera, a second image captured by the second camera from a second direction different from the first direction, the second image including the product held by the customer with the hand;
detecting, by the at least one processor executing the instructions, a first object from the first image;
detecting, by the at least one processor executing the instructions, a second object from the second image;
generating, by the at least one processor executing the instructions, positional information indicating a position of the first object in the first image and a position of the second object in the second image;
generating, by the at least one processor executing the instructions and based on the first image and the second image, product kind-related information indicating a product kind of the first object and a product kind of the second object;
extracting, by the at least one processor executing the instructions, a group of the first object and the second object that satisfies a positional condition and a product kind condition, the positional condition defined with respect to the position of the first object in the first image and the position of the second object in the second image, the product kind condition being defined with respect to the product kind of the first object and the product kind of the second object; and
outputting, by the at least one processor executing the instructions, a product recognition result of the first object and the second object of the extracted group.
8. A non-transitory storage medium storing a program including one or more instructions executable by at least one processor of a computer to perform processing comprising:
acquiring, by the at least one processor executing the instructions and from a first camera, a first image captured by the first camera from a first direction, the first image including a product held by a customer with a hand;
acquiring, by the at least one processor executing the instructions and from a second camera, a second image captured by the second camera from a second direction different from the first direction, the second image including the product held by the customer with the hand;
detecting, by the at least one processor executing the instructions, a first object from the first image;
detecting, by the at least one processor executing the instructions, a second object from the second image;
generating, by the at least one processor executing the instructions, positional information indicating a position of the first object in the first image and a position of the second object in the second image;
generating, by the at least one processor executing the instructions and based on the first image and the second image, product kind-related information indicating a product kind of the first object and a product kind of the second object;
extracting, by the at least one processor executing the instructions, a group of the first object and the second object that satisfies a positional condition and a product kind condition, the positional condition defined with respect to the position of the first object in the first image and the position of the second object in the second image, the product kind condition being defined with respect to the product kind of the first object and the product kind of the second object; and
outputting, by the at least one processor executing the instructions, a product recognition result of the first object and the second object of the extracted group.
9. The processing apparatus according to claim 1 , wherein
the at least one memory stores correspondence information indicating a relationship about which side of a product is captured by the second camera in a case where the first camera captures each side of the product, and
the at least one processor is further configured to execute the one or more instructions to:
determine, as a first side, which side of the product is captured by the first camera based on the first image;
determine, as a second side, which side of the product is captured by the second camera based on the second image;
judge if a relationship between the first side and the second side satisfies the relationship indicated in the correspondence information; and
output the product recognition result when the relationship between the first side and the second side satisfies the relationship indicated in the correspondence information.