Information processing device, information processing method, and program
The present invention is to predict a production amount of a target object with higher accuracy. An information processing device that predicts a production amount of a target object, the information processing device including: an area extraction module configured to extract a target object area in which the target object exists in a photographed image of the target object photographed by a photographing device; a correction module configured to correct a size of the target object area on the basis of a distance from the photographing device to the target object, detected by a distance sensor; and a prediction module configured to predict the production amount of the target object on the basis of a size of the target object area corrected by the correction module.
1 . An information processing device that predicts a production amount of a target object, the information processing device comprising:
a memory; and
a processor coupled to the memory, the processor being configured to:
extract a target object area in which the target object exists in a photographed image of the target object photographed by a photographing device;
correct a size of the target object area on a basis of a distance from the photographing device to the target object, detected by a distance sensor;
predict the production amount of the target object on a basis of a size of the target object area corrected;
extract, the target object area, from each of a plurality of the photographed images, each having a photographing range, the photographic ranges at least partly overlapping with each other, the plurality of photographed images being photographed by the photographing device moving in a predetermined direction,
when at least two of the plurality of photographed images include an image of the target object existing at a same position, select a largest target object area among a plurality of the target object areas corresponding to the target object existing at the same position, as the target object area of the target object existing at the same position;
wherein the processor further configured to extract a first photographed image photographed from an outside of a first boundary surface of the prediction area and a second photographed image photographed from an outside of a second boundary surface, the second boundary surface being on a back side of the first boundary surface, and
when the first photographed image and the second photographed image include the image of the target object existing at the same position, select a larger target object area from the two target object areas corresponding to the target object existing at the same position as the target object area of the target object existing at the same position, and predicts the production amount on a basis of a size of the selected target object area.
2 . The information processing device according to claim 1 , wherein the processor is further configured to extract the target objection area from each of a plurality of the photographed images corresponding to different photographing ranges in a prediction area in which the production amount is to be predicted and including an entire range in the prediction area, and extracts the target object area in the photographed image as the processing target, and
predict the production amount of the target object in the prediction area on a basis of a size of each of a plurality of the target object areas corrected.
3 . The information processing device according to claim 1 , wherein the plurality of photographed images are images photographed under a different condition of at least one of a photographing position and a photographing direction, and
wherein the processor further is configured to specify a photographing range corresponding to each of the photographed images photographed by the photographing device on a basis of the photographing position and the photographing direction of each photographed image, and extract a plurality of the photographed images whose photographing ranges overlap with each other by a predetermined length in a predetermined direction from a storage section in which the photographed images photographed by the photographing device are stored, as target images from which the target object area is to be extracted, on a basis of the photographing range.
4 . The information processing device according to claim 1 , wherein the processor further configured to extract the target object area from the photographed image using an estimation model of the target object, and
the estimation model is generated by:
applying learning data to a plurality of learning models, the learning data being a plurality of photographed images for learning including an annotated area as the target object, and
modifying the annotated area on a basis of a result of estimation by each learning model and the annotated area.
5 . The information processing device according to claim 1 , wherein the processor further configured to extract a first photographed image photographed from an outside of a first boundary surface of the prediction area and a second photographed image photographed from an outside of a second boundary surface, the second boundary surface being on a back side of the first boundary surface,
the processor further configured to extract the target object area existing in a range of a first distance from the first boundary surface toward the second boundary surface among target objects included in the first photographed image, and further extracts the target object area existing in a range of a second distance from the second boundary surface toward the first boundary surface, and
a sum of the first distance and the second distance is a distance between the first boundary surface and the second boundary surface.
6 . The information processing device according to claim 1 , wherein the processor is further configured to predict the production amount using a first regression model obtained from data on the size and the production amount of the target object.
7 . The information processing device according to claim 6 , wherein the first regression model indicates a relationship between the production amount and at least one of a barycentric coordinate of the target object, an inclination of the target object, a position of the photographing device, and a photographing direction of the photographing device, and
wherein the processor is further configured to predict the production amount on a basis of the first regression model and at least one of the barycentric coordinate, the inclination, the position, and the photographing direction.
8 . The information processing device according to claim 1 , wherein the processor is further configured to obtain a volume of the target object area on a basis of a size of the target object area and a three-dimensional position of the target object area obtained from a distance image, and predicts the production amount using a second regression model obtained from data on the volume and the production amount of the target object.
9 . An information processing method executed by an information processing device that predicts a production amount of a target object, the information processing method comprising:
an area extraction step of extracting a target object area in which the target object exists in a photographed image of the target object photographed by a photographing device;
a correction step of correcting a size of the target object area on a basis of a distance from the photographing device to the target object, detected by a distance sensor; and
a prediction step of predicting the production amount of the target object on a basis of a size of the target object area corrected at the correction step,
wherein at the area extraction step, extracting the target object area from each of a plurality of photographed images, each having a photographic range, the photographing ranges at least partly overlapping with each other, are set as the a processing target, the plurality of photographed images being photographed by the photographing device moving in a predetermined direction, and
at the prediction step, when at least two of the plurality of photographed images include an image of the target object existing at a same position, a largest target object area among a plurality of the target object areas corresponding to the target object existing at a same position, is selected as the target object area of the target object existing at the same position,
wherein the information processing method further comprises an extraction step of extracting a first photographed image photographed from an outside of a first boundary surface of the prediction area and a second photographed image photographed from an outside of a second boundary surface, the second boundary surface being on a back side of the first boundary surface, and
when the first photographed image and the second photographed image include the image of the target object existing at the same position, selecting a larger target object area from the two target object areas corresponding to the target object existing at the same position as the target object area of the target object existing at the same position, and predicting the production amount on a basis of a size of the selected target object area.
10 . A non-transitory computer-readable recording medium storing a program for causing a computer to execute a process comprising:
extracting a target object area in which the target object exists in a photographed image of the target object photographed by a photographing device;
collecting a size of the target object area on a basis of a distance from the photographing device to the target object, detected by a distance sensor; and
predicting a production amount of the target object on a basis of a size of the target object area corrected by the collecting,
wherein the extracting, the target object area from each of a plurality of the photographed images, each having a photographic range, the photographing ranges at least partly overlapping with each other, the plurality of photographed images being photographed by the photographing device moving in a predetermined direction, and
when at least two of the plurality of photographed images include an image of the target object, selecting a largest target object area among a plurality of the target object existing at a same position, areas corresponding to the target object existing at the same position, as the target object area of the target object existing at the same position,
wherein the process comprises extracting a first photographed image photographed from an outside of a first boundary surface of the prediction area and a second photographed image photographed from an outside of a second boundary surface, the second boundary surface being on a back side of the first boundary surface, and
when the first photographed image and the second photographed image include the image of the target object existing at the same position, selecting a larger target object area from the two target object areas corresponding to the target object existing at the same position as the target object area of the target object existing at the same position, and predicting the production amount on a basis of a size of the selected target object area.
11 . The information processing device according to claim 1 , wherein the plurality of photographed images overlap with each other in the predetermined direction by a range of a half or more of the photographing range.