Inference apparatus, image capturing apparatus, training apparatus, inference method, training method, and storage medium
There is provided an inference apparatus. An inference unit performs inference with use of a machine learning model based on a subject region including a subject within an image obtained through shooting, and on a plurality of distance information pieces detected from a plurality of focus detection regions inside the subject region, thereby generating an inference result indicating a distance information piece corresponding to the subject or a distance information range corresponding to the subject. The machine learning model is a model that has been trained to suppress a contribution made to the inference result by one or more distance information pieces that are not based on the subject among the plurality of distance information pieces.
1 . An image capturing apparatus comprising at least one processor and/or at least one circuit which functions as:
an image capturing unit configured to generate an image through shooting;
a first detection unit configured to detect a subject region from the image;
a second detection unit configured to detect a plurality of distance information pieces from a plurality of focus detection regions inside the subject region; and
an inference unit configured to perform inference with use of a machine learning model based on the subject region including a subject within the image obtained through shooting, and on the plurality of distance information pieces detected from the plurality of focus detection regions inside the subject region, thereby generating an inference result indicating a distance information piece corresponding to the subject or a distance information range corresponding to the subject,
wherein the machine learning model is a model that has been trained to suppress a contribution made to the inference result by one or more distance information pieces that are not based on the subject among the plurality of distance information pieces.
2 . The image capturing apparatus according to claim 1 , wherein
the subject region includes a first region where the subject exists, and a second region where the subject does not exist, and
the one or more distance information pieces that are not based on the subject among the plurality of distance information pieces include one or more distance information pieces corresponding to one or more focus detection regions corresponding to the second region among the plurality of focus detection regions.
3 . The image capturing apparatus according to claim 1 , wherein
the one or more distance information pieces that are not based on the subject among the plurality of distance information pieces include one or more distance information pieces with a detection error that exceeds a predetermined extent among the plurality of distance information pieces.
4 . The image capturing apparatus according to claim 1 , wherein the at least one processor and/or the at least one circuit further functions as
a first adjustment unit configured to adjust a focus of an optical system used in the shooting based on the inference result.
5 . The image capturing apparatus according to claim 4 , wherein
the inference result indicates the distance information range corresponding to the subject, and
the first adjustment unit adjusts the focus of the optical system based on a distance information piece that is included in the distance information range among the plurality of distance information pieces.
6 . The image capturing apparatus according to claim 1 , wherein
the inference result indicates the distance information range corresponding to the subject, and
the at least one processor and/or the at least one circuit further functions as
a second adjustment unit configured to adjust a diaphragm of an optical system used in the shooting based on the distance information range.
7 . The image capturing apparatus according to claim 6 , wherein
the second adjustment unit adjusts the diaphragm of the optical system based on the distance information range so that the subject is included in a depth of field.
8 . The image capturing apparatus according to claim 1 , wherein
the inference result indicates the distance information range corresponding to the subject, and
the at least one processor and/or the at least one circuit further functions as
a display unit configured to display information for giving notice of a focus detection region corresponding to a distance information piece that is included in the distance information range among the plurality of distance information pieces.
9 . The image capturing apparatus according to claim 1 , wherein
the plurality of distance information pieces detected from the plurality of focus detection regions are a plurality of defocus amounts detected from the plurality of focus detection regions.
10 . The image capturing apparatus according to claim 9 , wherein
the inference result indicates a defocus amount corresponding to the subject or a defocus amount range corresponding to the subject.
11 . The image capturing apparatus according to claim 1 , wherein
the first detection unit detects a plurality of subject regions each of which partially includes each of a plurality of subjects,
the second detection unit detects a plurality of distance information pieces for each of the plurality of subject regions, and
the inference unit generates an inference result for each of the plurality of subjects.
12 . The image capturing apparatus according to claim 11 , wherein
the plurality of subjects correspond to a plurality of parts of a subject of a predetermined type.
13 . An inference apparatus comprising at least one processor and/or at least one circuit which functions as:
an image capturing unit configured to generate an image through shooting;
an obtainment unit configured to obtain the image obtained through shooting, information of a subject region including a subject within the image, and a plurality of distance information pieces that respectively correspond to a plurality of regions inside the subject region; and
an inference unit configured to perform inference with use of a machine learning model using, as inputs, the image, the information of the subject region, and the plurality of distance information pieces, thereby generating an inference result indicating a distance information piece corresponding to the subject or a distance information range corresponding to the subject.
14 . The inference apparatus according to claim 13 , wherein
the obtainment unit obtains a plurality of distance information pieces corresponding to a plurality of parts of the subject, and
the inference unit generates an inference result indicating a plurality of distance information ranges corresponding to the plurality of parts.
15 . An image capturing apparatus, comprising:
the inference apparatus according to claim 14 ,
wherein the at least one processor and/or the at least one circuit further functions as:
a determination unit configured to, based on priority degrees of the plurality of parts, determine a part to be focused on from the plurality of distance information ranges corresponding to the plurality of parts output from the inference apparatus.
16 . The image capturing apparatus according to claim 15 , wherein
the determination unit compares the plurality of distance information ranges corresponding to the plurality of parts with one another, thereby determining the part to be focused on from the plurality of distance information ranges.
17 . A training apparatus comprising at least one processor and/or at least one circuit which functions as:
an image capturing unit configured to generate an image through shooting;
a first detection unit configured to detect a subject region from the image; and
a second detection unit configured to detect a plurality of distance information pieces from a plurality of focus detection regions inside the subject region;
an inference unit configured to perform inference with use of a machine learning model based on the subject region including a subject within the image obtained through shooting, and on the plurality of distance information pieces detected from the plurality of focus detection regions inside the subject region, thereby generating an inference result indicating a distance information piece corresponding to the subject or a distance information range corresponding to the subject; and
a training unit configured to train the machine learning model so that the inference result approaches ground truth information to which a contribution made by one or more distance information pieces that are not based on the subject among the plurality of distance information pieces is suppressed.
18 . An inference method executed by an inference apparatus, comprising:
generating an image through shooting;
detecting a subject region from the image;
detecting a plurality of distance information pieces from a plurality of focus detection regions inside the subject region; and
performing inference with use of a machine learning model based on the subject region including a subject within the image obtained through shooting, and on the plurality of distance information pieces detected from the plurality of focus detection regions inside the subject region, thereby generating an inference result indicating a distance information piece corresponding to the subject or a distance information range corresponding to the subject,
wherein the machine learning model is a model that has been trained to suppress a contribution made to the inference result by one or more distance information pieces that are not based on the subject among the plurality of distance information pieces.
19 . A training method executed by a training apparatus, comprising:
generating an image through shooting;
detecting a subject region from the image;
detecting a plurality of distance information pieces from a plurality of focus detection regions inside the subject region;
performing inference with use of a machine learning model based on the subject region including a subject within the image obtained through shooting, and on the plurality of distance information pieces detected from the plurality of focus detection regions inside the subject region, thereby generating an inference result indicating a distance information piece corresponding to the subject or a distance information range corresponding to the subject; and
training the machine learning model so that the inference result approaches ground truth information to which a contribution made by one or more distance information pieces that are not based on the subject among the plurality of distance information pieces is suppressed.
20 . A non-transitory computer-readable storage medium which stores a program for causing a computer to execute an inference method comprising:
generating an image through shooting;
detecting a subject region from the image;
detecting a plurality of distance information pieces from a plurality of focus detection regions inside the subject region;
performing inference with use of a machine learning model based on the subject region including a subject within the image obtained through shooting, and on the plurality of distance information pieces detected from the plurality of focus detection regions inside the subject region, thereby generating an inference result indicating a distance information piece corresponding to the subject or a distance information range corresponding to the subject,
wherein the machine learning model is a model that has been trained to suppress a contribution made to the inference result by one or more distance information pieces that are not based on the subject among the plurality of distance information pieces.
21 . A non-transitory computer-readable storage medium which stores a program for causing a computer to execute a training method comprising:
generating an image through shooting;
detecting a subject region from the image;
detecting a plurality of distance information pieces from a plurality of focus detection regions inside the subject region;
performing inference with use of a machine learning model based on the subject region including a subject within the image obtained through shooting, and on the plurality of distance information pieces detected from the plurality of focus detection regions inside the subject region, thereby generating an inference result indicating a distance information piece corresponding to the subject or a distance information range corresponding to the subject; and
training the machine learning model so that the inference result approaches ground truth information to which a contribution made by one or more distance information pieces that are not based on the subject among the plurality of distance information pieces is suppressed.