Main subject determining apparatus, image capturing apparatus, main subject determining method, and storage medium
There is provided a main subject determining apparatus. A subject detecting unit detects a plurality of subjects from a first image. An obtaining unit obtains posture information of each of the plurality of subjects. For each of the plurality of subjects, a calculating unit calculates, on the basis of the posture information of the subject, a reliability corresponding to a likelihood that the subject is a main subject in the first image. A determining unit determines the main subject of the first image from among the plurality of subjects on the basis of the plurality of reliabilities calculated for the plurality of subjects.
1 . A main subject determining apparatus comprising at least one processor and/or at least one circuit which functions as:
a subject detecting unit configured to detect a plurality of subjects from a first image;
an obtaining unit configured to obtain posture information of each of the plurality of subjects; and
a determining unit configured to determine, as a main subject of the first image, one of the plurality of subjects on the basis of the posture information of each of the plurality of subjects using a machine learning model learned by a decision tree,
wherein the determining unit determines whether or not the plurality of subjects include a subject having a reliability that exceeds a first threshold, and
wherein the determining unit changes the first threshold in accordance with a type of a shooting mode set in an image capturing apparatus that uses information of the main subject of the first image.
2 . The main subject determining apparatus according to claim 1 ,
wherein the subject detecting unit detects a plurality of persons as the plurality of subjects; and
the obtaining unit obtains positions of a plurality of joints of each of the persons.
3 . The main subject determining apparatus according to claim 1 , wherein the at least one processor and/or at least one circuit further functions as:
an object detecting unit configured to detect, from the first image, an object of a predetermined type that is different from the plurality of subjects; and
a calculating unit configured to calculate, for each of the plurality of subjects, a reliability on the basis of the posture information of the subject and at least one of a position and a size of the object,
wherein the determining unit determines the main subject of the first image from among the plurality of subjects on the basis of the plurality of reliability calculated for the plurality of subjects.
4 . The main subject determining apparatus according to claim 3 ,
wherein the object detecting unit detects an object of a type determined on the basis of a shooting scene of the first image as the object of the predetermined type.
5 . The main subject determining apparatus according to claim 3 , wherein the at least one processor and/or at least one circuit further functions as:
a setting unit configured to set a determination target region of the main subject in the first image on the basis of a position of the object,
wherein the plurality of subjects are subjects present in the determination target region.
6 . The main subject determining apparatus according to claim 1 , wherein the at least one processor and/or at least one circuit further functions as:
a predicting unit configured to predict a position of the main subject in the first image on the basis of a position of a main subject in a plurality of images which are earlier than the first image in time series; and
a setting unit configured to set a determination target region of the main subject in the first image on the basis of the predicted position,
wherein the plurality of subjects are subjects present in the determination target region.
7 . The main subject determining apparatus according to claim 1 , wherein the at least one processor and/or at least one circuit further functions as:
a predicting unit configured to predict a position of the main subject in the first image on the basis of a position of a main subject in a plurality of images which are earlier than the first image in time series,
wherein when the plurality of subjects do not include a subject having a reliability that exceeds the first threshold, the determining unit determines the main subject of the first image on the basis of the predicted position.
8 . The main subject determining apparatus according to claim 1 , wherein the at least one processor and/or at least one circuit further functions as:
a predicting unit configured to predict a position of the main subject in the first image on the basis of a position of a main subject in a plurality of images which are earlier than the first image in time series; and
a calculating unit configured to, for each of the plurality of subjects, a reliability corresponding to a likelihood that the subject is a main subject on the basis of the posture information of the subject,
wherein the determining unit obtains a maximum value of the plurality of reliabilities calculated for the plurality of subjects, and determines whether or not the plurality of reliabilities include a plurality of reliabilities for which an absolute value of a difference from the maximum value is less than a second threshold; and
when the plurality of reliabilities include a plurality of reliabilities for which an absolute value of a difference from the maximum value is less than the second threshold, the determining unit determines the main subject of the first image on the basis of the predicted position.
9 . The main subject determining apparatus according to claim 8 ,
wherein the determining unit changes the second threshold in accordance with a type of a shooting mode set in an image capturing apparatus that uses information of the main subject of the first image.
10 . The main subject determining apparatus according to claim 1 , wherein the at least one processor and/or at least one circuit further functions as:
a calculating unit configured to, for each of the plurality of subjects, a reliability corresponding to a likelihood that the subject is a main subject on the basis of the posture information of the subject,
wherein the determining unit obtains a maximum value of the plurality of reliabilities, and determines whether or not the plurality of reliabilities include a plurality of reliabilities for which an absolute value of a difference from the maximum value is less than a second threshold; and
when the plurality of reliabilities include a plurality of reliabilities for which an absolute value of a difference from the maximum value is less than the second threshold, the determining unit determines, as the main subject of the first image, a subject, among subjects having reliabilities for which an absolute value of the difference from the maximum value is less than the second threshold, that is closest to a center of the first image.
11 . The main subject determining apparatus according to claim 1 , wherein the at least one processor and/or at least one circuit further functions as:
a calculating unit configured to, for each of the plurality of subjects, a reliability corresponding to a likelihood that the subject is a main subject on the basis of the posture information of the subject.
12 . The main subject determining apparatus according to claim 11 ,
wherein the calculating unit calculates the reliability on the basis of a result of machine learning performed in advance.
13 . The main subject determining apparatus according to claim 11 ,
wherein on the basis of the plurality of reliabilities, the determining unit determines, as the main subject of the first image, a subject, among the plurality of subjects, that is most likely to be the main subject of the first image.
14 . The main subject determining apparatus according to claim 1 ,
wherein the obtaining unit obtains posture information of each of the plurality of subjects by obtaining positions of a plurality of joints of each of the plurality of subjects.
15 . An image capturing apparatus, comprising:
the main subject determining apparatus according to claim 1 ; and
an image sensor,
wherein the first image is an image captured by the image sensor.
16 . The image capturing apparatus according to claim 15 , wherein the at least one processor and/or at least one circuit further functions as:
a calculating unit configured to, for each of the plurality of subjects, a reliability corresponding to a likelihood that the subject is a main subject on the basis of the posture information of the subject; and
a recording control unit configured to, when the reliability corresponding to the main subject of the first image determined by the determining unit exceeds a third threshold, perform control to start a process of recording an image captured by the image sensor into a recording medium.
17 . The image capturing apparatus according to claim 15 , wherein the at least one processor and/or at least one circuit further functions as:
an image capturing control unit configured to perform image capturing control of the image sensor on the basis of information of the main subject of the first image determined by the determining unit.
18 . A main subject determining method executed by a main subject determining apparatus, comprising:
detecting a plurality of subjects from a first image;
obtaining posture information of each of the plurality of subjects; and
determining, as a main subject of the first image, one of the plurality of subjects on the basis of the posture information of each of the plurality of subjects using a machine learning model learned by a decision tree,
wherein the determining:
determines whether or not the plurality of subjects include a subject having a reliability that exceeds a first threshold, and
changes the first threshold in accordance with a type of a shooting mode set in an image capturing apparatus that uses information of the main subject of the first image.
19 . A non-transitory computer-readable storage medium which stores a program for causing a computer to execute a main subject determining method comprising:
detecting a plurality of subjects from a first image;
obtaining posture information of each of the plurality of subjects; and
determining, as a main subject of the first image, one of the plurality of subjects on the basis of the posture information of each of the plurality of subjects using a machine learning model learned by a decision tree,
wherein the determining:
determines whether or not the plurality of subjects include a subject having a reliability that exceeds a first threshold, and
changes the first threshold in accordance with a type of a shooting mode set in an image capturing apparatus that uses information of the main subject of the first image.
20 . An image capturing apparatus comprising:
an image sensor;
at least one processor and/or at least one circuit which functions as:
a subject detecting unit configured to detect a plurality of persons from a first image;
an obtaining unit configured to obtain posture information of each of the plurality of persons;
a determining unit configured to determine, as a main subject of the first image, one of the plurality of persons on the basis of the posture information of each of the plurality of persons using a machine learning model learned by a decision tree,
wherein the determining unit determines whether or not the plurality of persons include a person having a reliability that exceeds a first threshold, and
wherein the determining unit changes the first threshold in accordance with a type of a shooting mode set in the image capturing apparatus that uses information of the main subject of the first image.