Image shooting control apparatus and control method thereof, image capturing apparatus, and storage medium
In an image shooting control apparatus, an acquisition unit acquires image data obtained by an image sensor, a detection unit detects a subject from the image data, a map creation unit creates an environmental map, a storage unit stores history information that associates the subject with a divided area in which the subject exists among a plurality of divided areas obtained by dividing the environmental map, a determination unit finds probability of existence of the subject for each divided area based on the history information and determines an order of the plurality of divided areas to be searched for the subject based on the probability of existence, and a control unit controls an orientation of the image sensor so as to shoot the plurality of divided areas in the determined order.
1 . An image shooting control apparatus comprising one or more processors which function as:
an acquisition unit that acquires image data obtained through repeated shooting by an image sensor;
a detection unit that detects a subject from the image data;
a map creation unit that creates an environmental map based on information obtained by measuring surrounding environment;
a storage unit that stores a plurality of pieces of history information, each piece associating (i) the subject, (ii) time information indicating a time at which the subject was detected, and (iii) a divided area in which the subject was detected among a plurality of divided areas obtained by dividing the environmental map;
a determination unit that determines an order of the plurality of divided areas to be searched for the subject based on a plurality of stored pieces of the history information having different time stamps; and
a control unit that controls an orientation of the image sensor,
wherein the control unit controls the image sensor to shoot the plurality of divided areas in the order determined by the determination unit.
2 . The image shooting control apparatus according to claim 1 , wherein the map creation unit creates the environmental map based on information obtained from a sensor for Simultaneous Localization And Mapping (SLAM).
3 . The image shooting control apparatus according to claim 1 , wherein the control unit controls the orientation of the image sensor to shoot a divided area to be shot next determined in the order in a case where the subject is not detected from the image data of a divided area obtained by shooting.
4 . The image shooting control apparatus according to claim 1 , wherein the control unit controls the orientation of the image sensor to track the subject in a case where the subject is detected from the image data of a divided area obtained by shooting, and the storage unit stores the history information a predetermined number of times at predetermined time intervals.
5 . The image shooting control apparatus according to claim 1 , wherein the detection unit detects at least one subject, the storage unit stores the history information of each subject, and the determination unit finds the probability of existence of each subject.
6 . The image shooting control apparatus according to claim 1 , wherein, in a case where the subject is detected from the image data of a divided area obtained by shooting, after the storage unit stores the history information a predetermined number of time, the control unit controls the orientation of the image sensor to shoot a divided area to be shot next determined in the order.
7 . The image shooting control apparatus according to claim 1 , wherein the determination unit re-determines the order in a case where a predetermined time has passed since the order was determined.
8 . The image shooting control apparatus according to claim 1 , wherein the determination unit re-determines the order in a case where a predetermined number of pieces of the history information are stored since the order was determined.
9 . The image shooting control apparatus according to claim 1 , wherein the history information includes information on time when the image data in which the subject is detected was acquired.
10 . The image shooting control apparatus according to claim 9 , wherein the determination unit weights pieces of the history information based on the information on time to determine the probability of existence.
11 . The image shooting control apparatus according to claim 10 , further comprising an operation unit that accepts an operation from outside,
wherein the determination unit puts more weight on a piece of the history information having the information on time closer to time designated using the operation unit.
12 . The image shooting control apparatus according to claim 1 , further comprising an operation unit that accepts an operation from outside,
wherein the determination unit finds the probability of existence based on the history information of a subject designated using the operation unit.
13 . The image shooting control apparatus according to claim 1 , further comprising a communication unit that communicates with an image capturing apparatus that comprises the image sensor,
wherein the acquisition unit acquires the image data from the image capturing apparatus through the communication unit, and
wherein the control unit sends a control signal to the image capturing apparatus through the communication unit to control the orientation of the image sensor.
14 . The image shooting control apparatus according to claim 1 , wherein the map creation unit generates the environmental map in a world coordinate system.
15 . The image shooting control apparatus according to claim 1 , wherein the determination unit obtains the probability of existence of the subject for each of the plurality of divided areas based on a number of times the subject has been detected and a weight that is set based on the time at which the subject was detected.
16 . The image shooting control apparatus according to claim 1 having a mode for obtaining the probability of existence of the subject that is designated by a user.
17 . The image shooting control apparatus according to claim 1 further comprising an image sensor,
wherein the control unit controls the image sensor to track the subject detected by the detection unit and to shoot images of the subject.
18 . The image shooting control apparatus according to claim 1 , wherein the order of the plurality of divided areas to be searched by the determination unit initiates with a divided area having a highest probability of existence of the subject and terminates with a divided area having the lowest probability of existence of the subject.
19 . The image shooting control apparatus according to claim 1 , wherein the determination unit obtains probability of existence of the subject for each of the plurality of divided areas based on the plurality of stored pieces of the history information having different time stamps, and determines the order of the plurality of divided areas to be searched for the subject based on the probability of existence.
20 . An image capturing apparatus comprising:
an image shooting control apparatus having one or more processors which function as:
an acquisition unit that acquires image data obtained through repeated shooting by an image sensor;
a detection unit that detects a subject from the image data;
a map creation unit that creates an environmental map based on information obtained by measuring surrounding environment;
a storage unit that stores a plurality of pieces of history information, each piece associating (i) the subject, (ii) time information indicating a time at which the subject was detected, and (iii) a divided area in which the subject was detected among a plurality of divided areas obtained by dividing the environmental map;
a determination unit that determines an order of the plurality of divided areas to be searched for the subject based on a plurality of stored pieces of the history information having different time stamps; and
a control unit that controls an orientation of the image sensor, the image sensor; and
a drive unit that drives the image sensor to change its orientation based on the control of the control unit,
wherein the control unit controls the image sensor to shoot the plurality of divided areas in the order determined by the determination unit.
21 . A control method of an image shooting control apparatus comprising:
causing an image sensor to repeat shooting and output image data;
detecting a subject from the image data;
creating an environmental map based on information obtained by measuring surrounding environment;
storing a plurality of pieces of history information, each piece associating (i) the subject, (ii) time information indicating a time at which the subject was detected, and (iii) a divided area in which the subject was detected among a plurality of divided areas obtained by dividing the environmental map;
determining an order of the plurality of divided areas to be searched for the subject based on a plurality of stored pieces of the history information having different time stamps; and
changing an orientation of the image sensor based on the order.
22 . A non-transitory computer-readable storage medium, the storage medium storing a program that is executable by the computer, wherein the program includes program code for causing the computer to executes each step of the control method of the image capturing apparatus comprising:
causing an image sensor to repeat shooting and output image data;
detecting a subject from the image data;
creating an environmental map based on information obtained by measuring surrounding environment;
storing a plurality of pieces of history information, each piece associating (i) the subject, (ii) time information indicating a time at which the subject was detected, and (iii) a divided area in which the subject was detected among a plurality of divided areas obtained by dividing the environmental map;
determining an order of the plurality of divided areas to be searched for the subject based on a plurality of stored pieces of the history information having different time stamps; and
changing an orientation of the image sensor based on the order.