Region importance estimation
To provide a technique which makes it possible to suitably estimate an important region and a non-important region in an image, an information processing apparatus includes: an obtaining means for obtaining input data which includes at least one of image data and point cloud data; an estimating means for estimating levels of importance with respect to a respective plurality of regions which are included in a frame indicated by the input data; a replacing means for generating replaced data by replacing at least one of the plurality of regions, which are included in the input data, with alternative data in accordance with the levels of importance; an evaluating means for deriving an evaluation value by referring to the replaced data; and a training means for training the estimating means with reference to the evaluation value.
1 . An information processing apparatus comprising
at least one processor,
the at least one processor carrying out:
an obtaining process of obtaining input data which includes at least one of image data and point cloud data;
an estimating process of estimating, with use of an estimation model, levels of importance with respect to a respective plurality of regions which are included in a frame indicated by the input data;
a replacing process of generating replaced data by replacing at least one of the plurality of regions, which are included in the input data, with alternative data in accordance with the levels of importance, wherein the alternative data has a data size smaller than a data size of the input data for the at least one of the plurality of regions;
an evaluating process of deriving an evaluation value by referring to the replaced data; and
a training process of training the estimation model with reference to the evaluation value,
wherein in the evaluating process, the at least one processor derives the evaluation value by referring to
an output obtained from a given controller in a case where the input data is inputted into the given controller and
an output obtained from the given controller in a case where the replaced data is inputted into the given controller,
wherein the given controller is an autonomous operation controller of a movable body, and the output is a control command for the movable body.
2 . The information processing apparatus as set forth in claim 1 , wherein in the evaluating process, the at least one processor derives the evaluation value by further referring to the input data.
3 . The information processing apparatus as set forth in claim 1 , wherein:
in the evaluating process, the at least one processor derives the evaluation value as a difference between (i) the output obtained from the given controller in the case where the input data is inputted into the given controller and (ii) the output obtained from the given controller in the case where the replaced data is inputted into the given controller; and
in the training process, the at least one processor trains the estimation model so that the evaluation value becomes low.
4 . The information processing apparatus as set forth in claim 1 , wherein in the replacing process, the at least one processor replaces, with the alternative data, one or more of the plurality of regions which one or more have been selected in ascending order of the levels of importance and have a given proportion in the frame.
5 . The information processing apparatus as set forth in claim 4 , wherein:
in the replacing process, the at least one processor generates the replaced data for each of a plurality of given proportions which differ from each other; and
in the evaluating process, the at least one processor
derives a preliminary evaluation value with respect to each replaced data generated in the replacing, and
derives the evaluation value by averaging preliminary evaluation values.
6 . The information processing apparatus as set forth in claim 1 , wherein in the evaluating process, the at least one processor derives the evaluation value further with reference to a data size of the replaced data.
7 . The information processing apparatus as set forth in claim 6 , wherein in the training process, the at least one processor trains the estimation model so that the data size of the replaced data becomes small.
8 . The information processing apparatus as set forth in claim 1 , wherein the alternative data used in the replacing process is data which includes at least one of noise and image data that has a large quantization error than that in the input data.
9 . The information processing apparatus as set forth in claim 1 , wherein in the estimating process, the at least one processor estimates the levels of importance with respect to the respective plurality of regions which are included in the frame indicated by the input data, with use of reference data relating to the frame.
10 . The information processing apparatus as set forth in claim 9 , wherein the reference data includes a segmentation image which is obtained by applying a segmentation process to the frame.
11 . The information processing apparatus as set forth in claim 9 , wherein the reference data includes a depth map which corresponds to the frame.
12 . The information processing apparatus as set forth in claim 9 , wherein the reference data includes an object detecting result which is obtained by applying, to the frame, a process of detecting an object.
13 . The information processing apparatus as set forth in claim 1 , wherein in the estimating process, the at least one processor estimates the levels of importance with use of a self-attention module.
14 . The information processing apparatus as set forth in claim 1 , wherein:
the evaluation value includes a reward value derived from the output; and
in the training process, the at least one processor trains the estimation model so that the reward value becomes high.
15 . The information processing apparatus as set forth in claim 1 , wherein:
the evaluation value is a loss value derived from the output; and
in the training process, the at least one processor trains the estimation model so that the loss value becomes low.
16 . An information processing method comprising:
obtaining input data which includes at least one of image data and point cloud data;
estimating, with use of an estimation model, levels of importance with respect to a respective plurality of regions which are included in a frame indicated by the input data;
generating replaced data by replacing at least one of the plurality of regions, which are included in the input data, with alternative data in accordance with the levels of importance, wherein the alternative data has a data size smaller than a data size of the input data for the at least one of the plurality of regions;
deriving an evaluation value by referring to the replaced data; and
training the estimation model with reference to the evaluation value,
wherein the evaluation value is derived by referring to
an output obtained from a given controller in a case where the input data is inputted into the given controller and
an output obtained from the given controller in a case where the replaced data is inputted into the given controller,
wherein the given controller is an autonomous operation controller of a movable body, and the output is a control command for the movable body.
17 . A computer-readable non-transitory recording medium in which a program is recorded, the program being for causing a computer to carry out:
an obtaining process of obtaining input data which includes at least one of image data and point cloud data;
an estimating process of estimating, with use of an estimation model, levels of importance with respect to a respective plurality of regions which are included in a frame indicated by the input data;
a replacing process of generating replaced data by replacing at least one of the plurality of regions, which are included in the input data, with alternative data in accordance with the levels of importance, wherein the alternative data has a data size smaller than a data size of the input data for the at least one of the plurality of regions;
an evaluating process of deriving an evaluation value by referring to the replaced data; and
a training process of training the estimation model with reference to the evaluation value,
wherein in the evaluating process, the computer derives the evaluation value by referring to
an output obtained from a given controller in a case where the input data is inputted into the given controller and
an output obtained from the given controller in a case where the replaced data is inputted into the given controller,
wherein the given controller is an autonomous operation controller of a movable body, and the output is a control command for the movable body.