Training apparatus, classification apparatus, training method, classification method, and program
To provide an efficient training process even in a case where training images having a limited variation of shooting angles are available. Solution to Problem A training apparatus ( 10 ) comprises: feature extraction section ( 11 ) for extracting source domain feature values from input source domain image data and for extracting target domain feature values from input target domain image data; angle conversion section ( 12 ) for generating converted source domain feature values by converting the source domain feature values as if the converted source domain feature values are extracted from source domain image data having different angles from the input source domain image data, and generating converted target domain feature values by converting the target domain feature values as if the converted target domain feature values are extracted from target domain image data having different angles from the input target domain image data; class prediction section( 13 ) for predicting source domain class prediction values from the source domain feature values and the converted source domain feature values, and predicting target domain class prediction values from the target domain feature values and the converted target domain feature values; and updating section ( 14 ) for updating at least one of (i) the feature extraction section, (ii) the angle conversion section, and (iii) the class prediction section.
1 . A training apparatus comprising at least one processor, the at least one processor carrying out:
an extraction process for extracting, by one or more feature extractors, source domain feature values from input source domain image data and for extracting target domain feature values from input target domain image data;
an angle conversion process for generating, by one or more angle converters:
converted source domain feature values by converting the source domain feature values as if the converted source domain feature values are extracted from source domain image data having different angles from the input source domain image data, and
converted target domain feature values by converting the target domain feature values as if the converted target domain feature values are extracted from target domain image data having different angles from the input target domain image data;
a class prediction process for predicting, by one or more class predictors:
source domain class prediction values from the source domain feature values and the converted source domain feature values, and
target domain class prediction values from the target domain feature values and the converted target domain feature values; and
an updating process for updating at least one of (i) the one or more feature extractors, (ii) the one or more angle converters, and (iii) the one or more class predictors with reference to:
a source domain classification loss computed with reference to the source domain class prediction values,
a target domain classification loss computed with reference to the target domain class prediction values,
a conversion loss computed with reference to the source domain feature values, the converted source domain feature values, the target domain feature values and the converted target domain feature values, and
a grouping loss computed with reference to the source domain feature values, the converted source domain feature values, the target domain feature values, and the converted target domain feature values.
2 . The training apparatus according to claim 1 , wherein
the angle conversion process includes:
generating the converted source domain feature values with reference to one or more source domain angle conversion parameters, and
generating the converted target domain feature values with reference to one or more target domain angle conversion parameters.
3 . The training apparatus according to claim 1 , wherein
the updating process comprises:
a grouping process for outputting class groups from the source domain feature values, the converted source domain feature values, the target domain feature values, and the converted target domain feature values, wherein
each of the class groups contains feature values sharing a same class label.
4 . The training apparatus according to claim 3 , wherein
the updating process further comprises:
a grouping loss computation process for computing the grouping loss with reference to the class groups.
5 . The training apparatus according to claim 4 , wherein
the grouping loss computation process computes the grouping loss based on;
intra class metrics determined with reference to the feature values in a same class, and
inter class metrics determined with reference to the feature values in different classes.
6 . The training apparatus according to claim 1 , wherein
the updating process further comprises:
one or more classification loss computation processes for computing:
the source domain classification loss with reference to the source domain class prediction values and source domain class label data, and
the target domain classification loss with reference to the target domain class prediction values and target domain class label data.
7 . The training apparatus according to claim 1 , wherein
the updating process further comprises:
a conversion loss computation process for computing the conversion loss with reference to the source domain feature values, the converted source domain feature values, the target domain feature values, and the converted target domain feature values.
8 . The training apparatus according to claim 7 , wherein
the updating process further comprises:
a merged loss computation process for computing a merged loss with reference to the source domain classification loss, the target domain classification loss, the grouping loss, and the conversion loss, wherein
the updating process updates at least one of (i) the one or more feature extractors, (ii) the one or more angle converters, and (iii) the one or more class predictors with reference to the merged loss.
9 . The training apparatus according to claim 1 , the at least one processor further carrying out:
a domain discrimination process for carrying out, by one or more domain discriminators, a discrimination process to discriminate the target domain from the source domain; and
one or more domain loss computation processes for outputting a domain discrimination loss as a result of the discrimination process, wherein
the updating process further refers to the domain discrimination loss, and
the updating process further updates the one or more domain discriminators.
10 . The training apparatus according to claim 1 , the at least one processor further carrying out:
an angle prediction process for predicting, by one or more angle predictors:
source domain angle prediction values from the source domain feature values and the converted source domain feature values, and
target domain angle prediction values from the target domain feature values and the converted target domain feature values; and
one or more angle prediction loss computation processes for:
outputting a source domain angle prediction loss with reference to the source domain angle prediction values and source domain angle label data, and
outputting a target domain angle prediction loss with reference to the target domain angle prediction values and target domain angle label data, wherein
the updating process further refers to the source domain angle prediction loss and the target domain angle prediction loss, and
the updating process further updates the one or more angle predictors.
11 . The training apparatus according to claim 1 , the at least one processor further carrying out:
an image reconstruction process for generating, by one or more image reconstructors:
source domain reconstructed images from the source domain feature values and the converted source domain feature values, and
target domain reconstructed images from the target domain feature values and the converted target domain feature values; and
one or more reconstruction loss computation processes for:
outputting a source domain reconstruction loss with reference to the source domain reconstructed images and the input source domain image data, and
outputting a target domain reconstruction loss with reference to the target domain reconstructed images and the input target domain image data, wherein
the updating process further refers to the source domain reconstruction loss and the target domain reconstruction loss, and
the updating process further updates the one or more image reconstructors.
12 . The training apparatus according to claim 1 , the at least one processor further carrying out:
a conversion discrimination process for carrying out, by one or more conversion discriminators, a discrimination process to:
discriminate the source domain feature values from the converted source domain feature values, and
discriminate the target domain feature values from the converted target domain feature values; and
one or more conversion discrimination loss computation processes for outputting a source domain conversion discrimination loss and a target domain conversion discrimination loss with reference to a result of the conversion discrimination process, wherein
the updating process further refers to at least one of the source domain conversion discrimination loss and the target domain conversion discrimination loss, and
the updating process further updates the one or more conversion discriminators.
13 . A classification apparatus comprising at least one processor, the at least one processor carrying out:
a feature extraction process for extracting, by feature extractors, feature values from input image data; and
a class prediction process for predicting, by class predictors, class prediction values from the feature values,
wherein at least one of the feature extractors and the class predictors has been trained with reference to:
an extraction process for extracting, by one or more feature extractors, source domain feature values from input source domain image data and for extracting target domain feature values from input target domain image data
an angle conversion process for generating, by one or more angle converters:
converted source domain feature values by converting the source domain feature values as if the converted source domain feature values are extracted from source domain image data having different angles from the input source domain image data, and
converted target domain feature values by converting the target domain feature values as if the converted target domain feature values are extracted from target domain image data having different angles from the input target domain image data;
a second class prediction process for predicting, by one or more class predictors:
source domain class prediction values from the source domain feature values and the converted source domain feature values, and
target domain class prediction values from the target domain feature values and the converted target domain feature values; and
an updating process for updating at least one of (i) the one or more feature extractors, (ii) the one or more angle converters, and (iii) the one or more class predictors with reference to:
a source domain classification loss computed with reference to the source domain class prediction values,
a target domain classification loss computed with reference to the target domain class prediction values,
a conversion loss computed with reference to the source domain feature values, the converted source domain feature values, the target domain feature values and the converted target domain feature values, and
a grouping loss computed with reference to the source domain feature values, the converted source domain feature values, the target domain feature values, and the converted target domain feature values.
14 . A training method comprising:
extracting, by one or more feature extractors:
source domain feature values from input source domain image data, and
target domain feature values from input target domain image data;
generating, by one or more angle converters,
converted source domain feature values by converting the source domain feature values as if the converted source domain feature values are extracted from source domain image data having different angles from the input source domain image data, and
converted target domain feature values by converting the target domain feature values as if the converted target domain feature values are extracted from target domain image data having different angles from the input target domain image data;
predicting, by one or more class predictors:
source domain class prediction values from the source domain feature values and the converted source domain feature values, and
target domain class prediction values from the target domain feature values, and the converted target domain feature values; and
updating at least one of (i) the one or more feature extractors, (ii) the one or more angle converters, and (iii) the one or more class predictors with reference to
a source domain classification loss computed with reference to the source domain class prediction values,
a target domain classification loss computed with reference to the target domain class prediction values,
a conversion loss computed with reference to the source domain feature values, the converted source domain feature values, the target domain feature values and the converted target domain feature values, and
a grouping loss computed with reference to the source domain feature values, the converted source domain feature values, the target domain feature values, and the converted target domain feature values.
15 . A non-transitory storage medium storing therein a program for causing a computer to function as a training apparatus according to claim 1 , the program causing the computer to carrying out each of the extraction process, the class prediction process, the angle conversion process, and the updating process.
16 . A non-transitory storage medium storing therein a program for causing a computer to function as a classification apparatus according to claim 13 , the program causing the computer to carry out each of the feature extraction process, and the class prediction process.