Method, electronic device, and computer program product for generating image samples
Embodiments of the present disclosure provide a method, an electronic device, and a computer program product for generating image samples. The method includes processing a group of image samples using a class capsule network model, and obtaining a group of distilled samples, a feature distribution of the group of distilled samples indicating a feature distribution of the group of image samples; and determining a soft label of each distilled sample in the group of distilled samples on the basis of a label of the group of image samples, the soft label representing a probability that the distilled sample belongs to each of a plurality of classes. As such, class capsules in a trained capsule network model are extracted as distilled samples, and soft labels are assigned to the distilled samples on the basis of original image samples, so that the extracted distilled samples can be ensured to have high explainability.
1 . A method for generating image samples, comprising:
processing a group of image samples using a class capsule network model, trained with a loss function comprising L=Lor(Z)−I(X,Z), where Lor ensures minimal common features between different classes of distilled samples and I(X,Z) ensures meta-information preservation, and obtaining a group of distilled samples by extracting class capsules from the class capsule network model and visualizing the class capsules as images, a feature distribution of the group of distilled samples indicating a feature distribution of the group of image samples; and
determining a soft label of each distilled sample in the group of distilled samples on the basis of a label of the group of image samples, the soft label representing a probability that the distilled sample belongs to each of a plurality of classes.
2 . The method according to claim 1 , wherein obtaining a group of distilled samples comprises:
on the basis of the feature distribution of a plurality of features of the group of image samples and the feature distribution of the group of distilled samples, processing the group of image samples using the class capsule network model trained with the loss function L=Lor(Z)−I(X,Z) where I(X,Z)=∫∫p(z|x)p(x)log[p(z|x)/p(z)]dxdz, and obtaining the group of distilled samples by extracting and visualizing class capsules.
3 . The method according to claim 1 , wherein obtaining a group of distilled samples comprises:
on the basis of a similarity between distilled samples of the group of distilled samples, processing the group of image samples using the class capsule network model trained with a loss function comprising Lor=−Σφ(i,j) for i,j∈Z, so that the number of identical features comprised by different classes of distilled samples is less than a predetermined threshold, and extracting and visualizing class capsules to obtain the distilled samples.
4 . The method according to claim 1 , wherein determining a soft label of each distilled sample in the group of distilled samples comprises:
determining a relevancy between each distilled sample in the group of distilled samples and each image sample in the group of image samples using an N×d relevancy matrix, where N is the number of image samples and d is the number of distilled samples;
determining, on the basis of a relevancy between each distilled sample and an image sample in each class, a probability that each distilled sample belongs to the each class; and
determining the soft label for each distilled sample on the basis of the probability that each distilled sample belongs to the each class.
5 . The method according to claim 4 , wherein determining the relevancy comprises:
acquiring a distilled feature of each distilled sample in the group of distilled samples, wherein the distilled feature is obtained when the group of image samples is processed using the class capsule network model and the class capsules are extracted;
processing the group of image samples using a classification model separate from the class capsule network model, and acquiring an original feature of each image sample in the group of image samples; and
determining the relevancy on the basis of the distilled feature and the original feature by calculating the N×d relevancy matrix.
6 . The method according to claim 4 , wherein determining a probability that each distilled sample belongs to the each class comprises:
determining a weighted sum of similarities between each distilled sample and image samples in each class as the probability that each distilled sample belongs to the each class using the N×d relevancy matrix.
7 . The method according to claim 1 , further comprising:
training a machine learning model using the distilled samples obtained by extracting and visualizing class capsules and the soft labels.
8 . An electronic device, comprising:
at least one processing unit; and
at least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform acts comprising:
processing a group of image samples using a class capsule network model, trained with a loss function comprising L=Lor(Z)−I(X,Z), where Lor ensures minimal common features between different classes of distilled samples and I(X,Z) ensures meta-information preservation, and obtaining a group of distilled samples by extracting class capsules from the class capsule network model and visualizing the class capsules as images, a feature distribution of the group of distilled samples indicating a feature distribution of the group of image samples; and
determining a soft label of each distilled sample in the group of distilled samples on the basis of a label of the group of image samples, the soft label representing a probability that the distilled sample belongs to each of a plurality of classes.
9 . The electronic device according to claim 8 , wherein obtaining a group of distilled samples comprises:
on the basis of the feature distribution of a plurality of features of the group of image samples and the feature distribution of the group of distilled samples, processing the group of image samples using the class capsule network model trained with the loss function L=Lor(Z)−I(X,Z) where I(X,Z)=∫∫p(z|x)p(x)log[p(z|x)/p(z)]dxdz, and obtaining the group of distilled samples by extracting and visualizing class capsules.
10 . The electronic device according to claim 8 , wherein obtaining a group of distilled samples comprises:
on the basis of a similarity between distilled samples of the group of distilled samples, processing the group of image samples using the class capsule network model trained with a loss function comprising Lor=−Σφ(i,j) for i,j∈Z, so that the number of identical features comprised by different classes of distilled samples is less than a predetermined threshold, and extracting and visualizing class capsules to obtain the distilled samples.
11 . The electronic device according to claim 8 , wherein determining a soft label of each distilled sample in the group of distilled samples comprises:
determining a relevancy between each distilled sample in the group of distilled samples and each image sample in the group of image samples using an N×d relevancy matrix, where N is the number of image samples and d is the number of distilled samples;
determining, on the basis of a relevancy between each distilled sample and an image sample in each class, a probability that each distilled sample belongs to the each class; and
determining the soft label for each distilled sample on the basis of the probability that each distilled sample belongs to the each class.
12 . The electronic device according to claim 11 , wherein determining the relevancy comprises:
acquiring a distilled feature of each distilled sample in the group of distilled samples, wherein the distilled feature is obtained when the group of image samples is processed using the class capsule network model and the class capsules are extracted;
processing the group of image samples using a classification model separate from the class capsule network model, and acquiring an original feature of each image sample in the group of image samples; and
determining the relevancy on the basis of the distilled feature and the original feature by calculating the N×d relevancy matrix.
13 . The electronic device according to claim 11 , wherein determining a probability that each distilled sample belongs to the each class comprises:
determining a weighted sum of similarities between each distilled sample and image samples in each class as the probability that each distilled sample belongs to the each class using the N×d relevancy matrix.
14 . The electronic device according to claim 8 , wherein the acts further comprise:
training a machine learning model using the distilled samples obtained by extracting and visualizing class capsules and the soft labels.
15 . A computer program product, tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions, the machine-executable instructions, when executed, causing a machine to perform acts comprising:
processing a group of image samples using a class capsule network model, trained with a loss function comprising L=Lor(Z)−I(X,Z), where Lor ensures minimal common features between different classes of distilled samples and I(X,Z) ensures meta-information preservation, and obtaining a group of distilled samples by extracting class capsules from the class capsule network model and visualizing the class capsules as images, a feature distribution of the group of distilled samples indicating a feature distribution of the group of image samples; and
determining a soft label of each distilled sample in the group of distilled samples on the basis of a label of the group of image samples, the soft label representing a probability that the distilled sample belongs to each of a plurality of classes.
16 . The computer program product according to claim 15 , wherein obtaining a group of distilled samples comprises:
on the basis of the feature distribution of a plurality of features of the group of image samples and the feature distribution of the group of distilled samples, processing the group of image samples using the class capsule network model trained with the loss function L=Lor(Z)−I(X,Z) where I(X,Z)=∫∫p(z|x)p(x)log[p(z|x)/p(z)]dxdz, and obtaining the group of distilled samples by extracting and visualizing class capsules.
17 . The computer program product according to claim 15 , wherein obtaining a group of distilled samples comprises:
on the basis of a similarity between distilled samples of the group of distilled samples, processing the group of image samples using the class capsule network model trained with a loss function comprising Lor=−Σφ(i,j) for i,j∈Z, so that the number of identical features comprised by different classes of distilled samples is less than a predetermined threshold, and extracting and visualizing class capsules to obtain the distilled samples.
18 . The computer program product according to claim 15 , wherein determining a soft label of each distilled sample in the group of distilled samples comprises:
determining a relevancy between each distilled sample in the group of distilled samples and each image sample in the group of image samples using an N×d relevancy matrix, where N is the number of image samples and d is the number of distilled samples;
determining, on the basis of a relevancy between each distilled sample and an image sample in each class, a probability that each distilled sample belongs to the each class; and
determining the soft label for each distilled sample on the basis of the probability that each distilled sample belongs to the each class.
19 . The computer program product according to claim 18 , wherein determining the relevancy comprises:
acquiring a distilled feature of each distilled sample in the group of distilled samples, wherein the distilled feature is obtained when the group of image samples is processed using the class capsule network model and the class capsules are extracted;
processing the group of image samples using a classification model separate from the class capsule network model, and acquiring an original feature of each image sample in the group of image samples; and
determining the relevancy on the basis of the distilled feature and the original feature by calculating the N×d relevancy matrix.
20 . The computer program product according to claim 18 , wherein determining a probability that each distilled sample belongs to the each class comprises:
determining a weighted sum of similarities between each distilled sample and image samples in each class as the probability that each distilled sample belongs to the each class using the N×d relevancy matrix.