Neural network training method, image classification system, and related device
A neural network training method, an image classification system, and a related device, which may be applied to the artificial intelligence field. Feature extraction is performed on images in a training set (including a first set and a second set) by using a prototype network, to obtain first feature points, in a feature space, of a plurality of images in the first set and second feature points of a plurality of images in the second set. The first feature points are used for calculating a prototype of a class of an image, and the second feature points are used for updating a network parameter of the prototype network. A semantic similarity between classes of the images in the second set is obtained, to calculate a margin value between the classes of the images. Then, a loss function is adjusted based on the margin value.
1. A neural network training method, comprising:
obtaining a training set having a first set of a plurality of images and a second set of a plurality of images, wherein classes of the plurality of images in the first set correspond to classes of the plurality of images in the second set;
performing feature extraction on the plurality of images in the first set and the plurality of images in the second set using a prototype network, to obtain a first feature point of the plurality of images in the first set and a second feature point of the plurality of images in the second set in a feature space, wherein
the first feature point is used for calculating a prototype corresponding to a class of an image for indicating a class center of a position of the first feature point;
obtaining a semantic similarity between the classes of the plurality of images in the second set, and obtaining a margin value between the classes of the plurality of images in the second set based on the semantic similarity;
adjusting an objective loss function based on the margin value; and
training the prototype network based on the first feature point and the second feature point using the adjusted objective loss function.
2. The method according to claim 1 , wherein obtaining the semantic similarity between the classes of the plurality of images in the second set comprises:
extracting a plurality of semantic vectors corresponding to the classes of the plurality of images in the second set respectively using a semantic feature extraction module;
calculating a Euclidean distance between a first semantic vector and a second semantic vector in the plurality of semantic vectors; and
determining a semantic similarity between a class of a first image corresponding to the first semantic vector and a class of a second image corresponding to the second semantic vector in the classes of the plurality of images based on the Euclidean distance.
3. The method according to claim 1 , wherein obtaining the semantic similarity between the classes of the plurality of images in the second set comprises:
extracting a plurality of semantic vectors corresponding to the classes of the plurality of images in the second set respectively using a semantic feature extraction module;
performing an inner product operation between a first semantic vector and a second semantic vector in the plurality of semantic vectors, to obtain an inner product operation result; and
determining a semantic similarity between a class of a first image corresponding to the first semantic vector and a class of a second image corresponding to the second semantic vector in the classes of the plurality of images based on the inner product operation result.
4. The method according to claim 1 , wherein obtaining the margin value between the classes of the plurality of images in the second set based on the semantic similarity comprises:
performing a linear operation on the semantic similarity using an objective operation formula, to obtain a margin value between every two of the classes of the images in the second set.
5. The method according to claim 4 , wherein training the prototype network using the adjusted objective loss function comprises:
adjusting a value of at least one parameter variable of the objective operation formula.
6. The method according to claim 1 , wherein obtaining the margin value between the classes of the plurality of images in the second set based on the semantic similarity comprises:
using the semantic similarity as an input of a preset neural network to obtain an output result, wherein the output result comprises a margin value between every two of the classes of the images in the second set.
7. The method according to claim 6 , further comprising:
performing joint training on the prototype network and the preset neural network using the adjusted objective loss function.
8. The method according to claim 1 , wherein after training the prototype network based on the first feature point and the second feature point, the method further comprises:
outputting the trained prototype network.
9. The method according to claim 8 , further comprising:
performing feature extraction on a target image by using the trained prototype network, to obtain a target feature point in the feature space; and
outputting a predicted class of the target image corresponding to a prototype closest to the target feature point.
10. A training device, comprising:
a processor and a memory
to store a program instructions, which
when executed by the processor, cause the training device to perform operations, the operations comprising:
obtaining a training set having a first set of a plurality of images and a second set of a plurality of images, wherein classes of the plurality of images in the first set correspond to classes of the plurality of images in the second set;
performing feature extraction on the plurality of images in the first set and the plurality of images in the second set using a prototype network, to obtain a first feature point, of the plurality of images in the first set and a second feature point of the plurality of images in the second set in a feature space, wherein
the first feature point is used for calculating a prototype corresponding to a class of an image, and the prototype is used for indicating a class center of a position, in the feature space, of the first feature point;
obtaining a semantic similarity between the classes of the plurality of images in the second set, and obtaining a margin value between the classes of the plurality of images in the second set based on the semantic similarity;
adjusting an objective loss function based on the margin value; and
training the prototype network based on the first feature point and the second feature point using the adjusted objective loss function.
11. The training device according to claim 10 , wherein obtaining the semantic similarity between the classes of the plurality of images in the second set comprises:
extracting a plurality of semantic vectors corresponding to the classes of the plurality of images in the second set respectively using a semantic feature extraction module;
calculating a Euclidean distance between a first semantic vector and a second semantic vector in the plurality of semantic vectors; and
determining a semantic similarity between a class of a first image corresponding to the first semantic vector and a class of a second image corresponding to the second semantic vector in the classes of the plurality of images based on the Euclidean distance.
12. The training device according to claim 10 , wherein the obtaining the semantic similarity between the classes of the plurality of images in the second set comprises:
extracting a plurality of semantic vectors corresponding to the classes of the plurality of images in the second set respectively using a semantic feature extraction module;
performing an inner product operation between a first semantic vector and a second semantic vector in the plurality of semantic vectors, to obtain an inner product operation result; and
determining a semantic similarity between a class of a first image corresponding to the first semantic vector and a class of a second image corresponding to the second semantic vector in the classes of the plurality of images based on the inner product operation result.
13. The training device according to claim 10 , wherein obtaining the margin value between the classes of the plurality of images in the second set based on the semantic similarity comprises:
performing a linear operation on the semantic similarity using an objective operation formula, to obtain a margin value between every two of the classes of the images in the second set.
14. The training device according to claim 13 , wherein training the prototype network using the adjusted objective loss function comprises:
adjusting a value of at least one parameter variable of the objective operation formula.
15. The training device according to claim 10 , wherein obtaining the margin value between the classes of the plurality of images in the second set based on the semantic similarity comprises:
using the semantic similarity as an input of a preset neural network to obtain an output result, wherein the output result comprises a margin value between every two of the classes of the images in the second set.
16. The training device according to claim 15 , wherein training the prototype network using an adjusted objective loss function comprises:
performing joint training on the prototype network and the preset neural network using the adjusted objective loss function.
17. The training device according to claim 10 , wherein after training the prototype network based on the first feature point and the second feature point, the operations further comprise:
outputting the trained prototype network.
18. The training device according to claim 17 , wherein the operations further comprise:
performing feature extraction on a target image using the trained prototype network, to obtain a target feature point in the feature space; and
outputting a predicted class of the target image corresponding to a prototype closest to the target feature point.
19. A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a computer to perform operations, the operations comprising:
obtaining a training set having a first set of a plurality of images and a second set of a plurality of images, wherein classes of the plurality of images in the first set correspond to classes of the plurality of images in the second set;
performing feature extraction on the plurality of images in the first set and the plurality of images in the second set using a prototype network, to obtain a first feature point, of the plurality of images in the first set and a second feature point of the plurality of images in the second set in a feature space, wherein
the first feature point is used for calculating a prototype corresponding to a class of an image, and the prototype is used for indicating a class center of a position, in the feature space, of the first feature point;
obtaining a semantic similarity between the classes of the plurality of images in the second set, and obtaining a margin value between the classes of the plurality of images in the second set based on the semantic similarity;
adjusting an objective loss function based on the margin value; and
training the prototype network based on the first feature point and the second feature point and using the adjusted objective loss function.
20. The non-transitory computer-readable storage medium according to claim 19 , wherein the obtaining the semantic similarity between the classes of the plurality of images in the second set comprises:
extracting a plurality of semantic vectors corresponding to the classes of the plurality of images in the second set respectively using a semantic feature extraction module;
calculating a Euclidean distance between a first semantic vector and a second semantic vector in the plurality of semantic vectors; and
determining a semantic similarity between a class of a first image corresponding to the first semantic vector and a class of a second image corresponding to the second semantic vector in the classes of the plurality of images based on the Euclidean distance.