Neural network construction method and apparatus
A neural network construction method and apparatus in the field of artificial intelligence, to accurately and efficiently construct a target neural network. The constructed target neural network has high output accuracy, may be further applied to different application scenarios, and has a strong generalization capability. The method includes: obtaining a start point network, where the start point network includes a plurality of serial subnets; performing at least one time of transformation on the start point network based on a preset first search space to obtain a serial network, where the first search space includes a range of parameters used for transforming the start point network; and if the serial network meets a preset condition, training the serial network by using a preset dataset to obtain a trained serial network; and if the trained serial network meets a termination condition, obtaining a target neural network based on the trained serial network.
1 . A neural network construction method, comprising:
obtaining a start point network comprising a plurality of serial subnets;
performing at least one time of transformation on the start point network based on a first search space to obtain a serial network, wherein the first search space comprises a range of parameters used for transforming the start point network, wherein types of parameters included in the first search space include at least one of width, depth, a quantity of times of downsampling, or a location of downsampling;
if the serial network meets a preset condition, training the serial network by using a preset dataset to obtain a trained serial network; and
if the trained serial network meets a termination condition, obtaining a target neural network based on the trained serial network, wherein the obtaining the target neural network based on the trained serial network comprises:
if the trained serial network meets an output condition, using the trained serial network as the target neural network; or
if the trained serial network does not meet the output condition, obtaining a parallel subnet corresponding to the trained serial network, and constructing the target neural network based on the trained serial network and the parallel subnet corresponding to the trained serial network.
2 . The method according to claim 1 , wherein one of the at least one time of transformation performed on the start point network based on the first search space comprises:
searching the first search space for a transformation policy comprising a group of parameters used for transforming the start point network;
transforming the start point network according to the transformation policy to obtain a transformed start point network, wherein a weight of the transformed start point network inherits a weight of the start point network before transformation;
updating the weight of the transformed start point network by using a lightweight dataset, to obtain a start point network with an updated weight, wherein data comprised in the preset dataset is more than data comprised in the lightweight dataset;
evaluating the start point network with the updated weight to obtain a first evaluation result; and
if the first evaluation result meets a preset update condition, re-searching the first search space for a transformation policy, and re-transforming the start point network according to a re-found transformation policy; or
if the first evaluation result does not meet the preset update condition, using the start point network with the updated weight as the serial network.
3 . The method according to claim 2 , further comprising:
if the serial network does not meet the preset condition, using the serial network as a new start point network; or if the trained serial network does not meet the termination condition, using the trained serial network as a new start point network; and
updating the first search space, searching for a transformation policy based on an updated first search space, and transforming the new start point network according to the found transformation policy.
4 . The method according to claim 2 , wherein the preset update condition comprises:
a quantity of times of transformation performed on the start point network does not exceed a preset quantity of times, or the first evaluation result does not meet a preset convergence condition comprising one or more of: the first evaluation result better than a threshold, or a difference between the first evaluation result and an evaluation result obtained in previous evaluation of the start point network less than a preset value.
5 . The method according to claim 2 , wherein the preset condition comprises:
a second evaluation result of the serial network meets a first preset evaluation condition, a quantity of times of transformation performed on the serial network exceeds a preset quantity of times of transformation, or duration from last time of transformation performed on the start point network exceeds a preset value, wherein the second evaluation result is obtained by evaluating the serial network; and
the first preset evaluation condition comprises one or more of: a loss value of the serial network not greater than a first loss threshold, a precision value of an output result of the serial network greater than a first precision threshold, or inference duration of the serial network less than a first inference threshold.
6 . The method according to claim 1 , wherein each serial subnet in the start point network comprises one or more blocks, and the transforming the start point network comprises:
swapping blocks comprised in two adjacent serial subnets in the plurality of serial subnets; or
inserting a processed block into at least one serial subnet in the plurality of serial subnets, wherein the processed block is obtained by initializing a weight of a block as an identity matrix and/or performing group normalization on the block.
7 . The method according to claim 1 , wherein the termination condition comprises:
a second evaluation result of the serial network meets a second preset evaluation condition, or a quantity of times of training of the serial network exceeds a preset quantity of times of training.
8 . The method according to claim 1 , wherein the target neural network is used to perform at least one of image recognition, semantic segmentation, or object detection, and wherein the preset dataset comprises an ImageNet dataset.
9 . A neural network construction apparatus, comprising:
a processor,
a memory coupled to the processor to store instructions, which when executed by the processor, cause the neural network construction apparatus to perform operations, the operations comprising:
obtaining a start point network comprising a plurality of serial subnets;
performing at least one time of transformation on the start point network based on a first search space to obtain a serial network, wherein the first search space comprises a range of parameters used for transforming the start point network, wherein types of parameters included in the first search space include at least one of width, depth, a quantity of times of downsampling, or
a location of downsampling;
if the serial network meets a preset condition, training the serial network by using a preset dataset to obtain a trained serial network; and
if the trained serial network meets a termination condition, obtaining a target neural network based on the trained serial network, wherein the obtaining the target neural network based on the trained serial network comprises:
if the trained serial network meets an output condition, using the trained serial network as the target neural network; or
if the trained serial network does not meet the output condition, obtaining a parallel subnet corresponding to the trained serial network, and constructing the target neural network based on the trained serial network and the parallel subnet corresponding to the trained serial network.
10 . The neural network construction apparatus according to claim 9 , wherein one of the at least one time of transformation performed on the start point network based on the first search space comprises:
searching the first search space for a transformation policy comprising a group of parameters used for transforming the start point network;
transforming the start point network according to the transformation policy to obtain a transformed start point network, wherein a weight of the transformed start point network inherits a weight of the start point network before transformation;
updating the weight of the transformed start point network by using a lightweight dataset, to obtain a start point network with an updated weight, wherein data comprised in the preset dataset is more than data comprised in the lightweight dataset;
evaluating the start point network with the updated weight to obtain a first evaluation result; and
if the first evaluation result meets a preset update condition, re-searching the first search space for a transformation policy, and re-transforming the start point network according to a re-found transformation policy; or
if the first evaluation result does not meet the preset update condition, using the start point network with the updated weight as the serial network.
11 . The neural network construction apparatus according to claim 10 , wherein the operations further comprises:
if the serial network does not meet the preset condition, using the serial network as a new start point network; or if the trained serial network does not meet the termination condition, using the trained serial network as a new start point network; and
updating the first search space, searching for a transformation policy based on an updated first search space, and transforming the new start point network according to the found transformation policy.
12 . The neural network construction apparatus according to claim 10 , wherein the preset update condition comprises: a quantity of times of transformation performed on the start point network does not exceed a preset quantity of times, or the first evaluation result does not meet a preset convergence condition comprising one or more of: the first evaluation result better than a threshold, or a difference between the first evaluation result and an evaluation result obtained in previous evaluation of the start point network less than a preset value.
13 . The neural network construction apparatus according to claim 10 , wherein the preset condition comprises: a second evaluation result of the serial network meets a first preset evaluation condition, a quantity of times of transformation performed on the serial network exceeds a preset quantity of times of transformation, or duration from last time of transformation performed on the start point network exceeds a preset value, wherein the second evaluation result is obtained by evaluating the serial network; and
the first preset evaluation condition comprises one or more of: a loss value of the serial network not greater than a first loss threshold, a precision value of an output result of the serial network greater than a first precision threshold, or inference duration of the serial network less than a first inference threshold.
14 . The neural network construction apparatus according to claim 9 , wherein each serial subnet in the start point network comprises one or more blocks, and the transforming the start point network comprises:
swapping blocks comprised in two adjacent serial subnets in the plurality of serial subnets; or
inserting a processed block into at least one serial subnet in the plurality of serial subnets, wherein the processed block is obtained by initializing a weight of a block as an identity matrix and/or performing group normalization on the block.
15 . The neural network construction apparatus according to claim 9 , wherein the termination condition comprises:
a second evaluation result of the serial network meets a second preset evaluation condition, or a quantity of times of training of the serial network exceeds a preset quantity of times of training.
16 . The neural network construction apparatus according to claim 9 , wherein the target neural network is used to perform at least one of image recognition, semantic segmentation, or object detection, and wherein the preset dataset comprises an ImageNet dataset.
17 . A non-transitory machine-readable storage medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
obtaining a start point network comprising a plurality of serial subnets;
performing at least one time of transformation on the start point network based on a first search space to obtain a serial network, wherein the first search space comprises a range of parameters used for transforming the start point network, wherein types of parameters included in the first search space include at least one or more of width, depth, a quantity of times of downsampling, or a location of downsampling;
if the serial network meets a preset condition, training the serial network by using a preset dataset to obtain a trained serial network; and
if the trained serial network meets a termination condition, obtaining a target neural network based on the trained serial network, wherein the obtaining the target neural network based on the trained serial network comprises:
if the trained serial network meets an output condition, using the trained serial network as the target neural network; or
if the trained serial network does not meet the output condition, obtaining a parallel subnet corresponding to the trained serial network, and constructing the target neural network based on the trained serial network and the parallel subnet corresponding to the trained serial network.
18 . The non-transitory machine-readable storage medium according to claim 17 , wherein one of the at least one time of transformation performed on the start point network based on the first search space comprises:
searching the first search space for a transformation policy comprising a group of parameters used for transforming the start point network;
transforming the start point network according to the transformation policy to obtain a transformed start point network, wherein a weight of the transformed start point network inherits a weight of the start point network before transformation;
updating the weight of the transformed start point network by using a lightweight dataset, to obtain a start point network with an updated weight, wherein data comprised in the preset dataset is more than data comprised in the lightweight dataset;
evaluating the start point network with the updated weight to obtain a first evaluation result; and
if the first evaluation result meets a preset update condition, re-searching the first search space for a transformation policy, and re-transforming the start point network according to a re-found transformation policy; or
if the first evaluation result does not meet the preset update condition, using the start point network with the updated weight as the serial network.
19 . The non-transitory machine-readable storage medium according to claim 18 , wherein the operations further comprises:
if the serial network does not meet the preset condition, using the serial network as a new start point network; or if the trained serial network does not meet the termination condition, using the trained serial network as a new start point network; and
updating the first search space, searching for a transformation policy based on an updated first search space, and transforming the new start point network according to the found transformation policy.
20 . The non-transitory machine-readable storage medium according to claim 18 , wherein the preset update condition comprises:
a quantity of times of transformation performed on the start point network does not exceed a preset quantity of times, or the first evaluation result does not meet a preset convergence condition comprising one or more of: the first evaluation result better than a threshold, or a difference between the first evaluation result and an evaluation result obtained in previous evaluation of the start point network less than a preset value.