Granted Patent B2

US 12,737,583 · App. 17/507,687 · Granted Sep 15, 2026

Inventors: Yin Zheng (Shenzhen, CN); Yong Guo (Shenzhen, CN); Bingzheng Wei (Shenzhen, CN); Hanyu Huang (Shenzhen, CN); Yan Huang (Shenzhen, CN); Su Wang (Shenzhen, CN); Yanxiong Lu (Shenzhen, CN); Leyu Lin (Shenzhen, CN)

Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

G06N3/04

Abstract

Claims (98)

1 . A method for training a network structure optimizer using an image recognition neural network performed by an electronic device, the method comprising:

performing feature extraction on an initial network structure of the image recognition neural network by using the network structure optimizer, to obtain feature information corresponding to the initial network structure of the image recognition neural network, further including:

establishing a directed graph for the initial network structure of the image recognition neural network by using the network structure optimizer, wherein the directed graph is represented by using an ordered triplet (V (D), A (D), wD), wherein D represents the directed graph and wD defines a correlation relationship between an ordered pair of nodes (V (D), A (D)) in the directed graph; and

performing feature extraction on the directed graph, to obtain the feature information corresponding to the initial network structure of the image recognition neural network;

predicting the feature information by using the network structure optimizer, to det ermine a plurality of optimization manners for optimizing the initial network structure of the image recognition neural network;

updating the initial network structure of the image recognition neural network according to the optimization manners for the network structure, to obtain a plurality of optimized network structures of the image recognition neural network;

determining an image recognition performance parameter of each of the plurality of optimized network structures of the image recognition neural network, further comprising:

for each of the plurality of optimized network structures of the image recognition neural network, determining an accuracy rate of the optimized network structure relative to the initial network structure of the image recognition neural network by performing image recognition of the same image using the optimized network structure and the initial network structure of the image recognition neural network, respectively;

using the accuracy rate as the image recognition performance parameter of the optimized network structure; and

constructing a loss function of the network structure optimizer based on the image recognition performance parameters and calculation costs of the plurality of optimized network structures of the image recognition neural network;

updating a parameter of the network structure optimizer according to the image recognition performance parameters and the calculation costs of the plurality of optimized network structures of the image recognition neural network by backpropagating a value of the loss function in the network structure optimizer until the value of the loss function is less than a preset threshold, the updated network structure optimizer being configured to optimize the network structure of the image recognition neural network to have a maximized accuracy rate and a calculation cost under a predefined limit.

2 . The method according to claim 1 , wherein the establishing a directed graph for the network structure of the image recognition neural network by using the network structure optimizer comprises:

performing sampling processing on the network structure of the image recognition neural network by using the network structure optimizer, to obtain network layer information for the network structure of the image recognition neural network; and

establishing the directed graph for the network structure of the image recognition neural network according to the network layer information.

3 . The method according to claim 2 , wherein the establishing the directed graph for the network structure of the image recognition neural network according to the network layer information comprises:

determining an input feature or an output feature in the network layer information as node information in the directed graph;

determining operation information in the network layer information as edge information in the directed graph, the operation information comprising at least one of the following: a convolution operation, a skip connection operation, a pooling operation, and a null connection operation; and

establishing the directed graph for the network structure according to the node information and the edge information.

4 . The method according to claim 1 , wherein the performing feature extraction on the directed graph, to obtain the feature information corresponding to the network structure comprises:

determining node information and edge information in the directed graph; and

determining the feature information corresponding to the network structure according to learnable parameter matrixes and an activation function in a graph convolutional network (GCN), the node information, and the edge information.

5 . The method according to claim 4 , wherein the determining the feature information corresponding to the network structure according to learnable parameter matrixes and an activation function in a GCN, the node information, and the edge information comprises:

performing a multiplication operation on the edge information, the node information, and a learnable parameter matrix in a first-layer GCN, to obtain an output of the first-layer GCN;

performing nonlinear transformation on the output of the first-layer GCN by using the activation function in the GCN, to obtain a transformed output of the first-layer GCN;

performing a multiplication operation on the transformed output of the first-layer GCN and a learnable parameter matrix in a second-layer GCN, to obtain an output of the second-layer GCN; and

performing a multiplication operation on the output of the second-layer GCN, the edge information, and a learnable parameter matrix of a fully-connected layer in the GCN, to obtain the feature information corresponding to the network structure.

6 . The method according to claim 1 , wherein the predicting the feature information by using the network structure optimizer, to determine a plurality of optimization manners for the network structure comprises:

classifying the feature information by using the network structure optimizer, and determining probabilities corresponding to the optimization manners of the network structure; and

determining, according to the probabilities of the optimization manners of the network structure, optimization manners for the network structure that meet a probability requirement in the plurality of optimization manners.

7 . The method according to claim 1 , wherein the updating the network structure of the image recognition neural network according to the optimization manners for the network structure, to obtain an optimized network structure of the image recognition neural network comprises:

determining operation information, sizes of network layers, and a quantity of network layers in the optimization manners; and

replacing operation information, sizes of network layers, and a quantity of network layers in the network structure of the image recognition neural network with the operation information, the sizes of the network layers, and the quantity of network layers in the optimization manners, to obtain the optimized network structure,

the sizes of the network layers comprising at least one of the following: a size of a convolution kernel and a size of a pooling layer.

8 . The method according to claim 1 , the method further comprising:

performing feature extraction on the network structure of the image recognition neural network by using the updated network structure optimizer, to obtain feature information corresponding to the network structure;

predicting the feature information by using the updated network structure optimizer, to determine optimization manners for the network structure; and

updating the network structure of the image recognition neural network according to the optimization manners for the network structure, to obtain an optimized network structure, the optimized network structure being used for performing an image recognition task.

9 . The method according to claim 8 , wherein the updating the network structure of the image recognition neural network according to the optimization manners for the network structure, to obtain an optimized network structure comprises:

determining operation information, sizes of network layers, and a quantity of network layers in the optimization manners; and

replacing operation information, sizes of network layers, and a quantity of network layers in the network structure of the image recognition neural network with the operation information, the sizes of the network layers, and the quantity of network layers in the optimization manners, to obtain the optimized network structure,

the operation information comprising at least one of the following: a convolution operation, a skip connection operation, a pooling operation, and a null connection operation; and

the sizes of the network layers comprising at least one of the following: a size of a convolution kernel and a size of a pooling layer.

10 . An electronic device, comprising:

a memory, configured to store executable instructions; and

a processor, configured to perform, when executing the executable instructions stored in the memory, a plurality of operations for training a network structure optimizer using an image recognition neural network including:

performing feature extraction on an initial network structure of the image recognition neural network by using the network structure optimizer, to obtain feature information corresponding to the initial network structure of the image recognition neural network, further including:

establishing a directed graph for the initial network structure of the image recognition neural network by using the network structure optimizer, wherein the directed graph is represented by using an ordered triplet (V (D), A (D), wD), wherein D represents the directed graph and wD defines a correlation relationship between an ordered pair of nodes (V (D), A (D)) in the directed graph; and

performing feature extraction on the directed graph, to obtain the feature information corresponding to the initial network structure of the image recognition neural network;

predicting the feature information by using the network structure optimizer, to determine a plurality of optimization manners for optimizing the initial network structure of the image recognition neural network;

updating the initial network structure of the image recognition neural network according to the optimization manners for the network structure, to obtain a plurality of optimized network structures of the image recognition neural network;

determining an image recognition performance parameter of each of the plurality of optimized network structures of the image recognition neural network, further comprising:

for each of the plurality of optimized network structures of the image recognition neural network, determining an accuracy rate of the optimized network structure relative to the initial network structure of the image recognition neural network by performing image recognition of the same image using the optimized network structure and the initial network structure of the image recognition neural network, respectively;

using the accuracy rate as the image recognition performance parameter of the optimized network structure; and

constructing a loss function of the network structure optimizer based on the image recognition performance parameters and calculation costs of the plurality of optimized network structures of the image recognition neural network;

updating a parameter of the network structure optimizer according to the image recognition performance parameters and the calculation costs of the plurality of optimized network structures of the image recognition neural network by backpropagating a value of the loss function in the network structure optimizer until the value of the loss function is less than a preset threshold, the updated network structure optimizer being configured to optimize the network structure of the image recognition neural network to have a maximized accuracy rate and a calculation cost under a predefined limit.

11 . The electronic device according to claim 10 , wherein the establishing a directed graph for the network structure of the image recognition neural network by using the network structure optimizer comprises:

performing sampling processing on the network structure of the image recognition neural network by using the network structure optimizer, to obtain network layer information for the network structure of the image recognition neural network; and

establishing the directed graph for the network structure of the image recognition neural network according to the network layer information.

12 . The electronic device according to claim 11 , wherein the establishing the directed graph for the network structure of the image recognition neural network according to the network layer information comprises:

determining an input feature or an output feature in the network layer information as node information in the directed graph;

determining operation information in the network layer information as edge information in the directed graph, the operation information comprising at least one of the following: a convolution operation, a skip connection operation, a pooling operation, and a null connection operation; and

establishing the directed graph for the network structure according to the node information and the edge information.

13 . The electronic device according to claim 10 , wherein the performing feature extraction on the directed graph, to obtain the feature information corresponding to the network structure comprises:

determining node information and edge information in the directed graph; and

determining the feature information corresponding to the network structure according to learnable parameter matrixes and an activation function in a graph convolutional network (GCN), the node information, and the edge information.

14 . The electronic device according to claim 13 , wherein the determining the feature information corresponding to the network structure according to learnable parameter matrixes and an activation function in a GCN, the node information, and the edge information comprises:

performing a multiplication operation on the edge information, the node information, and a learnable parameter matrix in a first-layer GCN, to obtain an output of the first-layer GCN;

performing nonlinear transformation on the output of the first-layer GCN by using the activation function in the GCN, to obtain a transformed output of the first-layer GCN;

performing a multiplication operation on the transformed output of the first-layer GCN and a learnable parameter matrix in a second-layer GCN, to obtain an output of the second-layer GCN; and

performing a multiplication operation on the output of the second-layer GCN, the edge information, and a learnable parameter matrix of a fully-connected layer in the GCN, to obtain the feature information corresponding to the network structure.

15 . The electronic device according to claim 10 , wherein the predicting the feature information by using the network structure optimizer, to determine a plurality of optimization manners for the network structure comprises:

classifying the feature information by using the network structure optimizer, and determining probabilities corresponding to the optimization manners of the network structure; and

determining, according to the probabilities of the optimization manners of the network structure, optimization manners for the network structure that meet a probability requirement in the plurality of optimization manners.

16 . The electronic device according to claim 10 , wherein the updating the network structure of the image recognition neural network according to the optimization manners for the network structure, to obtain an optimized network structure of the image recognition neural network comprises:

determining operation information, sizes of network layers, and a quantity of network layers in the optimization manners; and

replacing operation information, sizes of network layers, and a quantity of network layers in the network structure of the image recognition neural network with the operation information, the sizes of the network layers, and the quantity of network layers in the optimization manners, to obtain the optimized network structure,

the sizes of the network layers comprising at least one of the following: a size of a convolution kernel and a size of a pooling layer.

17 . The electronic device according to claim 10 , wherein the plurality of operations further comprises:

performing feature extraction on the network structure of the image recognition neural network by using the updated network structure optimizer, to obtain feature information corresponding to the network structure;

predicting the feature information by using the updated network structure optimizer, to determine optimization manners for the network structure; and

updating the network structure of the image recognition neural network according to the optimization manners for the network structure, to obtain an optimized network structure, the optimized network structure being used for performing an image recognition task.

18 . A non-transitory computer-readable storage medium, storing executable instructions, when executed by a processor of an electronic device, configured to cause the electronic device to perform a plurality of operations for training a network structure optimizer using an image recognition neural network including:

performing feature extraction on an initial network structure of the image recognition neural network by using the network structure optimizer, to obtain feature information corresponding to the initial network structure of the image recognition neural network, further including:

establishing a directed graph for the initial network structure of the image recognition neural network by using the network structure optimizer, wherein the directed graph is represented by using an ordered triplet (V (D), A (D), wD), wherein D represents the directed graph and wD defines a correlation relationship between an ordered pair of nodes (V (D), A (D)) in the directed graph; and

performing feature extraction on the directed graph, to obtain the feature information corresponding to the initial network structure of the image recognition neural network;

predicting the feature information by using the network structure optimizer, to determine a plurality of optimization manners for optimizing the initial network structure of the image recognition neural network;

updating the initial network structure of the image recognition neural network according to the optimization manners for the network structure, to obtain a plurality of optimized network structures of the image recognition neural network;

determining an image recognition performance parameter of each of the plurality of optimized network structures of the image recognition neural network, further comprising:

for each of the plurality of optimized network structures of the image recognition neural network, determining an accuracy rate of the optimized network structure relative to the initial network structure of the image recognition neural network by performing image recognition of the same image using the optimized network structure and the initial network structure of the image recognition neural network, respectively;

using the accuracy rate as the image recognition performance parameter of the optimized network structure; and

constructing a loss function of the network structure optimizer based on the image recognition performance parameters and calculation costs of the plurality of optimized network structures of the image recognition neural network;

updating a parameter of the network structure optimizer according to the image recognition performance parameters and the calculation costs of the plurality of optimized network structures of the image recognition neural network by backpropagating a value of the loss function in the network structure optimizer until the value of the loss function is less than a preset threshold, the updated network structure optimizer being configured to optimize the network structure of the image recognition neural network to have a maximized accuracy rate and a calculation cost under a predefined limit.

19 . The non-transitory computer-readable storage medium according to claim 18 , wherein the establishing a directed graph for the network structure of the image recognition neural network by using the network structure optimizer comprises:

performing sampling processing on the network structure of the image recognition neural network by using the network structure optimizer, to obtain network layer information for the network structure of the image recognition neural network; and

establishing the directed graph for the network structure of the image recognition neural network according to the network layer information.

20 . The non-transitory computer-readable storage medium according to claim 18 , wherein the performing feature extraction on the directed graph, to obtain the feature information corresponding to the network structure comprises:

determining node information and edge information in the directed graph; and

determining the feature information corresponding to the network structure according to learnable parameter matrixes and an activation function in a graph convolutional network (GCN), the node information, and the edge information.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (20)