IP Library › Granted Patent US 11,580,408
Granted Patent B2
US 11,580,408 · App. 16/830,566 · Granted Feb 14, 2023

Search method, device and storage medium for neural network model structure

Inventors: Xiangxiang Chu (Beijing, CN); Ruijun Xu (Beijing, CN); Bo Zhang (Beijing, CN); Jixiang Li (Beijing, CN); Qingyuan Li (Beijing, CN)
Assignee: Beijing Xiaomi Intelligent Technology Co., Ltd.
G06N3/086G06F7/58G06K9/6262G06N3/0454G06N3/0472
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,580,408
App. No.
16/830,566
Granted
Feb 14, 2023
Kind
B2
Abstract

A search method for a neural network model structure, includes: generating an initial generation population of network model structure based on multi-objective optimization hyper parameters, as a current generation population of network model structure; performing selection and crossover on the current generation population of network model structure; generating a part of network model structure based on reinforcement learning mutation, and generating a remaining part of network model structure based on random mutation on the selected and crossed network model structure; generating a new population of network model structure based on the part of network model structure generated by reinforcement learning mutation and the remaining part of network model structure generated by random mutation; and searching a next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure.

Claims (56)

1. A search method for a neural network model structure, comprising:

generating an initial generation population of network model structure based on multi-objective optimization hyper parameters, as a current generation population of network model structure;

performing selection and crossover on the current generation population of network model structure;

generating a part of network model structure based on reinforcement learning mutation, and generating a remaining part of network model structure based on random mutation on the selected and crossed network model structure;

generating a new population of network model structure based on the part of network model structure generated by reinforcement learning mutation and the remaining part of network model structure generated by random mutation;

searching a next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure; and

using the next generation population of network model structure as the current generation population of network model structure, repeating the performing selection and crossover, the generating a part of network model structure and generating a remaining part of network model structure, the generating a new population of network model structure, and the searching a next generation population of network model structure, until a multi-objective optimization state is optimal, and selecting a neural network model structure suitable for different scenarios from a final generation population of network model structure.

2. The search method according to claim 1 , wherein the multi-objective optimization parameters comprise an inference time for running the network model structure; and

after generating the new population of the network model structure, the search method further comprises:

sending each network model structure in the new population of network model structure to a device side running the network model structure, and running the network model structure by a neural network engine of the device side; and

obtaining an inference time of the neural network engine of the device side for running each network model structure in the new population of the network model structure;

wherein the searching a next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure comprises:

searching the next generation population of network model structure based on the inference time, the current generation population of network model structure, and the new population of network model structure.

3. The search method according to claim 1 , wherein the searching a next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure comprises:

searching the next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure using a predictive evaluation index;

wherein the predictive evaluation index is determined according to an automatic encoding regressor, and the automatic encoding regressor is an automatic encoding regressor for predicting an evaluation index and is obtained by pre-training the network model structure and the evaluation index.

4. The search method according to claim 1 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

5. The search method according to claim 2 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

6. The search method according to claim 3 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

7. A search device for a neural network model structure, comprising:

a processor; and

a memory storing instructions executable by the processor, wherein the processor is configured to;

generate an initial generation population of network model structure based on multi-objective optimization hyperparameters, as a current generation population of network model structure;

perform selection and crossover on the current generation population of network model structure;

generate a part of network model structure based on reinforcement learning mutation, and generate a remaining part of network model structure based on random mutation on the selected and crossed network model structure;

generate a new population of network model structure based on the part of network model structure generated by reinforcement learning mutation and the remaining part of network model structure generated by random mutation; and

search a next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure; and

using the next generation population of network model structure as the current generation population of network model structure, repeat the performing selection and crossover, the generating a part of network model structure and generating a remaining part of network model structure, the generating a new population of network model structure, and the searching a next generation population of network model structure, until a multi-objective optimization state is optimal, and select a neural network model structure suitable for different scenarios from a final generation population of network model structure.

8. The search device according to claim 7 , wherein the multi-objective optimization parameters comprise an inference time for running the network model structure; and the processor is further configured to:

after generating the new population of the network model structure, send each network model structure in the new population of network model structure to a device side running the network model structure, run the network model structure by a neural network engine of the device side, and obtain inference time of the neural network engine of the device side for running each network model structure in the new population of the network model structure; and

search the next generation population of network model structure based on the inference time, the current generation population of network model structure, and the new population of network model structure.

9. The search device according to claim 7 , wherein the processor is further configured to:

search the next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure using a predictive evaluation index;

wherein the predictive evaluation index is determined according to the automatic encoding regressor, and the automatic encoding regressor is an automatic encoding regressor for predicting an evaluation index and is obtained by pre-training the network model structure and the evaluation index.

10. The search device according to claim 7 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

11. The search device according to claim 8 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

12. The search device according to claim 9 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

13. A non-transitory computer-readable storage medium having stored therein instructions that, when executed by a processor of a terminal, cause the terminal to perform a search method for a neural network model structure, the search method comprising:

generating an initial generation population of network model structure based on multi-objective optimization hyper parameters, as a current generation population of network model structure;

performing selection and crossover on the current generation population of network model structure;

generating a part of network model structure based on reinforcement learning mutation, and generating a remaining part of network model structure based on random mutation on the selected and crossed network model structure;

generating a new population of network model structure based on the part of network model structure generated by reinforcement learning mutation and the remaining part of network model structure generated by random mutation;

searching a next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure;

using the next generation population of network model structure as the current generation population of network model structure, repeating the performing selection and crossover, the generating a part of network model structure and generating a remaining part of network model structure, the generating a new population of network model structure, and the searching a next generation population of network model structure, until a multi-objective optimization state is optimal, and selecting a neural network model structure suitable for different scenarios from a final generation population of network model structure.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the multi-objective optimization parameters comprise an inference time for running the network model structure; and

after generating the new population of the network model structure, the search method further comprises:

sending each network model structure in the new population of network model structure to a device side running the network model structure, and running the network model structure by a neural network engine of the device side; and

obtaining an inference time of the neural network engine of the device side for running each network model structure in the new population of the network model structure;

wherein the searching a next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure comprises:

searching the next generation population of network model structure based on the inference time, the current generation population of network model structure, and the new population of network model structure.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the searching a next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure comprises:

searching the next generation population of network model structure based on the current generation population of network model structure and the new population of network model structure using a predictive evaluation index;

wherein the predictive evaluation index is determined according to an automatic encoding regressor, and the automatic encoding regressor is an automatic encoding regressor for predicting an evaluation index and is obtained by pre-training the network model structure and the evaluation index.

16. The non-transitory computer-readable storage medium of claim 13 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

17. The non-transitory computer-readable storage medium of claim 14 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the part of network model structure generated based on reinforcement learning mutation and the remaining part of network model structure generated by random mutation satisfy a preset ratio, the preset ratio being a random number.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 26, 2020
From: CHU, XIANGXIANG; XU, RUIJUN; ZHANG, BO; LI, JIXIANG; LI, QINGYUAN
To: BEIJING XIAOMI INTELLIGENT TECHNOLOGY CO., LTD.
Reel/Frame 052232/0872 →
Priority Claims (1)
CN 201910979571.7 · Oct 15, 2019 · national
Continuity (1)
Related Publication 20210110276A1 · Apr 15, 2021