Automatically ascertaining an optimal architecture for a neural network
A computer-implemented method for ascertaining an optimal architecture for a neural network that solves a given task in accordance with given boundary conditions and/or optimization goals. The method includes: providing a graph of the possible architectures of nodes and edges, wherein nodes correspond to data, edges correspond to parameterized operations to be carried out on the data, and a path which traverses the entire graph corresponds to an architecture; in a search phase, generating candidate architectures based on already known architectures, wherein the candidate architectures are similar but not identical to the known architectures in accordance with a predetermined criterion; evaluating the candidate architectures using the given boundary conditions and/or optimization goals; ascertaining a candidate architecture having the best rating as the sought optimal architecture.
1 . A computer-implemented method for ascertaining an optimal architecture for a neural network that solves a given task in accordance with given boundary conditions and/or optimization goals, the method comprising the following steps:
providing a graph of possible architectures of nodes and edges, wherein;
each of the nodes corresponds to data;
each of the edges corresponds to one or more parameterized operations to be carried out on the data; and
a path which traverses the entire graph corresponds to an architecture of the possible architectures;
generating, in a search phase, candidate architectures based on already known architectures, wherein the candidate architectures are similar but not identical to the already known architectures in accordance with a predetermined criterion;
evaluating the candidate architectures using the given boundary conditions and/or optimization goals;
ascertaining a candidate architecture of the candidate architectures having a best rating as the ascertained optimal architecture;
providing the ascertained optimal architecture with measurement data recorded with at least one sensor;
ascertaining a control signal from output obtained from the ascertained optimal architecture; and
controlling, using the control signal, a vehicle, and/or a robot, and/or a driving assistance system, and/or a system for quality control, and/or a system for monitoring areas and/or a system for medical imaging.
2 . The method according to claim 1 , wherein an already known architecture of the already known architectures is modified to at least one candidate architecture by directly changing at least one edge of the already known architecture.
3 . The method according to claim 1 , wherein the boundary conditions and/or optimization goals include:
how quickly, and/or with what accuracy, the candidate architecture solves the given task; and/or
to what extent predetermined types of errors are avoided when solving the given task; and/or
how much working memory is needed to execute the candidate architecture; and/or
how many processors and/or GPUs are needed to execute the candidate architecture.
4 . The method according to claim 1 , wherein fulfilment of multiple boundary conditions and/or optimization goals is aggregated to a rating number.
5 . The method according to claim 1 , wherein the generating of the candidate architectures includes selecting a subset of nodes of the path that corresponds to an already known architecture of the already known architectures and generating a modification of the path that corresponds to the already known architecture that still passes through all of the selected subset of nodes.
6 . The method according to claim 5 , wherein the generating of the candidate architectures includes generating a modification of the path that corresponds to the already known architecture, which, in accordance with a predetermined metric, extends at no more than a predetermined distance from the path that corresponds to the already known architecture.
7 . The method according to claim 1 , wherein the generating of the candidate architectures includes sampling paths through the graph from a probability distribution that maximizes proximity to and/or intersections with the path that corresponds to an already known architecture of the already known architectures.
8 . The method according to claim 1 , wherein, as part of the evaluation using the boundary conditions and/or optimization goals, a candidate architecture of the candidate architectures is provided with unseen test data and/or validation data in a training phase.
9 . The method according to claim 1 , wherein the given task includes evaluating measurement data in the form of image data and/or point clouds.
10 . The method according to claim 1 , wherein the given task includes mapping input of the neural network to classification scores relating to one or more classes of a predetermined classification and/or to a regression value relating to at least one quantity of interest.
11 . The method according to claim 1 , wherein the candidate architectures are generated as part of an evolutionary algorithm.
12 . The method according to claim 1 , wherein, before the search phase, learning steps regarding the given task are carried out in a training phase for architectures sampled from the graph.
13 . The method according to claim 1 , wherein the ascertained optimal architecture is trained or further trained with training examples respectively labeled with target outputs relating to the given task.
14 . A non-transitory machine-readable storage medium on which is stored one or more computer programs for ascertaining an optimal architecture for a neural network that solves a given task in accordance with given boundary conditions and/or optimization goals, the one or more computer programs, when executed by one or more processors, causing the one or more processors to perform the following steps:
providing a graph of possible architectures of nodes and edges, wherein:
each of the nodes corresponds to data;
each of the edges corresponds to one or more parameterized operations to be carried out on the data; and
a path which traverses the entire graph corresponds to an architecture of the possible architectures;
generating, in a search phase, candidate architectures based on already known architectures, wherein the candidate architectures are similar but not identical to the already known architectures in accordance with a predetermined criterion;
evaluating the candidate architectures using the given boundary conditions and/or optimization goals;
ascertaining a candidate architecture of the candidate architectures having a best rating as the ascertained optimal architecture;
providing the ascertained optimal architecture with measurement data recorded with at least one sensor;
ascertaining a control signal from output obtained from the ascertained optimal architecture; and
controlling, using the control signal, a vehicle, and/or a robot, and/or a driving assistance system, and/or a system for quality control, and/or a system for monitoring areas and/or a system for medical imaging.
15 . One or more computers comprising one or more processors and a non-transitory machine-readable storage medium on which is stored one or more computer programs for ascertaining an optimal architecture for a neural network that solves a given task in accordance with given boundary conditions and/or optimization goals, the one or more computer programs, when executed by the one or more processors, causing the one or more processors to perform the following steps:
providing a graph of possible architectures of nodes and edges, wherein:
each of the nodes corresponds to data;
each of the edges corresponds to one or more parameterized operations to be carried out on the data, and
a path which traverses the entire graph corresponds to an architecture of the possible architectures;
generating, in a search phase, candidate architectures based on already known architectures, wherein the candidate architectures are similar but not identical to the already known architectures in accordance with a predetermined criterion
evaluating the candidate architectures using the given boundary conditions and/or optimization goals;
ascertaining a candidate architecture of the candidate architectures having a best rating as the ascertained optimal architecture;
providing the ascertained optimal architecture with measurement data recorded with at least one sensor;
ascertaining a control signal from output obtained from the ascertained optimal architecture; and
controlling, using the control signal, a vehicle, and/or a robot, and/or a driving assistance system, and/or a system for quality control, and/or a system for monitoring areas and/or a system for medical imaging.