MULTIOBJECTIVE COEVOLUTION OF DEEP NEURAL NETWORK ARCHITECTURES

1 . A method for co-evolution of deep neural network (DNN) hyperparameters and architecture in accordance with multiple objective optimization, the method comprising:

evaluating assembled DNNs in an initial population in accordance with a primary and secondary objective, wherein each assembled DNN in the initial population is comprised of modules and is assembled in accordance with an architecture blueprint, and further wherein the evaluation results in selection of a first group of selected assembled DNNs and a second group of leftover assembled DNNs;

for each subsequent generation of an evolution run:

creating an archive of assembled DNNs from the second group of leftover assembled DNNs from a last generation;

evaluating each assembled DNN in the archive based on each of the primary and secondary objectives;

determining a Pareto front of assembled DNN in the archive;

removing assembled DNNs in the Pareto front and adding the removed DNNs to a new archive; and

truncating the new archive by removing a last fraction of added assembled DNNs;

wherein the method for co-evolution ends when values for at least the primary objective converge.

2 . The method of claim 1 , wherein determining Pareto front of the assembled DNN in the archive includes:

sorting the assembled DNN in the archive in a descending order by the primary objective;

creating a new Pareto front with a first individual assembled DNN in the archive; and

for each successive individual,

if a corresponding value for a secondary objective of the successive individual is greater than that of a preceding individual, adding the successive individual to the Pareto front.

3 . The method of claim 2 , further including sorting the Pareto front in a descending order based on values of the secondary objective.

4 . The method of claim 1 , wherein the primary objective is fitness and the secondary objective is novelty.

5 . The method of claim 1 , wherein the novelty is a behavior vector of multiple concatenated features.

6 . The method of claim 1 , wherein the concatenated features include:

number of parameters of the assembled DNN;

number of layers (L) in the assembled DNN;

number of connections between layers (C);

length of a longest path within the assembled DNN;

density of the assembled DNN, defined as D=|C|/|L|(|L|−1)

a maximum of the number of connections for each layer;

a mean of the number of connections for each layer;

a standard deviation of the number of connections for each layer;

a maximum of page rank for each layer;

a mean of page rank for each layer; and

standard deviation of page rank for each layer.