Method for determining the encoder architecture of a neural network
A method for determining an encoder architecture of a convolutional neural network configured to process image processing tasks. For each image processing task), characteristic scale distribution is calculated based on training data. Encoder architecture candidates are generated, each including a shared encoder layer providing computational operations for image processing tasks and branches which span over encoder layers providing at least partly different computational operations for the image processing tasks. Each branch is associated with a certain image processing task. Receptive encoder layer field sizes and assessment measures are calculated, each assessment measure referring to a combination of a certain encoder architecture and a certain image processing task, and including information regarding matching quality of characteristic scale distribution associated with the assessment measure to the receptive field sizes of the encoder layers. The assessment measures are compared and a comparison result established. An encoder architecture is selected based on the comparison result.
1 . A method for determining an architecture of an encoder of a convolutional neural network, the neural network being configured to process multiple different image processing tasks, the method comprising:
for each image processing task, calculating a characteristic scale distribution based on training data, the characteristic scale distribution indicating a size distribution of objects to be detected by the respective image processing task;
generating multiple encoder architecture candidates, each encoder architecture of the encoder architecture candidates comprising at least one shared encoder layer which provides computational operations for multiple image processing tasks and multiple branches which span over one or more encoder layers which provide at least partly different computational operations for the image processing tasks, wherein each branch is associated with a certain image processing task;
calculating receptive field sizes of the encoder layers of the multiple encoder architectures;
calculating multiple assessment measures, each assessment measure referring to a combination of a certain encoder architecture of the multiple encoder architectures and a certain image processing task, each assessment measure including information regarding the quality of matching of characteristic scale distribution of the image processing task associated with the assessment measure to the receptive field sizes of the encoder layers of the encoder architecture associated with the assessment measure;
comparing the calculated assessment measures and establishing a comparison result; and
selecting an encoder architecture based on the comparison result.
2 . The method according to claim 1 , further comprising determining for the characteristic scale distribution of each image processing task, one or more percentiles, values of the one or more percentiles depending on an interface of a task-specific decoder.
3 . The method according to claim 2 , wherein a number of determined percentiles is chosen according to a number of input connections required by the task-specific decoder.
4 . The method according to claim 2 , wherein the values of the percentiles are distributed across a percentile range defined by a minimum and a maximum percentile value.
5 . The method according to claim 4 , wherein the values of the percentiles are equally distributed across the percentile range.
6 . The method according to claim 1 , wherein at least some of the encoder layers of a task-specific branch of the encoder comprise a feature map with a resolution which matches to a feature resolution of an input connection of a task-specific decoder.
7 . The method according to claim 1 , wherein generating multiple encoder architecture candidates comprises selecting certain building blocks, which are adapted for an encoder of a neural network, out of a set of building blocks and connecting the selected building blocks in order to obtain an encoder architecture.
8 . The method according to claim 1 , wherein the encoder architectures of the encoder architecture candidates are determined by defining a runtime limit and the encoder architectures are created such that, at a given hardware, the encoder architectures provide computational results in a runtime range of 90% to 100% of the runtime limit.
9 . The method according to claim 1 , wherein calculating multiple assessment measures comprises comparing the receptive field size of an encoder layer which matches with a resolution of a task-specific decoder with a determined percentile value and determining a distance between the receptive field size and the percentile value.
10 . The method according to claim 1 , wherein calculating multiple assessment measures comprises comparing multiple receptive field sizes of encoder layers which match with a resolution of a task-specific decoder with multiple determined percentile values and determining a sum of distances between the receptive field sizes and the percentile values.
11 . The method according to claim 1 , wherein calculating multiple assessment measures uses a loss function with a least absolute deviations distance measure, the absolute deviations distance measure comprising an L1 distance measure or a cosine distance measure.
12 . The method according to claim 1 , wherein calculating multiple assessment measures uses a loss function with a penalty term, the penalty term being adapted to increase the assessment measure with a decreasing number of layers shared between multiple image processing tasks of an encoder architecture.
13 . The method according to claim 1 , wherein comparing the calculated assessment measures and establishing a comparison result comprises determining an encoder architecture for each image processing task which has a lowest assessment measure.
14 . The method to claim 1 , wherein selecting an encoder architecture based on the comparison result comprises selecting, for each image processing task, the encoder architecture which has a lowest assessment measure.
15 . A computer program product for determining the architecture of an encoder of a convolutional neural network, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions being executable by a processor to cause the processor to execute a method comprising:
for each image processing task, calculating a characteristic scale distribution based on training data, the characteristic scale distribution indicating a size distribution of objects to be detected by the respective image processing task;
generating multiple encoder architecture candidates, each encoder architecture of the encoder architecture candidates comprising at least one shared encoder layer which provides computational operations for multiple image processing tasks and multiple branches which span over one or more encoder layers which provide at least partly different computational operations for the image processing tasks, wherein each branch is associated with a certain image processing task;
calculating receptive field sizes of the encoder layers of the multiple encoder architectures;
calculating multiple assessment measures, each assessment measure referring to a combination of a certain encoder architecture of the multiple encoder architectures and a certain image processing task, each assessment measure including information regarding the quality of matching of characteristic scale distribution of the image processing task associated with the assessment measure to the receptive field sizes of the encoder layers of the encoder architecture associated with the assessment measure;
comparing the calculated assessment measures and establishing a comparison result; and
selecting an encoder architecture based on the comparison result.
16 . The computer program product according to claim 15 , wherein the method executed by the processor further comprises determining, for the characteristic scale distribution of each image processing task, one or more percentiles, values of the one or more percentiles depending on an interface of a task-specific decoder.