Neural network systems implementing conditional neural processes for efficient learning
According to a first aspect a network system to generate output data values from input data values according to one or In more learned data distributions comprises an input to receive a set of observations, each comprising a respective first data value for a first variable and a respective second data value for a second variable dependent upon the first variable. The system may comprise an encoder neural network system configured to encode each observation of the set of observations to provide an encoded output for each observation. The system may further comprise an aggregator configured to aggregate the encoded outputs for the set of observations and provide an aggregated output. The system may further comprise a decoder neural network system configured to receive a combination of the aggregated output and a target input value and to provide a decoder output. The target input value may comprise a value for the first variable and the decoder output may predict a corresponding value for the second variable.
1 . A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
receiving a set of observations, each observation comprising image data and a classification of the image data;
for each observation in the set of observations, inputting the observation into an encoder neural network system to encode the observation including the image data and the classification of the image data to provide an encoded observation for the observation;
inputting the encoded observations for the set of observations that include both the encoded image data and the encoded classifications of the image data into an aggregator to aggregate the encoded observations and provide an aggregated parameter, the aggregating comprising applying a mean function to the encoded observations; and
inputting a combination of the aggregated parameter and target image data for a target observation that includes both the target image data and a target classification of the target image data into a decoder neural network system to provide a predicted classification of the target image data.
2 . The system of claim 1 , wherein the operations comprise, for each of a plurality of training steps:
selecting, the set of observations, a subset of training observations for input to the encoder neural network system and a set of training target observations for training the system;
encoding each training observation using the encoder neural network system to provide an encoded observation for each training observation;
aggregating the encoded observations;
combining the aggregate of the encoded observations with respective training target image data for each training target observation in the set of training target observations to provide a set of combinations for decoding;
decoding each of the set of combinations using the decoder neural network system to provide a predicted classification of the training target image data; and
training the encoder neural network system and the decoder neural network system with back propagation using a loss function dependent upon the predicted classifications for the set of training target observations and ground truth classifications for the set of training target observations.
3 . The system of claim 1 , wherein the operations comprise:
generating, by the decoder neural network system, a predicted distribution for the predicted classification of the target image data; and
determining the predicted classification of the target image data from the predicted distribution for the predicted classification of the target image data.
4 . The system of claim 3 wherein each of the image data comprises a vector defining the image data and wherein the predicted distribution for the predicted classification of the target image data defines a probability distribution over the classifications.
5 . The system of claim 4 , wherein the aggregator is configured to:
for each class of a plurality of potential classes for the classifications of the image data of the set of observations, determine an aggregate for the class based on the encoded image data for the observations in that class; and
determine the aggregated parameter by combining the aggregates for each class of the plurality of potential classes.
6 . The system of claim 1 , wherein the aggregated parameter is invariant under permutations of the set of observations.
7 . The system of claim 1 , wherein the combination received by the decoder neural network system further comprises a set of latent variables which together characterize the target image data.
8 . The system of claim 7 , wherein the operations further comprise determining parameters of a distribution of the set of latent variables, and sampling, by the decoder neural network system, from the distribution of the set of latent variables to provide the predicted classification of the target image data.
9 . The system of claim 8 , the operations further comprise determining, by a prior neural network, the parameters of a prior distribution of the set of latent variables from the set of observations and determining, by a posterior neural network, the parameters of a posterior distribution of the set of latent variables from the target observation.
10 . The system of claim 1 , wherein the combination of the aggregated parameter and the target image data for the target observation comprises a concatenation of the aggregated parameter and the target image data.
11 . A computer implemented method for generating decoder output data values from input data values according to one or more learned data distributions, the method comprising:
receiving a set of observations, each observation comprising image data and a classification of the image data;
for each observation in the set of observations, inputting the observation into an encoder neural network system to encode the observation including the image data and the classification of the image data to provide an encoded observation for the observation;
inputting the encoded observations for the set of observations that include both the encoded image data and the encoded classifications of the image data into an aggregator to aggregate the encoded observations and provide an aggregated parameter, the aggregating comprising applying a mean function to the encoded observations; and
inputting a combination of the aggregated parameter and target image data for a target observation that includes both the target image data and a target classification of the target image data into a decoder neural network system to provide a predicted classification of the target image data.
12 . The method of claim 11 , comprising, for each of a plurality of training steps:
selecting, the set of observations, a subset of training observations for input to the encoder neural network system and a set of training target observations for training the system;
encoding each training observation using the encoder neural network system to provide an encoded observation for each training observation;
aggregating the encoded observations;
combining the aggregate of the encoded observations with respective training target image data for each training target observation in the set of training target observations to provide a set of combinations for decoding;
decoding each of the set of combinations using the decoder neural network system to provide a predicted classification of the training target image data; and
training the encoder neural network system and the decoder neural network system with back propagation using a loss function dependent upon the predicted classifications for the set of training target observation and ground truth classifications for the set of training target observations.
13 . The method of claim 11 , further comprising:
generating, by the decoder neural network system, a predicted distribution for the predicted classification of the target image data; and
determining the predicted classification of the target image data from the predicted distribution for the predicted classification of the target image data.
14 . The method of claim 11 , wherein the aggregated parameter is invariant under permutations of the set of observations.
15 . The method of claim 11 , wherein the combination received by the decoder neural network system further comprises a set of latent variables which together characterize the target image data.
16 . The method of claim 15 , further comprising determining parameters of a distribution of the set of latent variables, and sampling, by the decoder neural network system, from the distribution of the set of latent variables to provide the predicted classification of the target image data.
17 . The method of claim 16 , further comprising determining, by a prior neural network, the parameters of a prior distribution of the set of latent variables from the set of observations and determining, by a posterior neural network, the parameters of a posterior distribution of the set of latent variables from the target observation.
18 . The method of claim 11 , wherein the combination of the aggregated parameter and the target image data for the target observation comprises a concatenation of the aggregated parameter and the target image data.
19 . A method for generating decoder output data values from input data values according to one or more learned data distributions, the method comprising:
receiving a target image data for a target observation;
combining the target image data with an aggregated parameter that is an aggregate of a plurality of encoded observations encoded via an encoder neural network from a set of training observations, each of the training observations comprising respective image data for a training observation and one or more respective classification for the training observation, the aggregating comprising applying a mean function to the encoded observations; and
inputting the combination of the target image data and the aggregated parameter into a decoder neural network system to provide a predicted classification for the target image data.
20 . The method of claim 19 , further comprising, for each of a plurality of training steps:
receiving a training set of observations, each comprising image data for the observation and corresponding ground truth classification for the observation;
combining the aggregated parameter with each of the image data-from the training set of observations to provide a set of combinations for decoding;
decoding each of the set of combinations using the decoder neural network system to provide a set of predicted classifications for the image data from the training set of observations; and
training the decoder neural network system with back propagation using a loss function dependent upon the set of predicted classifications and the ground truth classifications for the image data from the training set of observations.