Learning from delayed outcomes using neural networks

1. A method performed by one or more computers, the method comprising:

receiving an input observation;

generating, from the input observation, an output label distribution over possible labels for the input observation, the generating comprising:

processing the input observation using a first neural network configured to process the input observation to generate a distribution over possible values for an intermediate indicator, wherein the intermediate indicator is a second, different observation;

generating, from the distribution over possible values for the intermediate indicator, a predicted input value for the intermediate indicator; and

processing the predicted input value for the intermediate indicator independently of the input observation using a second neural network configured to process the predicted input value for the intermediate indicator to determine the output label distribution over possible values for the input observation, and

providing an output derived from the output label distribution.

2. The method of claim 1 wherein the first neural network is configured to apply a softmax transform to the distribution for the intermediate indicator.

3. The method of claim 1 wherein the second neural network is configured to receive the input value of the intermediate indicator as a one-hot encoded input value.

4. The method of claim 1 , further comprising:

processing, using a third neural network, one or more of: the input observation, the predicted input value for the intermediate indicator, and the output label distribution over possible labels for the input observation, to generate a correction to the output label distribution.

5. The method of claim 4 , further comprising:

generating a corrected output label distribution based on the determined output label distribution and the determined correction.

6. The method of claim 5 , wherein the provided output is the corrected output label distribution or data identifying one or more highest-scoring labels according to the corrected output label distribution.

7. The method of claim 4 , wherein the third neural network is configured to receive the input value of the intermediate indicator as a one-hot encoded input value.

8. The method of claim 1 , wherein generating, from the distribution over possible values for the intermediate indicator, the predicted input value for the intermediate indicator comprises:

sampling the predicted input value from the distribution over possible values or selecting a possible value having the highest score in the distribution.

9. A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving an input observation;

generating, from the input observation, an output label distribution over possible labels for the input observation, the generating comprising:

processing the input observation using a first neural network configured to process the input observation to generate a distribution over possible values for an intermediate indicator, wherein the intermediate indicator is a second, different observation;

generating, from the distribution over possible values for the intermediate indicator, a predicted input value for the intermediate indicator; and

processing the predicted input value for the intermediate indicator independently of the input observation using a second neural network configured to process the predicted input value for the intermediate indicator to determine the output label distribution over possible values for the input observation, and

providing an output derived from the output label distribution.

10. The system of claim 9 wherein the first neural network is configured to apply a softmax transform to the distribution for the intermediate indicator.

11. The system of claim 9 wherein the second neural network is configured to receive the input value of the intermediate indicator as a one-hot encoded input value.

12. The system of claim 9 , the operations further comprising:

processing, using a third neural network, one or more of: the input observation, the predicted input value for the intermediate indicator, and the output label distribution over possible labels for the input observation, to generate a correction to the output label distribution.

13. The system of claim 12 , the operations further comprising:

generating a corrected output label distribution based on the determined output label distribution and the determined correction.

14. The system of claim 13 , wherein the provided output is the corrected output label distribution or data identifying one or more highest-scoring labels according to the corrected output label distribution.

15. The system of claim 12 , wherein the third neural network is configured to receive the input value of the intermediate indicator as a one-hot encoded input value.

16. The system of claim 9 , wherein generating, from the distribution over possible values for the intermediate indicator, the predicted input value for the intermediate indicator comprises:

sampling the predicted input value from the distribution over possible values or selecting a possible value having the highest score in the distribution.

17. One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving an input observation;

generating, from the input observation, an output label distribution over possible labels for the input observation, the generating comprising:

processing the input observation using a first neural network configured to process the input observation to generate a distribution over possible values for an intermediate indicator, wherein the intermediate indicator is a second, different observation;

generating, from the distribution over possible values for the intermediate indicator, a predicted input value for the intermediate indicator; and

processing the predicted input value for the intermediate indicator independently of the input observation using a second neural network configured to process the predicted input value for the intermediate indicator to determine the output label distribution over possible values for the input observation, and

providing an output derived from the output label distribution.

18. The computer-readable storage media of claim 17 wherein the second neural network is configured to receive the input value of the intermediate indicator as a one-hot encoded input value.

19. The computer-readable storage media of claim 17 , the operations further comprising:

processing, using a third neural network, one or more of: the input observation, the predicted input value for the intermediate indicator, and the output label distribution over possible labels for the input observation, to generate a correction to the output label distribution.

20. The computer-readable storage media of claim 19 , the operations further comprising:

generating a corrected output label distribution based on the determined output label distribution and the determined correction.