Neural network based solution
A method for generating an output signal of a system based on input data received by the system includes receiving training data and training a neural network for generating the output signal by optimizing a primary cost function and an auxiliary cost function and modulating the auxiliary cost function with a gradient-based attention mask during the training.
1. A method, comprising:
receiving training data; and
training a neural network for generating an output signal by optimizing a primary cost function and an auxiliary cost function and modulating the auxiliary cost function with a gradient-based attention mask during the training, wherein the gradient-based attention mask includes relevance values that each represent relevance of elements from the training data.
2. The method of claim 1 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training focuses attention of the training on relevant features from the training data.
3. The method of claim 1 , wherein the primary cost function relates to a primary task, the auxiliary cost function relates to an auxiliary task, and modulating the auxiliary cost function with the gradient-based attention mask during the training focuses attention of the training on features from the data that are useful for the primary task.
4. The method of claim 1 , wherein, during the training, the training data are corrupted by adding noise to define corrupted training data, a first decoder outputs a reconstruction of the training data based on the corrupted training data, and a second decoder outputs the gradient-based attention mask based on the corrupted training data.
5. The method of claim 1 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training restricts the auxiliary cost function to account only for a subset of the elements from the training data.
6. The method of claim 1 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training applies weighting to the auxiliary cost function according the relevance values for the elements from the training data to define attention-weighted auxiliary costs for the elements.
7. The method of claim 1 , further comprising:
subsequent to training of the neural network:
receiving input data,
inputting the received input data to the neural network, and
generating the output signal in accordance with processing of the received input data by the neural network.
8. A non-transitory computer readable storage device including program instructions, wherein the program instructions, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising:
receiving training data; and
training a neural network for generating an output signal by optimizing a primary cost function and an auxiliary cost function and modulating the auxiliary cost function with a gradient-based attention mask during the training, wherein the gradient-based attention mask includes relevance values that each represent relevance of elements from the training data.
9. The non-transitory computer readable storage device of claim 8 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training focuses attention of the training on relevant features from the training data.
10. The non-transitory computer readable storage device of claim 8 , wherein the primary cost function relates to a primary task, the auxiliary cost function relates to an auxiliary task, and modulating the auxiliary cost function with the gradient-based attention mask during the training focuses attention of the training on features from the data that useful for the primary task.
11. The non-transitory computer readable storage device of claim 8 , wherein, during the training, the training data are corrupted by adding noise to define corrupted training data, a first decoder outputs a reconstruction of the training data based on the corrupted training data, and a second decoder outputs the gradient-based attention mask based on the corrupted training data.
12. The non-transitory computer readable storage device of claim 8 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training restricts the auxiliary cost function to account only for a subset of the elements from the training data.
13. The non-transitory computer readable storage device of claim 8 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training applies weighting to the auxiliary cost function according the relevance values for the elements from the training data to define attention-weighted auxiliary costs for the elements.
14. The non-transitory computer readable storage device of claim 8 , the operations further comprising:
subsequent to training of the neural network:
receiving input data,
inputting the received input data to the neural network, and
generating the output signal in accordance with processing of the received input data by the neural network.
15. A system, comprising:
a memory; and
one or more processors that are configured to execute program instructions that are stored in the memory, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:
receive training data, and
train a neural network for generating an output signal by optimizing a primary cost function and an auxiliary cost function and modulating the auxiliary cost function with a gradient-based attention mask during the training, wherein the gradient-based attention mask includes relevance values that each represent relevance of elements from the training data.
16. The system of claim 15 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training focuses attention of the training on relevant features from the training data.
17. The system of claim 15 , wherein the primary cost function relates to a primary task, the auxiliary cost function relates to an auxiliary task, and modulating the auxiliary cost function with the gradient-based attention mask during the training focuses attention of the training on features from the data are useful for the primary task.
18. The system of claim 15 , wherein, during the training, the training data are corrupted by adding noise to define corrupted training data, a first decoder outputs a reconstruction of the training data based on the corrupted training data, and a second decoder outputs the gradient-based attention mask based on the corrupted training data.
19. The system of claim 15 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training restricts the auxiliary cost function to account only for a subset of the elements from the training data.
20. The system of claim 15 , wherein modulating the auxiliary cost function with the gradient-based attention mask during the training applies weighting to the auxiliary cost function according the relevance values for the elements from the training data to define attention-weighted auxiliary costs for the elements.
21. The system of claim 15 , wherein the program instructions further cause the one or more processors to:
subsequent to training of the neural network:
receive input data,
input the received input data to the neural network, and
generate the output signal in accordance with processing of the received input data by the neural network.