Granted Patent B2

US 12,210,975 · App. 15/906,807 · Granted Jan 28, 2025

Inventor: Takuma Shibahara (Tokyo, JP)

Assignee: Hitachi, Ltd.

G06N3/084G06F17/16G06F18/24317G06N3/045G06N3/08

Abstract

Claims (49)

1. A data analysis apparatus, comprising:

a processor,

a memory comprising instructions, when executed by the processor, cause the processor to

use a first neural network that includes a first input layer, a first output layer, and a first intermediate layer having at least two layers between the first input layer and the first output layer, the first intermediate layer being configured to give data from a previous layer and a first adjustable learning parameter, wherein all learning parameters are adjustable weight vectors, to a first activation function for calculation and output a calculation result to a subsequent layer and;

generate degenerated output data, which is data where the number of dimensions of each output data has been reduced, based on the calculation result from each of the first intermediate layer;

receive output data from the first input layer and degenerated output data derived from each of the first intermediate layer, set a weight of each layer in the first intermediate layer based on the output data, the degenerated output data and a second adjustable learning parameter, and output said weight of each layer in the first intermediate layer to the first output layer;

generate weighted output data by weighting the calculation result with the weight of each layer of the first intermediate layer based on the output data and the degenerated output data; and

calculate prediction data based on each weighted output data and a third adjustable learning parameter;

wherein the data analysis apparatus is configured to receive output data from each first intermediate layer, reduce the number of dimensions of each output data, and output each degenerated output data,

wherein the data analysis apparatus receives each degenerated output data, sets a weight of each first intermediate layer based on said degenerated output data and the second learning parameter, and outputs the weight to the first output layer.

2. The data analysis apparatus according to claim 1 , wherein the data analysis apparatus is further configured to:

adjust the first learning parameter, the second learning parameter, and the third learning parameter using stochastic gradient decent when training data is given to the first input layer;

receive output data from each first intermediate layer;

generate degenerated output data by reducing the number of dimensions of each output data;

set a weight of each first intermediate layer based on said degenerated output data and the second learning parameter;

weight each second degenerated output data based on the weight of each first intermediate layer; and

output the weight to the first output layer.

3. The data analysis apparatus according to claim 1 , wherein the data analysis apparatus analyzes image, or text and/or numerical data.

4. The data analysis apparatus according to claim 1 , wherein:

the data analysis apparatus is further configured to adjust the first learning parameter, the second learning parameter, and the third learning parameter when training data is given to the first input layer.

5. The data analysis apparatus according to claim 4 , wherein the data analysis apparatus is configured to:

adjust the fourth learning parameter using a second neural network including a second input layer that receives the training data, a second output layer that outputs a hyperparameter of the first neural network, and a second intermediate layer interposed between the second input layer and the second output layer, the second intermediate layer being configured to give data from a previous layer and a fourth learning parameter to a second activation function for calculation and output a calculation result to a subsequent layer, when the training data is given to the second input layer;

output the hyperparameter from the second output layer by giving the training data to the second input layer of the second neural network after the fourth learning parameter is adjusted;

determine a structure of the first neural network based on the hyperparameter; and

adjust the first learning parameter, the second learning parameter, and the third learning parameter when the training data is given to the first input layer of the first neural network after the structure thereof is determined.

6. The data analysis apparatus according to claim 1 , further comprising:

the data analysis apparatus is further configured to receive output data from each first intermediate layer, reduce the number of dimensions of each output data, and output each second degenerated output data,

wherein the data analysis apparatus weights each second degenerated output data based on the weight of each first intermediate layer.

7. The data analysis apparatus according to claim 5 , wherein the hyperparameter is to determine a pattern of elements constituting the first neural network.

8. The data analysis apparatus according to claim 7 , wherein said hyperparameter that is to determine the pattern is a parameter indicating a type of the first activation function.

9. The data analysis apparatus according to claim 5 , wherein the hyperparameter is to determine a sequence of elements constituting the first neural network.

10. The data analysis apparatus according to claim 9 , wherein said hyperparameter that is to determine the sequence is a parameter indicating the number of layers in the first intermediate layer.

11. A data analysis method using a first neural network that includes a first input layer, a first output layer, and a first intermediate layer having at least two layers between the first input layer and the first output layer, the first intermediate layer being configured to give data from a previous layer and a first adjustable learning parameter, wherein all learning parameters are adjustable weight vectors, to a first activation function for calculation and output a calculation result to a subsequent layer,

wherein the data analysis method includes a processor and a storage device to store the first neural network, and

wherein the processor is configured to conduct:

a degeneration process to generate degenerated output data, which is data where the number of dimensions of each output data has been reduced, based on the calculation result from each of the first intermediate layer;

a setting process to receive output data from the first input layer and degenerated output data derived from each of the first intermediate layer, set a weight of each layer in the first intermediate layer based on the output data, the degenerated output data and a second adjustable learning parameter, and output said weight of each layer in the first intermediate layer to the first output layer;

a weighting process to generate weighted output data by weighting the calculation result with the weight of each layer of the first intermediate layer that was set in the setting process; and

a calculation process to calculate prediction data based on each output data that was weighted in the weighting process and a third adjustable learning parameter;

wherein the data analysis method includes receiving output data from each first intermediate layer, reducing the number of dimensions of each output data, and outputting each degenerated output data,

wherein the data analysis method includes receiving each degenerated output data, setting a weight of each first intermediate layer based on said degenerated output data and the second learning parameter, and outputting the weight to the first output layer.

12. A non-transitory recording medium having stored thereon a data analysis program that causes a processor to conduct prescribed processes, the processor being able to access a storage device having stored therein a first neural network that includes a first input layer, a first output layer, and a first intermediate layer having at least two layers between the first input layer and the first output layer, the first intermediate layer being configured to give data from a previous layer and a first adjustable learning parameter, wherein all learning parameters are adjustable weight vectors, to a first activation function for calculation and output a calculation result to a subsequent layer, the non-transitory recording medium being readable by the processor,

the data analysis program causing the processor to execute:

a degeneration process to generate degenerated output data, which is data where the number of dimensions of each output data has been reduced, based on the calculation result from each of the first intermediate layer;

a setting processing for receiving output data from the first input layer and degenerated output data derived from each of the first intermediate layer, set a weight of each layer in the first intermediate layer based on the output data, the degenerated output data and a second adjustable learning parameter, and output the weight of each layer in the first intermediate layer to the first output layer;

a weighting processing for generating weighted output data by weighting the calculation result with the weight of each layer of the first intermediate layer that was set in the setting process; and

a calculation processing for calculating prediction data based on each output data that was weighted in the weighting process and a third adjustable learning parameter;

wherein the data analysis program is configured to receive output data from each first intermediate layer, reduce the number of dimensions of each output data, and output each degenerated output data,

wherein the data analysis program receives each degenerated output data, sets a weight of each first intermediate layer based on said degenerated output data and the second learning parameter, and outputs the weight to the first output layer.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (9)