Learning device and learning method
A learning device is configured to perform learning of a decision tree. The learning device includes a branch score calculator, and a scaling unit. The branch score calculator is configured to calculate a branch score used for determining a branch condition for a node of the decision tree based on a cumulative sum of gradient information corresponding to each value of a feature amount of learning data. The scaling unit is configured to perform scaling on a value related to the cumulative sum used for calculating the branch score by the branch score calculator to fall within a numerical range with which the branch score is capable of being calculated.
1. A learning device for learning of a decision tree, the learning device comprising:
a controller, and
learning circuitry configured to:
calculate a branch score used for determining a branch condition for a node of the decision tree based on a cumulative sum of gradient information corresponding to each value of a feature amount of learning data; and
perform scaling on a value related to the cumulative sum used for calculating the branch score by the learning circuitry to fall within a numerical range with which the branch score is capable of being calculated,
wherein the learning circuitry is configured by a field-programmable gate array, and the controller performs data communication with the field-programmable gate array,
the learning circuitry is further configured to perform an operation on respective histograms of all feature amounts input at a time and store the histograms, and perform an operation on all of the feature amounts in parallel, and
the learning circuitry is configured to calculate Gain as the branch score by the following expression (1):
G
ain
=
1
2
[
G
L
2
H
L
+
λ
+
G
R
2
H
R
+
λ
-
(
G
L
+
G
R
)
2
H
L
+
H
R
+
λ
]
-
γ
wherein:
Gain: the branch score,
G L : a sum of gradient information (first order gradients) of samples branched to a left node,
G R : a sum of gradient information (first order gradients) of samples branched to a right node,
H L : a sum of gradient information (second order gradients) of samples branched to the left node,
H R : a sum of gradient information (second order gradients) of samples branched to the right node, and
γ,λ: regularization terms.
2. The learning device according to claim 1 , wherein
the learning circuitry includes a divider configured to calculate the branch score, and
the learning circuitry is configured to perform the scaling on the value related to the cumulative sum and input to the divider.
3. The learning device according to claim 1 , wherein
the learning circuitry includes an approximation arithmetic circuit configured to perform an approximation operation on the branch score, and
the learning circuitry performs scaling on the value related to the cumulative sum and input to the approximation arithmetic circuit.
4. The learning device according to claim 3 , wherein the approximation arithmetic circuit is configured to calculate an approximate value of a logarithm of the branch score by performing linear interpolation on the logarithm.
5. The learning device according to claim 1 , wherein the learning circuitry is further configured to:
calculate a scaling amount for the value related to the cumulative sum based on a maximum value of an absolute value of the cumulative sum, and
perform the scaling on the value related to the cumulative sum to fall within the numerical range using the scaling amount calculated.
6. The learning device according to claim 1 , wherein the learning circuitry is further configured to:
calculate a scaling amount for the value related to the cumulative sum based on a sum total of the gradient information, and
perform the scaling on the value related to the cumulative sum to fall within the numerical range using the scaling amount calculated.
7. The learning device according to claim 1 , wherein the learning circuitry is further configured to:
calculate a scaling amount for the value related to the cumulative sum based on a number of samples of the learning data at each node of the decision tree, and
perform the scaling on the value related to the cumulative sum to fall within the numerical range using the scaling amount calculated.
8. The learning device according to claim 1 , wherein the learning circuitry is further configured to:
calculate a scaling amount for the value related to the cumulative sum based on the value related to the cumulative sum; and
perform inverse scaling that restores, based on the scaling amount, a value operated by the learning circuitry using the value related to the cumulative sum and subjected to the scaling performed by the learning circuitry, to an original scale.
9. The learning device according to claim 1 , wherein the learning of the decision tree is performed by gradient boosting.
10. The learning device according to claim 1 , wherein, in a case in which the feature amount has two or more dimensions, the learning circuitry is provided for each feature amount.
11. The learning device according to claim 1 , wherein the learning circuitry is further configured to:
calculate a leaf weight as an output with respect to an input to the decision tree using a division circuit in a case in which the node of the decision tree is a terminal node; and
perform scaling on a value related to the cumulative sum and used for calculating the leaf weight by the learning circuitry to cause the value to fall within a numerical range with which the leaf weight is capable of being calculated.
12. A non-transitory computer readable memory device that includes instructions that when executed by a processor cause the processor to perform a method, the method comprising:
calculating a branch score used for determining a branch condition for a node of the decision tree based on a cumulative sum of gradient information corresponding to each value of a feature amount of learning data;
performing scaling on a value related to the cumulative sum used for calculating the branch score to cause the value to fall within a numerical range with which the branch score is capable of being calculated;
performing an operation on respective histograms of all feature amounts input at a time and storing the histograms; and
performing an operation on all of the feature amounts in parallel,
wherein the calculating the branch score comprises calculating Gain as the branch score by the following expression (1):
G
ain
=
1
2
[
G
L
2
H
L
+
λ
+
G
R
2
H
R
+
λ
-
(
G
L
+
G
R
)
2
H
L
+
H
R
+
λ
]
-
γ
wherein:
Gain: the branch score,
G L : a sum of gradient information (first order gradients) of samples branched to a left node,
G R : a sum of gradient information (first order gradients) of samples branched to a right node,
H L : a sum of gradient information (second order gradients) of samples branched to the left node,
H R : a sum of gradient information (second order gradients) of samples branched to the right node, and
γ,λ: regularization terms.