Data pruning in tree-based fitted Q iteration
View Patent ↗A computer-implemented method is provided for data reduction in a memory device for machine learning. The method includes storing, in the memory device, data that has been used for training in a tree-based fitted Q iteration session which learns an action value function with an ensemble of decision trees from the data. The method further includes determining, by a processor device, samples to be removed from the data based on a number of samples which belong to leaf nodes of the decision trees. The method also includes removing, from the memory device, the determined samples from the data to reduce an amount of the data. The method additionally includes learning, by the processor device, a new ensemble of decision trees using the data from which the determined samples have been removed together with new data.
1 . A computer-implemented method for data reduction in a memory device for machine learning, comprising:
storing, in the memory device, data that has been used for training in a tree-based fitted Q iteration session which learns an action value function with an ensemble of decision trees from the data;
determining, by a processor device, samples to be removed from the data based on a number of samples which belong to leaf nodes of the decision trees having identical positions within different decision trees from the ensemble of decision trees based on sample similarity to obtain determined samples;
removing, from the memory device, the determined samples from the data to reduce an amount of the data based on determined leaf node statistics for sample similarity by using the ensemble of decision trees as a regressor to obtain pruned data; and
learning, by the processor device, a new ensemble of decision trees using the pruned data together with new data.
2 . The computer-implemented method of claim 1 , wherein the samples which belong to a leaf node are removed from the memory device, when a number of the samples which belong to the leaf node exceeds a pruning threshold based on sample similarity.
3 . The computer-implemented method of claim 1 , wherein said storing, determining, removing, and learning steps are repeated recursively to efficiently learn a true action-value function representing the pruned data and the new data.
4 . The computer-implemented method of claim 1 , wherein each of the samples is removed with a probability that is determined responsive to an average number of the samples, where the average number of samples is taken over all leaf nodes which a given sample belongs to.
5 . The computer-implemented method of claim 1 , wherein the number of samples used to determine whether to remove data comprises an average number of samples that belong to a given leaf node.
6 . The computer-implemented method of claim 1 , wherein the number of samples used to determine whether to remove data comprises a minimum number of samples that belong to a given leaf node.
7 . The computer-implemented method of claim 1 , wherein the number of samples used to determine whether to remove data comprises a maximum number of samples that belong to a given leaf node.
8 . The computer-implemented method of claim 1 , wherein the number of samples used to determine whether to remove data comprises a median of samples that belong to a given leaf node.
9 . The computer-implemented method of claim 1 , wherein the machine learning comprises a gradient tree boosting process.
10 . The computer-implemented method of claim 1 , further comprising generating a prediction which controls a motion of a motor vehicle using remaining data in the memory device.
11 . The computer-implemented method of claim 1 , wherein the fitted Q iteration session computes, from a set of four-tuples, an approximation of an optimal stationary policy, and wherein the set of four-tuples comprises a state at time t, an action at time t, a reward at time t, and a discount factor γ.
12 . A computer program product for data reduction in a memory device for machine learning, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
storing, in the memory device, data that has been used for training in a tree-based fitted Q iteration session which learns an action value function with an ensemble of decision trees from the data;
determining, by a processor device, samples to be removed from the data based on a number of samples which belong to leaf nodes of the decision trees having identical positions within different decision trees from the ensemble of decision trees based on sample similarity to obtain determined samples;
removing, from the memory device, the determined samples from the data to reduce an amount of the data based on determined leaf node statistics for sample similarity by using the ensemble of decision trees as a regressor to obtain pruned data; and
learning, by the processor device, a new ensemble of decision trees using the pruned data together with new data.
13 . The computer program product of claim 12 , wherein the samples which belong to a leaf node are removed from the memory device, when a number of the samples which belong to the leaf node exceeds a pruning threshold based on sample similarity.
14 . The computer program product of claim 12 , wherein said storing, determining, removing, and learning steps are repeated recursively to efficiently learn a true action-value function representing the pruned data and the new data.
15 . The computer program product of claim 12 , wherein each of the samples is removed with a probability that is determined responsive to an average number of the samples, where the average number of samples is taken over all leaf nodes which a given sample belongs to.
16 . The computer program product of claim 12 , wherein the number of samples used to determine whether to remove data comprises an average number of samples that belong to a given leaf node.
17 . The computer program product of claim 12 , wherein the number of samples used to determine whether to remove data comprises a minimum number of samples that belong to a given leaf node.
18 . The computer program product of claim 12 , wherein the number of samples used to determine whether to remove data comprises a maximum number of samples that belong to a given leaf node.
19 . The computer program product of claim 12 , wherein the number of samples used to determine whether to remove data comprises a median of samples that belong to a given leaf node.
20 . A computer processing system for data reduction in a memory device for machine learning, comprising:
a memory device configured to store program code;
a processor device operatively coupled to the memory device for running the program code to
store, in the memory device, data that has been used for training in a tree-based fitted Q iteration session which learns an action value function with an ensemble of decision trees from the data;
determine samples to be removed from the data based on a number of samples which belong to leaf nodes of the decision trees having identical positions within different decision trees from the ensemble of decision trees based on sample similarity to obtain determined samples;
remove, from the memory device, the determined samples from the data to reduce an amount of the data based on leaf node statistics for sample similarity by using the ensemble of decision trees as a regressor to obtain pruned data; and
learn a new ensemble of decision trees using the pruned data together with new data.