Method and system for training model by using training data

1. A method for training a model by using training data, wherein the training data comprises a plurality of samples, each sample comprises N features, and features in the plurality of samples form N feature columns, and the method comprises:

determining an importance value of each of the N feature columns;

determining whether the importance value of each of the N feature columns satisfies a threshold condition;

performing, among the N feature columns, a dimension reduction on M feature columns to generate P feature columns in response to a determination that importance values of the M feature columns do not satisfy the threshold condition, wherein M<N and P<M;

merging (N−M) feature columns having importance values that satisfy the threshold condition and the generated P feature columns to obtain (N−M+P) feature columns; and

training the model based on the training data including the (N−M+P) feature columns.

2. The method according to claim 1 , wherein performing, among the N feature columns, the dimension reduction on M feature columns to generate P feature columns in response to the determination that the importance values of the M feature columns do not satisfy the threshold condition further comprises:

in response to the determination that the importance values of M feature columns in the N feature columns do not satisfy the threshold condition, performing dimension reduction processing on the M feature columns by using a minwise Hashing algorithm to generate P feature columns, wherein P=k×2 b , and k and b are specified by the minwise Hashing algorithm.

3. The method according to claim 1 , wherein the importance values of the (N−M) feature columns are greater than or equal to a threshold.

4. The method according to claim 1 , wherein the training data further comprises a tag value corresponding to each sample, and determining an importance value of each of the N feature columns comprises:

determining at least one of an information value and an information gain of each of the N feature columns by using the tag value and features in each of the N feature columns, and

using the at least one of the information value and the information gain as the importance value.

5. The method according to claim 4 , wherein a threshold corresponding to the information value is a first threshold, and a threshold corresponding to the information gain is a second threshold.

6. The method according to claim 1 , wherein the model is a classification algorithm model or a regression algorithm model.

7. A system for training a model by using training data, wherein the training data comprises a plurality of samples, each sample comprises N features, and features in the samples form N feature columns, and the system comprises:

a memory storing a set of instructions; and

one or more processors configured to execute the set of instructions to cause the system to perform:

determining an importance value of each of the N feature columns;

determining whether the importance value of each feature column satisfies a threshold condition;

performing, among the N feature columns, a dimension reduction on M feature columns to generate P feature columns in response to a determination that importance values of the M feature columns in the N feature columns do not satisfy the threshold condition, wherein M<N and P<M;

merging (N−M) feature columns having importance values that satisfy the threshold condition and the generated P feature columns to obtain (N−M+P) feature columns; and

training the model based on the training data comprising the (N−M+P) feature columns.

8. The system according to claim 7 , wherein the one or more processors are configured to execute the set of instructions to cause the system to further perform:

performing the dimension reduction on the M feature columns by using a minwise Hashing algorithm to generate the P feature columns, wherein P=k×2 b , and k and b are specified by the minwise Hashing algorithm.

9. The system according to claim 7 , wherein the training data further comprises a tag value corresponding to each sample, and the one or more processors are configured to execute the set of instructions to cause the system to further perform:

determining at least one of an information value and an information gain of each of the N feature columns by using the tag value and features in each of the N feature columns, and

using the at least one of the information value and the information gain as the importance value.

10. The system according to claim 9 , wherein a threshold corresponding to the information value is a first threshold, and a threshold corresponding to the information gain is a second threshold.

11. The system according to claim 7 , wherein the model is a classification algorithm model or a regression algorithm model.

12. The system according to claim 7 , wherein the importance values of the (N−M) feature columns are greater than or equal to a threshold.

13. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computer system to cause the computer system to perform a method for training a model by using training data, wherein the training data comprises a plurality of samples, each sample comprises N features, and features in the plurality of samples form N feature columns, and the method comprises:

determining an importance value of each of the N feature columns;

determining whether the importance value of each of the N feature columns satisfies a threshold condition;

performing, among the N feature columns, a dimension reduction on M feature columns to generate P feature columns in response to a determination that importance values of the M feature columns do not satisfy the threshold condition, wherein M<N and P<M;

merging (N−M) feature columns having importance values that satisfy the threshold condition and the generated P feature columns to obtain (N−M+P) feature columns; and

training the model based on the training data including the (N−M+P) feature columns.

14. The non-transitory computer readable medium according to claim 13 , wherein performing, among the N feature columns, the dimension reduction on M feature columns to generate P feature columns in response to the determination that the importance values of the M feature columns do not satisfy the threshold condition further comprises:

in response to the determination that the importance values of M feature columns in the N feature columns do not satisfy the threshold condition, performing dimension reduction processing on the M feature columns by using a minwise Hashing algorithm to generate P feature columns, wherein P=k×2 b , and k and b are specified by the minwise Hashing algorithm.

15. The non-transitory computer readable medium according to claim 13 , wherein the importance values of the (N-M) feature columns are greater than or equal to a threshold.

16. The non-transitory computer readable medium according to claim 13 , wherein the training data further comprises a tag value corresponding to each sample, and determining an importance value of each of the N feature columns comprises:

determining at least one of an information value and an information gain of each of the N feature columns by using the tag value and features in each of the N feature columns, and

using the at least one of the information value and the information gain as the importance value.

17. The non-transitory computer readable medium according to claim 16 , wherein a threshold corresponding to the information value is a first threshold, and a threshold corresponding to the information gain is a second threshold.

18. The non-transitory computer readable medium according to claim 13 , wherein the model is a classification algorithm model or a regression algorithm model.