IP Library Granted Patent US 10,943,186
Granted Patent B2
US 10,943,186 · App. 16/813,268 · Granted Mar 9, 2021

Machine learning model training method and device, and electronic device

Inventor: Long Guo (Hangzhou, CN)
Assignee: ADVANCED NEW TECHNOLOGIES CO., LTD.
G06N20/00G06K9/623G06K9/6256G06K9/6268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,943,186
App. No.
16/813,268
Granted
Mar 9, 2021
Kind
B2
Abstract

A machine learning model training method includes: classifying samples having risk labels in a training sample set as positive samples and classifying samples without risk labels in the training sample set as negative samples; training a risk model with a machine learning method based on the positive samples and the negative samples; obtaining a risk score for each of the negative samples based on the trained risk model; identifying one or more negative samples in the training sample set that have a risk score greater than a preset threshold value; re-classifying the one or more negative samples in the training sample set that have a risk score greater than the preset threshold value as re-classified positive samples to generate an updated training sample set from the training sample set; and re-training the risk model with the machine learning method based on the updated training sample set.

Claims (76)

1. A machine learning model training method, comprising:

classifying samples having risk labels in a training sample set as positive samples and classifying samples without risk labels in the training sample set as negative samples;

training a risk model with a machine learning method based on the positive samples and the negative samples;

obtaining a risk score for each of the negative samples based on the trained risk model;

identifying one or more negative samples in the training sample set that have a risk score greater than a preset threshold value;

re-classifying the one or more negative samples in the training sample set as re-classified positive samples to generate an updated training sample set from the training sample set;

configuring weights for loss functions corresponding to the re-classified positive samples and the positive samples in the training sample set, respectively, wherein a weight of the loss function corresponding to the re-classified positive samples is smaller than a weight of the loss function corresponding to the positive samples; and

re-training the risk model with the machine learning method based on the updated training sample set including the re-classified positive samples, the positive samples in the training sample set, and remaining negative samples in the training sample set,

wherein the risk scores represent probabilities at which a risk occurs, and the method further comprises:

after re-classifying the negative samples that have a risk score greater than the preset threshold value as the re-classified positive samples, configuring the risk scores of the re-classified positive samples as the weights of the loss function corresponding to the re-classified positive samples, and setting the weight of the loss function corresponding to original negative samples to be one.

2. The method of claim 1 , further comprising:

deleting from the updated training sample set the one or more negative samples that have a risk score greater than the preset threshold value; and

re-training the risk model based on the positive samples and remaining negative samples in the training sample set.

3. The method of claim 1 , further comprising:

constructing a prediction sample based on online data; and

obtaining a risk score of the prediction sample based on the re-trained model.

4. The method of claim 1 , wherein the loss functions comprise one of a log-loss function or a hinge-loss function.

5. The method of claim 1 , further comprising:

obtaining a risk score for online transaction data based on the re-trained model;

determining whether the risk score for online transaction data is greater than a risk threshold; and

in response to determining that the risk score for online transaction data is greater than the risk threshold, determining that an online transaction corresponding to the online transaction data is risky.

6. The method of claim 5 , further comprising:

in response to determining that the online transaction corresponding to the online transaction data is risky, terminating the online transaction.

7. The method of claim 5 , further comprising:

in response to determining that the online transaction corresponding to the online transaction data is risky, requesting verification of a user identity for the online transaction before approving the online transaction.

8. An apparatus comprising:

at least one processor; and

a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform operations comprising:

classifying samples having risk labels m a training sample set as positive samples and classifying samples without risk labels in the training sample set as negative samples;

training a risk model with a machine learning method based on the positive samples and the negative samples;

obtaining a risk score for each of the negative samples based on the trained risk model;

identifying one or more negative samples in the training sample set that have a risk score greater than a preset threshold value;

re-classifying the one or more negative samples in the training sample set that have a risk score greater than the preset threshold value as re-classified positive samples to generate an updated training sample set from the training sample set;

configuring weights for loss functions corresponding to the re-classified positive samples and the positive samples in the training sample set, respectively, wherein a weight of the loss function corresponding to the re-classified positive samples is smaller than a weight of the loss function corresponding to the positive samples; and

re-training the risk model with the machine learning method based on the updated training sample set including the re-classified positive samples, the positive samples in the training sample set, and remaining negative samples in the training sample set,

wherein the risk scores represent probabilities at which a risk occurs, and the operations further comprise:

after re-classifying the negative samples that have a risk score greater than the preset threshold value as the re-classified positive samples, configuring the risk scores of the re-classified positive samples as the weights of the loss function corresponding to the re-classified positive samples, and setting the weight of the loss function corresponding to original negative samples to be one.

9. The apparatus of claim 8 , wherein the operations further comprise:

deleting from the updated training sample set the one or more negative samples that have a risk score greater than the preset threshold value; and

re-training the risk model based on the positive samples and remaining negative samples in the training sample set.

10. The apparatus of claim 8 , wherein the operations further comprise:

constructing a prediction sample based on online data; and

obtaining a risk score of the prediction sample based on the re-trained model.

11. The apparatus of claim 8 , wherein the loss functions comprise one of a log-loss function or a hinge-loss function.

12. The apparatus of claim 8 , wherein the operations further comprise:

obtaining a risk score for online transaction data based on the re-trained model;

determining whether the risk score for online transaction data is greater than a risk threshold; and

in response to determining that the risk score for online transaction data is greater than the risk threshold, determining that an online transaction corresponding to the online transaction data is risky.

13. The apparatus of claim 12 , wherein the operations further comprise:

in response to determining that the online transaction corresponding to the online transaction data is risky, terminating the online transaction.

14. The apparatus of claim 12 , wherein the operations further comprise:

in response to determining that the online transaction corresponding to the online transaction data is risky, requesting verification of a user identity for the online transaction before approving the online transaction.

15. A non-transitory computer-readable storage medium storing instructions executable by a processor to cause the processor to perform operations comprising:

classifying samples having risk labels in a training sample set as positive samples and classifying samples without risk labels in the training sample set as negative samples;

training a risk model with a machine learning method based on the positive samples and the negative samples;

obtaining a risk score for each of the negative samples based on the trained risk model;

identifying one or more negative samples in the training sample set that have a risk score greater than a preset threshold value;

re-classifying the one or more negative samples in the training sample set that have a risk score greater than the preset threshold value as re-classified positive samples to generate an updated training sample set from the training sample set; and

configuring weights for loss functions corresponding to the re-classified positive samples and the positive samples in the training sample set, respectively, wherein a weight of the loss function corresponding to the re-classified positive samples is smaller than a weight of the loss function corresponding to the positive samples; and

re-training the risk model with the machine learning method based on the updated training sample set including the re-classified positive samples, the positive samples in the training sample set, and remaining negative samples in the training sample set,

wherein the risk scores represent probabilities at which a risk occurs, and the operations further comprise:

after re-classifying the negative samples that have a risk score greater than the preset threshold value as the re-classified positive samples, configuring the risk scores of the re-classified positive samples as the weights of the loss function corresponding to the re-classified positive samples, and setting the weight of the loss function corresponding to original negative samples to be one.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

deleting from the updated training sample set the one or more negative samples that have a risk score greater than the preset threshold value; and

re-training the risk model based on the positive samples and remaining negative samples in the training sample set.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

constructing a prediction sample based on online data; and

obtaining a risk score of the prediction sample based on the re-trained model.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

obtaining a risk score for online transaction data based on the re-trained model;

determining whether the risk score for online transaction data is greater than a risk threshold; and

in response to determining that the risk score for online transaction data is greater than the risk threshold, determining that an online transaction corresponding to the online transaction data is risky.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

in response to determining that the online transaction corresponding to the online transaction data is risky, terminating the online transaction.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

in response to determining that the online transaction corresponding to the online transaction data is risky, requesting verification of a user identity for the online transaction before approving the online transaction.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 16, 2020
From: ADVANTAGEOUS NEW TECHNOLOGIES CO., LTD.
To: ADVANCED NEW TECHNOLOGIES CO., LTD.
Reel/Frame 053796/0281 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 3, 2020
From: ALIBABA GROUP HOLDING LIMITED
To: ADVANTAGEOUS NEW TECHNOLOGIES CO., LTD.
Reel/Frame 053702/0392 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2020
From: GUO, LONG
To: ALIBABA GROUP HOLDING LIMITED
Reel/Frame 052057/0607 →
Priority Claims (1)
CN 201711174427.3 · Nov 22, 2017 · national
Continuity (2)
Continuation PCTCN2018107499 · Sep 26, 2018
Related Publication 20200210899A1 · Jul 2, 2020
Cited By (1)
US 12,387,820