IP Library Granted Patent US 10,885,593
Granted Patent B2
US 10,885,593 · App. 14/735,022 · Granted Jan 5, 2021

Hybrid classification system

Inventors: Fan Yang (Redwood City, CA); Craig Martell (San Juan Bautista, CA); Dan Shacham (Cupertino, CA)
Assignee: Microsoft Technology Licensing, LLC
G06Q50/01G06N20/00G06Q30/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,885,593
App. No.
14/735,022
Granted
Jan 5, 2021
Kind
B2
Abstract

Hybrid classification system and method are described. The method commences when an input detector detects a raw input string that represents a value of a category in a member profile maintained by the on-line social network. The machine learning classifier derives a standardized value corresponding to the raw input string. The trigger module provides the raw input string to the correcting filter. The correcting filter determines a corrected standardized value corresponding to the raw input string based on the raw input string and a corrective rule. The label module then identifies the member profile as associate with the corrected standardized value.

Claims (58)

1. A computer-implemented method comprising:

detecting a raw input string for a category in a member profile maintained by an on-line network for a first member, the category having a plurality of predefined standardized values;

executing a machine-learning classifier to select one of the predefined standardized values corresponding to the raw input string, the machine-learning classifier being obtained by training a machine-learning tool that estimates relationships among variables, the training using training data having raw strings and corresponding standardized values for the raw strings;

displaying the selected standardized value to the first member;

from the first member, receiving a correction for the selected standardized value, the correction including a corrected standardized value from the plurality of predefined standardized values;

including into a correcting filter the received correction as a corrective rule that matches the raw input string for the category to the corrected standardized value for the category;

receiving the raw input string from a second member of the on-line network;

executing the machine-learning classifier based on the received raw input string from the second member;

providing the received raw input string to the correcting filter to obtain, based on the raw input string and the corrective rule, the corrected standardized value corresponding to the raw input string, using at least one processor;

determining that the standardized value that was derived by the executing of the machine-learning classifier is different from the corrected standardized value obtained by the correcting filter; and

identifying a member profile of the second member as associated with the corrected standardized value in response to determining that the standardized value that was derived by the executing of the machine-learning classifier is different from the corrected standardized value obtained by the correcting filter.

2. The method of claim 1 , wherein the corrective rule includes an identification of the member profile.

3. The method of claim 1 , wherein the corrective rule references one or more professional attributes maintained by the on-line network.

4. The method of claim 1 , comprising:

determining that the corrected standardized value is the same as the standardized value obtained by the machine-learning classifier; and

removing the corrective rule from a set of rules used by the correcting filter in response to the determining.

5. The method of claim 1 , comprising:

obtaining corrective data; and

generating the corrective rule based on the corrective data.

6. The method of claim 5 , comprising:

adding the corrective data to the training data used for re-training of the machine-learning classifier.

7. The method of claim 1 , comprising determining the corrected standardized value in response to a potential error indicator.

8. The method of claim 7 , wherein the potential error indicator is an identification of the member profile.

9. The method of claim 1 , wherein the category represents one of a professional title, a company, or a school.

10. A computer-implemented system comprising one or more processors and a non-transitory computer readable storage medium comprising instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:

detecting a raw input string for a category in a member profile maintained by an on-line network for a first member, the category having a plurality of predefined standardized values;

executing a machine-learning classifier to select one of the predefined standardized values corresponding to the raw input string, the machine-learning classifier being obtained by training a machine-learning tool that estimates relationships among variables, the training using training data having raw strings and corresponding standardized values for the raw strings;

displaying the selected standardized value to the first member;

from the first member, receiving a correction for the selected standardized value, the correction including a corrected standardized value from the plurality of predefined standardized values;

including into a correcting filter the received correction as a corrective rule that matches the raw input string for the category to the corrected standardized value for the category;

receiving the raw input string from a second member of the on-line network;

executing the machine-learning classifier based on the received raw input string from the second member;

providing the received raw input string to the correcting filter to obtain, based on the raw input string and the corrective rule, the corrected standardized value corresponding to the raw input string, using at least one processor;

determining that the standardized value that was derived by the executing of the machine-learning classifier is different from the corrected standardized value obtained by the correcting filter; and

identifying a member profile of the second member as associated with the corrected standardized value in response to determining that the standardized value that was derived by the executing of the machine-learning classifier is different from the corrected standardized value obtained by the correcting filter.

11. The system of claim 10 , wherein the corrective rule includes an identification of the member profile.

12. The system of claim 10 , wherein the corrective rule references one or more professional attributes maintained by the on-line network.

13. The system of claim 10 , comprising:

determining that the corrected standardized value is the same as the standardized value obtained by the machine-learning classifier; and

removing the corrective rule from a set of rules used by the correcting filter in response to the determining.

14. The system of claim 10 , comprising:

accessing corrective data; and

generating the corrective rule based on the corrective data.

15. The system of claim 14 , comprising:

adding the corrective data to the training data used for re-training of the machine-learning classifier.

16. The system of claim 10 , comprising determining the corrected standardized value in response to a potential error indicator.

17. The system of claim 16 , wherein the potential error indicator is an identification of the member profile.

18. A machine-readable non-transitory storage medium having instruction data executable by a machine to cause the machine to perform operations comprising:

detecting a raw input string for a category in a member profile maintained by an on-line network for a first member, the category having a plurality of predefined standardized values;

executing a machine-learning classifier to select one of the predefined standardized values corresponding to the raw input string, the machine-learning classifier being obtained by training a machine-learning tool that estimates relationships among variables, the training using training data having raw strings and corresponding standardized values for the raw strings;

displaying the selected standardized value to the first member;

from the first member, receiving a correction for the selected standardized value, the correction including a corrected standardized value from the plurality of predefined standardized values;

including into a correcting filter the received correction as a corrective rule that matches the raw input string for the category to the corrected standardized value for the category;

receiving the raw input string from a second member of the on-line network;

executing the machine-learning classifier based on the received raw input string from the second member;

providing the received raw input string to the correcting filter to obtain, based on the raw input string and the corrective rule, the corrected standardized value corresponding to the raw input string, using at least one processor;

determining that the standardized value that was derived by the executing of the machine-learning classifier is different from the corrected standardized value obtained by the correcting filter; and

identifying a member profile of the second member as associated with the corrected standardized value in response to determining that the standardized value that was derived by the executing of the machine-learning classifier is different from the corrected standardized value obtained by the correcting filter.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2017
From: LINKEDIN CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 044746/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2015
From: YANG, FAN; MARTELL, CRAIG; SHACHAM, DAN
To: LINKEDIN CORPORATION
Reel/Frame 035880/0095 →