Secondary profiles with credibility scores
A system, apparatus, and method are provided for implementing secondary profiles for members of an online application or service. Each member has a corresponding primary profile populated by the member, and a secondary profile populated with information from data sources other than the member. Each fact or entry in the secondary (or inferred) profile is accompanied by a confidence score reflecting confidence in the source of the fact, confidence that the fact is correctly associated with this member, and/or other factors. A given fact may be obtained or extracted from multiple sources, with each copy or version assigned a separate confidence score. In response to a request to identify members having a particular attribute, in addition to identifying members that have the attribute in their primary profiles, members having the attribute in their secondary profiles may be identified if the corresponding confidence scores are greater than a threshold.
1. A method, comprising:
populating primary profiles of members of a user community with information provided by the corresponding members;
obtaining a plurality of facts from multiple data sources other than the user community;
for each data source, calculating a corresponding credibility score by:
(a) assigning an initial weight to each of the multiple data sources, including the data source;
(b) assigning an initial agreement factor to each of the multiple data sources, including the data source;
(c) for each fact of a first type, incrementing the agreement factor of the data source if the fact was obtained from the data source and at least one other data source among the multiple data sources; and
(d) adjusting the weight of the data source, based on the agreement factors of other data sources among the multiple data sources with which the data source has agreed;
wherein the credibility score corresponding to the data source is derived from the adjusted weight of the data source; and
wherein the data source is deemed to have agreed with a given other data source if at least one fact was extracted from the data source and the other data source; and
for each fact, with one or more computers:
for each of one or more members, generating a confidence score representing a confidence that the fact is associated with the member;
adjusting the generated confidence scores based on the credibility score of the data source from which the fact was extracted; and
for each member having a confidence score above a threshold, adding the fact to a secondary profile of the member;
wherein each member's secondary profile is populated without requiring action by the corresponding member.
2. The method of claim 1 , wherein adjusting the weight of the data source comprises:
(d1) for each other data source with which the data source agreed, dividing the weight of the other data source by the agreement factor of the other data source to produce an associated quotient; and
(d2) summing the associated quotients to produce a sum;
wherein the adjusted weight of the first data source is based on the sum.
3. The method of claim 2 , wherein adjusting the weight of the data source further comprises:
(d3) multiplying the sum by a damping factor to yield a product;
wherein the adjusted weight of the first data source is based on the product.
4. The method of claim 2 , wherein adjusting the weight of the data source further comprises:
(d3) multiplying the sum by a damping factor to yield a product;
(d4) subtracting the damping factor from one to yield a difference; and
(d5) subtracting the product from the difference to yield the adjusted weight of the data source.
5. The method of claim 1 , further comprising:
repeating said (a) through (d) for one or more other fact types;
wherein each type of fact corresponds to one attribute of a secondary profile.
6. The method of claim 1 , wherein:
the initial weight is 1; and
the initial agreement factor is 1.
7. The method of claim 1 , wherein adjusting the confidence score comprises:
when the credibility score is greater than zero, increasing the confidence score proportional to the credibility score.
8. An apparatus, comprising:
one or more hardware processors; and
memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
populate primary profiles of members of a user community with information provided by the corresponding members;
obtain a plurality of facts from multiple data sources other than the user community;
for each data source, calculate a corresponding credibility score by:
(a) assigning an initial weight to each of the multiple data sources, including the data source;
(b) assigning an initial agreement factor to each of the multiple data sources, including the data source;
(c) for each fact of a first type, incrementing the agreement factor of the data source if the fact was obtained from the data source and at least one other data source among the multiple data sources; and
(d) adjusting the weight of the data source, based on the agreement factors of other data sources among the multiple data sources with which the data source has agreed;
wherein the credibility score corresponding to the data source is derived from the adjusted weight of the data source; and
wherein the data source is deemed to have agreed with a given other data source if at least one fact was extracted from the data source and the other data source; and
for each fact:
for each of one or more members, generate a confidence score representing a confidence that the fact is associated with the member;
adjust the generated confidence scores based on the credibility score of the data source from which the fact was extracted; and
for each member having a confidence score above a threshold, add the fact to a secondary profile of the member;
wherein each member's secondary profile is populated without requiring action by the corresponding member.
9. The apparatus of claim 8 , wherein adjusting the weight of the data source comprises:
(d1) for each other data source with which the data source agreed, dividing the weight of the other data source by the agreement factor of the other data source to produce an associated quotient; and
(d2) summing the associated quotients to produce a sum;
wherein the adjusted weight of the data source is based on the sum.
10. The apparatus of claim 9 , wherein adjusting the weight of the data source further comprises:
(d3) multiplying the sum by a damping factor to yield a product;
wherein the adjusted weight of the data source is based on the product.
11. The apparatus of claim 9 , wherein adjusting the weight of the data source further comprises:
(d3) multiplying the sum by a damping factor to yield a product;
(d4) subtracting the damping factor from one to yield a difference; and
(d5) subtracting the product from the difference to yield the adjusted weight of the data source.
12. The apparatus of claim 8 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
repeat said (a) through (d) for one or more other fact types;
wherein each type of fact corresponds to one attribute of a secondary profile.
13. The apparatus of claim 8 , wherein:
the initial weight is 1; and
the initial agreement factor is 1.
14. The apparatus of claim 8 , wherein adjusting the confidence score comprises:
when the credibility score is greater than zero, increasing the confidence score proportional to the credibility score.
15. A system, comprising:
a profile module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to populate primary profiles of members of a user community with information provided by the corresponding members;
an extraction module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to obtain a plurality of facts from multiple data sources other than the user community;
a correlation module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to, for each fact:
for each of one or more members, generate a confidence score representing a confidence that the fact is associated with the member; and
for each member having a confidence score above a threshold, add the fact to a secondary profile of the member; and
a credibility module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to:
for each data source, calculate a corresponding credibility score by:
(a) assigning an initial weight to each of the multiple data sources, including the data source;
(b) assigning an initial agreement factor to each of the multiple data sources, including the data source;
(c) for each fact of a first type, incrementing the agreement factor of the data source if the fact was obtained from the data source and at least one other data source among the multiple data sources; and
(d) adjusting the weight of the data source, based on the agreement factors of other data sources among the multiple data sources with which the data source has agreed;
wherein the credibility score corresponding to the data source is derived from the adjusted weight of the data source; and
wherein the data source is deemed to have agreed with a given other data source if at least one fact was extracted from the data source and the other data source; and
for each fact, adjust the generated confidence scores based on the credibility score of the data source from which the fact was extracted, prior to said adding;
wherein each member's secondary profile is populated without requiring action by the corresponding member.
16. The system of claim 15 , wherein adjusting the weight of the data source comprises:
(d1) for each other data source with which the data source agreed, dividing the weight of the other data source by the agreement factor of the other data source to produce an associated quotient; and
(d2) summing the associated quotients to produce a sum;
wherein the adjusted weight of the data source is based on the sum.
17. The system of claim 16 , wherein adjusting the weight of the data source further comprises:
(d3) multiplying the sum by a damping factor to yield a product;
(d4) subtracting the damping factor from one to yield a difference; and
(d5) subtracting the product from the difference to yield the adjusted weight of the data source.