Method And System For Scoring Credibility Of Information Sources
A method for classifying information sources and content based on credibility, reliability, or trust. A content item describing an event is retrieved from an information provider and parsed for links. Each link is evaluated and attributed a sentiment score. The same event is identified in a set of know sources and an event score is calculated based on the credibility of each of the known sources. Finally, the content item is ranked based on the event and sentiment scores.
1 . A computer-implemented method for ranking information stored on a computer readable medium; the method comprising:
extracting a content item describing an event from an information source;
parsing by a parsing module the content item for a plurality of source links;
attributing by a content analysis module a sentiment score to each source link; wherein said sentiment score is indicative of the relative credibility of each of said source links;
scoring by a scoring module said information source based on said source links and on said sentiment score
ranking said content item based on a score associated with said information source.
2 . The method according to claim 1 , wherein said scoring comprises
calculating r from equation (1):
r=α*T +(1−α)* d
where d is a non-zero static score distribution vector, T is a transition matrix, and α is a predetermined constant; and, wherein each term in said transition matrix is modified by a non-zero sentiment score.
3 . The method of claim 1 , further comprising storing on a score database implemented on a computer readable medium said score for said information source.
4 . The method according to claim 3 , further comprising prior to said parsing step determining whether said information source has an associated score in said score database, and upon determining that said information source has an associated score in said score database, retrieving said score and returning to said extracting step.
5 . The method according to claim 1 , further comprising
identifying an event from each source in a set of information sources;
calculating an event score for the content item describing said event based on a credibility score for each of the known sources in said set of information sources; and
combining said score for a respective information source with said credibility score to determine a cumulative event score.
6 . The method according to claim 5 , wherein each of said sources in said set of information sources is classified as one of a known good source, a known bad source and an unknown reliability source, and wherein said calculating an event score is biased towards sources identified as known good sources.
7 . The method according to claim 6 , wherein said event score is calculated as:
EventScore
=
{
A
if
event
contains
a
KG
,
A
m
if
event
contains
b
unknown
and
no
KG
nor
KB
A
n
if
event
has
less
than
b
unknowns
and
no
KG
nor
KB
A
p
if
event
has
no
KG
and
includes
a
KB
where A, m, n and p are parameters selected such that A/p<A/n<A/m<A; KG is a known good source; KB is a known bad source.
8 . The method according to claim 5 , wherein said identified events are compared to identify contradictions, and said calculating an event score includes biasing events from known good sources to resolve said contradictions.
9 . The method according to claim 5 , wherein said combining comprises calculating an event ranking representative of said event being reliable.
10 . The method according to claim 9 , wherein calculating said event ranking is calculated as:
EventRank= a *EventScore+ b *ΣLinkScore+ c *LinksToEvent
where a, b and c are weighted coefficients and LinksToEven is calculated as:
LinksToEvent
=
∑
i
=
1
n
LinkScore
(
i
)
×
Sent
where n is the number of sources and LinkScore is the score of said information source, and Sent is the sentiment score.
11 . The method according to claim 10 , further comprising calculating an accumulated event rank for the information provider from a plurality of event ranks by:
AccumulatedEvent
Rank
=
1
N
i
∑
j
=
1
N
i
EventRank
(
j
)
where EventRank(j) is a plurality of event scores for a plurality of content items and N i is a total number of content items of the information provider in the known source database.
12 . A computer-implemented method for ranking information stored on a computer readable medium; the method comprising:
identifying an event from each source in a set of information sources;
calculating an event score for a content item describing said event based on a credibility score for each of the known sources in said set of information sources; and
combining a score for a respective information source with said credibility score to determine a cumulative event score.
13 . The method according to claim 12 , wherein each of said sources in said set of information sources is classified as one of a known good source, a known bad source and an unknown reliability source, and wherein said calculating an event score is biased towards sources identified as known good sources.
14 . The method according to claim 13 , wherein said event score is calculated as:
EventScore
=
{
A
if
event
contains
a
KG
,
A
m
if
event
contains
b
unknown
and
no
KG
nor
KB
A
n
if
event
has
less
than
b
unknowns
and
no
KG
nor
KB
A
p
if
event
has
no
KG
and
includes
a
KB
where A, m, n and p are parameters selected such that A/p<A/n<A/m<A; KG is a known good source; KB is a known bad source.
15 . The method according to claim 12 , wherein said identified events are compared to identify contradictions, and said calculating an event score includes biasing events from known good sources to resolve said contradictions.
16 . The method according to claim 12 , wherein said combining comprises calculating an event ranking representative of said event being reliable.
17 . The method according to claim 16 , wherein calculating said event ranking is calculated as:
EventRank= a *EventScore+ b *ΣLinkScore+ c *LinksToEvent
where a, b and c are weighted coefficients and LinksToEven is calculated as:
LinksToEvent
=
∑
i
=
1
n
LinkScore
(
i
)
×
Sent
where n is the number of sources and LinkScore is the score of said information source, and Sent is the sentiment score.
18 . The method according to claim 17 , further comprising calculating an accumulated event rank for the information provider from a plurality of event ranks by:
AccumulatedEvent
Rank
=
1
N
i
∑
j
=
1
N
i
EventRank
(
j
)
where EventRank(j) is a plurality of event scores for a plurality of content items and N i is a total number of content items of the information provider in the known source database.
19 . The method according to claim 23 , wherein said score for a respective information source is determined by:
extracting a content item describing an event from an information source;
parsing by a parsing module the content item for a plurality of source links;
attributing by a content analysis module a sentiment score to each source link; wherein said sentiment score is indicative of the relative credibility of each of said source links;
scoring by a scoring module said information source based on said source links and on said sentiment score; and
ranking said content item based on a score associated with said information source.
20 . The method according to claim 19 , wherein said scoring comprises
calculating r from equation (1):
r=α*T +(1−α)* d
where d is a non-zero static score distribution vector, T is a transition matrix, and α is a predetermined constant; and, wherein each term in said transition matrix is modified by a non-zero sentiment score.
21 . The method of claim 19 , further comprising storing on a score database implemented on a computer readable medium said score for said information source.
22 . The method according to claim 21 , further comprising prior to said parsing step determining whether said information source has an associated score in said score database, and upon determining that said information source has an associated score in said score database, retrieving said score and returning to said extracting step.
23 . The method according to claim 17 , whereby the sentiment score is derived using a sentiment scorer that was created at least in part using:
a training set of known true and known false events; and
known links and the associated text of said known links to content that specifies the known true and known false events.
24 . The method of claim 1 , wherein the ranking score is used for any one or more of insurance underwriting, assessing suspected fraudulent activity, credit decisioning, securities trading, insurance underwriting.
25 . The method of claim 12 , wherein the ranking score is used for any one or more of insurance underwriting, assessing suspected fraudulent activity, credit decisioning, securities trading, insurance underwriting.