Statistical credibility metric for online question answerers
View Patent ↗Techniques for statistically estimating a rating or other “figure of merit” for a user are disclosed. According to one such technique, a first quantity of submissions that were submitted by a user is determined. A second quantity of submissions that (a) were submitted by the user and (b) obtained a particular rating from a rating mechanism also is determined. A user rating for the user is determined based at least in part on the first quantity, the second quantity, and a factor that is independent of both the first quantity and the second quantity—such as the probability that an answer submitted by any answerer in a population will obtain the particular rating from the rating mechanism. The influence that the second quantity has on the user rating relative to the influence that the factor has on the user rating may depend at least in part on the first quantity.
1. A computer-implemented method of rating a particular user, the method comprising:
determining a first quantity of submissions that were submitted by a particular user;
determining a second quantity of submissions that (a) were submitted by the particular user and (b) obtained a particular submission rating from a rating mechanism;
determining a particular user rating for the particular user based at least in part on the first quantity, the second quantity, and a factor that is independent of both the first quantity and the second quantity; and
displaying the particular user rating;
wherein, in determining the particular user rating, an influence that the second quantity has on the particular user rating relative to an influence that the factor has on the particular user rating is based at least in part on the first quantity;
wherein determining the particular user rating comprises determining the particular user rating based on a two-dimensional contingency table h(n,n B ) that contains counts of submitters who have answered n questions and had n B of those answers given the particular submission rating by submitters of those questions;
wherein the method is performed by one or more computing devices.
2. The method of claim 1 , wherein, in determining the particular user rating, as the first quantity increases, the influence that the second quantity has on the particular user rating increases and the influence that the factor has on the particular user rating decreases.
3. The method of claim 1 , further comprising determining the particular user rating using a formula
θ
*
=
[
1
-
β
(
n
)
]
(
n
B
n
)
+
β
(
n
)
θ
_
,
wherein
θ
_
is
a
probability of any submission given by any submitter in the population being selected as a best answer to a question, and wherein β(n) is a monotonically decreasing function.
4. A non-transitory computer-readable storage medium that stores instructions which, when executed by one or more processors, cause the one or more processors to perform steps comprising:
determining a first quantity of submissions that were submitted by a particular user;
determining a second quantity of submissions that (a) were submitted by the particular user and (b) obtained a particular submission rating from a rating mechanism;
determining a particular user rating for the particular user based at least in part on the first quantity, the second quantity, and a factor that is independent of both the first quantity and the second quantity; and
displaying the particular user rating;
wherein, in determining the particular user rating, an influence that the second quantity has on the particular user rating relative to an influence that the factor has on the particular user rating is based at least in part on the first quantity;
wherein determining the particular user rating comprises determining the particular user rating based on a two-dimensional contingency table h(n,n B ) that contains counts of submitters who have answered n questions and had n B of those answers given the particular submission rating by submitters of those questions.
5. The non-transitory computer-readable storage medium of claim 4 , wherein, in determining the particular user rating, as the first quantity increases, the influence that the second quantity has on the particular user rating increases and the influence that the factor has on the particular user rating decreases.
6. The non-transitory computer-readable storage medium of claim 5 , wherein said steps further comprise determining the particular user rating using a formula
θ
*
=
[
1
-
β
(
n
)
]
(
n
B
n
)
+
β
(
n
)
θ
_
,
wherein θ is a probability of any submission given by any submitter in the population being selected as a best answer to a question, and wherein β(n) is a monotonically decreasing function.