Adaptive confidence calibration for real-time swarm intelligence systems
Systems and methods are for enabling a group of individuals, each using an individual computing device, to collaboratively answer questions or otherwise express a collaborative will/intent in real-time as a unified intelligence. The collaboration system comprises a plurality of computing devices, each of the devices being used by an individual user, each of the computing devices enabling its user to contribute to the emerging real-time group-wise intent. A collaboration server is disclosed that communicates remotely to the plurality of individual computing devices. Herein, a variety of inventive methods are disclosed for interfacing users and calibrating for their variable confidence in a real-time synchronized group-wise experience, and for deriving a convergent group intent from the collective user input.
1. A method for determining an adaptive confidence calibration weighting factor associated with a user included in a group of users, wherein each user in the group is associated with a computing device in communication with a central collaboration server, wherein the adaptive confidence calibration weighting factor is used to weight input of the user during a real-time collaborative session, comprising the steps of:
displaying, to the user by a computing device configured for the collaborative session, a plurality of questions;
inputting, to the computing device by the user, a user answer for each question;
inputting, to the computing device by the user, a user prediction value corresponding to the percentage of questions the user predicts he answered correctly;
inputting, to the computer device by the user, a group prediction value corresponding to an average percentage of questions the user predicts the group answered correctly;
determining a user percentage score for each user in the group of users, the user percentage score for each user indicating a percentage of the plurality of questions that the user answered correctly;
determining a group percentage score for the group of users, the group percentage score indicating an average percentage of questions answered correctly across the group of users;
calculating an adaptive confidence weighting factor for at least one user based on at least one of comparing the user prediction value input by that user with the user percentage score for that user, and comparing the group prediction value input by that user to the group percentage score for the group of users; and
enabling the group to collaboratively select an answer to a question, wherein the adaptive confidence weighting factor associated with at least one user is used to scale a relative impact of that user on the collaborative session with respect to the impact of other users.
2. The method of claim 1 , wherein:
the adaptive confidence calibration weighting factor is a value between 0 and 1.
3. The method of claim 1 , wherein:
the calculating of the adaptive confidence calibration weighting factor further comprising calculating a self-assessment accuracy value based on a difference between the user prediction value and the user percentage score.
4. The method of claim 1 , wherein:
the calculating of the adaptive confidence calibration weighting factor further comprising calculating a group-assessment accuracy value based on a difference between the group prediction value and the group percentage score.
5. The method of claim 1 , wherein:
at least one of the plurality of questions asks for the prediction of the outcome of a sporting event.
6. The method of claim 1 , wherein:
at least one of the questions asks the user to rate his or her confidence in a prediction.
7. The method of claim 1 , wherein:
at least one of the questions asks the user to rate his or her knowledge of a topic.
8. The method of claim 1 , wherein:
at least one of the questions asks the user to predict what percentage of the other participants gave the same answer as that user did on a different one of the questions.
9. The method of claim 1 , wherein:
the user answers are input to the computing device using graphical controls.
10. The method of claim 1 , wherein:
at least one of the questions asks the user to assess a difficulty of a different one of the questions.
11. The method of claim 1 , wherein the calculating of the adaptive confidence weighting factor is performed by the central collaboration server for a given user and communicated to the computing device associated with that user.
12. A system comprising a computing device and a central collaboration server, configured for determining an adaptive confidence calibration weighting factor having a value between 0 and 1 and associated with a user included in a group of users, wherein each user in the group is associated with a computing device in communication with the central collaboration server, wherein the adaptive confidence calibration weighting factor is used to weight input of the user during a real-time collaborative session, wherein the system is configured to:
display, to each user by the computing device configured for the collaborative session, a plurality of questions;
for each user, receive input of a user answer for each question;
for each user, receive input of a user prediction value indicating to the percentage of the plurality of questions the user predicts he answered correctly;
for each user, receive input of a group prediction value corresponding to an average percentage of questions the user predicts the group answered correctly;
determine a user percentage score for each user in the group of users, the user percentage score for each user indicating a percentage of the plurality of questions answered correctly by the user;
determine a group percentage score for the group of users, the group percentage score indicating an average percentage of questions answered correctly across the group of users;
calculate the adaptive confidence calibration weighting factor for each user based on at least one of comparing the user prediction value to the user percentage score and comparing the group prediction value to the group percentage score; and
enable the group to collaboratively select an answer to a question, wherein the adaptive confidence weighting factor associated with at least one user is used to scale a relative impact of that user on the collaborative session with respect to the impact of other users.