Adaptive experimentation method and system
A computer-implemented adaptive experimentation method and system is described that automatically selects and executes information gathering actions. The adaptive experimentation method and system integrates value of information considerations, experimental design, and inferences from experimental results. The experimental results may include behaviors of users of a computer-based system. The process enables an automatic, adaptive process for attaining additional information and applying the attained information in making subsequent experiment decisions.
1. A computer-implemented experimentation method, comprising:
selecting a first experiment, wherein the selecting of the first experiment is based, at least in part, on an expected value of information of the experiment;
performing the first experiment, wherein the first experiment is executed on a processor-based computing device;
inferring automatically a user preference from a plurality of user behaviors that occur after execution of the first experiment; and
selecting a second experiment based, at least in part, on the inferred preference.
2. The method of claim 1 , further comprising:
generating a recommendation.
3. The method of claim 1 , further comprising:
generating a response to a computer-implemented search request.
4. The method of claim 1 , further comprising:
performing a computer-implemented information retrieval.
5. The method of claim 1 , further comprising:
modifying a computer-based structural element.
6. The method of claim 1 , further comprising:
applying information about the intrinsic characteristics of an item of content.
7. The method of claim 1 , further comprising:
applying a statistical learning algorithm.
8. The method of claim 1 , further comprising:
applying an experimental design algorithm.
9. The method of claim 1 , further comprising:
accessing an expected cost of the first experiment.
10. The method of claim 1 , further comprising:
determining an expected net value of information.
11. An adaptive experimentation system, comprising:
an information gathering function executed on a processor-based computing device that conducts a first experiment;
a function that infers a user preference from a plurality of user behaviors occurring after the first experiment is conducted; and
an experimental design function, wherein the experimental design function automatically selects a second online experiment to perform based, at least in part, the inferred user preference and an expected value of information.
12. The system of claim 11 , further comprising:
a recommendation generating function.
13. The system of claim 11 , further comprising:
a function that modifies a computer-based structural element.
14. The system of claim 11 , further comprising:
a statistical learning function.
15. The system of claim 11 , further comprising:
an experimental design function.
16. The system of claim 11 , further comprising:
a function that applies an intrinsic characteristic of an item of content.
17. The system of claim 11 , further comprising:
an expected cost of the second experiment.
18. An adaptive decision method, comprising:
simulating on a processor-based computing device an experimental infrastructure, wherein the simulating comprises applying a plurality of probabilities;
determining a value of information that is expected to be generated by an implementation of the experimental infrastructure; based, at least in part, on the simulation of the experimental infrastructure; and
generating an expected value of the experimental infrastructure based on the expected value of information and an expected cost of the experimental infrastructure.
19. The method of claim 18 , further comprising:
simulating the experimental infrastructure, wherein the simulating includes simulating an instrument.
20. The method of claim 18 , further comprising:
simulating the experimental infrastructure, wherein the simulating includes applying a digitized knowledge base.