Determining causal models for controlling environments
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining causal models for controlling environments. One of the methods includes repeatedly selecting, by a control system for the environment, control settings for the environment based on internal parameters of the control system, wherein: at least some of the control settings for the environment are selected based on a causal model, and the internal parameters include a first set of internal parameters that define a number of previously received performance metric values that are used to generate the causal model for a particular controllable element; obtaining, for each selected control setting, a performance metric value; determining that generating the causal model for the at particular controllable element would result in higher system performance; and adjusting, based on the determining, the first set of internal parameters.
1. A method of controlling an environment by selecting control settings that include a respective setting for each of a plurality of controllable elements of the environment, the method comprising repeatedly performing the following:
repeatedly selecting, by a control system for the environment, control settings for the environment based on internal parameters of the control system, wherein:
at least some of the control settings for the environment are selected based on a causal model that identifies, for each controllable element, causal relationships between possible settings for the controllable element and a performance metric that measures a performance of the control system in controlling the environment, and
the internal parameters include a first set of internal parameters that define a number of previously received performance metric values that are used to generate the causal model for a particular one the controllable elements;
obtaining, for each selected control setting, a performance metric value;
determining, based on the measures of the performance metric values, that generating the causal model for the particular controllable element using a different number of previously received performance metric values would result in higher system performance; and
adjusting, based on the determining, the first set of internal parameters.
2. The method of claim 1 , wherein the causal model is a causal model for a particular cluster of one or more clusters of procedural instances, and wherein each of the procedural instances are segments of the environment for which control settings are selected.
3. The method of claim 1 , wherein the first set of internal parameters includes a window value that specifies the number of previous instances.
4. The method of claim 3 , wherein the causal model includes a respective distribution around an impact measurement for each possible setting for the controllable element and wherein determining that generating the causal model for the particular controllable element using a different number of previously received performance metric values would result in higher system performance comprises:
updating the causal model based on the measures of the performance metric values; and
determining that the impact measurements for the possible settings for the particular element in the updated causal model are not normally distributed.
5. The method of claim 4 , wherein adjusting, based on the determining, the first set of internal parameters comprises:
adjusting the window value to specify a number of previously received instances for which the impact measurements for the possible settings for the particular element in the updated causal model would be normally distributed.
6. The method of claim 1 , wherein the first set of internal parameters specifies a range of possible values for a window value that specifies the number of previously received instances, and wherein the system samples window values from the range of possible values to compute the causal model.
7. The method of claim 6 , wherein:
some of the control settings are selected based on baseline probabilities for the possible settings that are independent of the causal model; and
the method further comprises:
maintaining a second causal model that measures causal effects between possible values of the window value and a difference in system performance between (i) control settings selected using the causal model when the causal model is generated based on the possible value of the window value and (ii) control settings selected based on baseline probabilities.
8. The method of claim 7 , wherein determining that generating the causal model for the particular controllable element using a different number of previously received performance metric values would result in higher system performance comprises:
determining that the causal effects identified in the second causal model between possible values of the window value have changed once the second causal model has been updated based on the measures of performance values.
9. The method of claim 8 , wherein adjusting, based on the determining, the first set of internal parameters comprises:
mapping the causal effects identified in the updated second causal model to updated probabilities for the possible window values; and
using the updated probabilities to sample window values.
10. The method of claim 7 , wherein determining that generating the causal model for the particular controllable element using a different number of previously received performance metric values would result in higher system performance comprises:
determining that a value outside of the range of window values is likely to improve system performance.
11. The method of claim 10 , wherein adjusting, based on the determining, the first set of internal parameters comprises:
adjusting the range of possible window values.
12. The method of claim 10 , wherein determining that a value outside of the range of window values is likely to improve system performance comprises:
determining a measure of stability of the causal effects in the causal model over time.
13. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform the operations of the method of claim 1 .
14. One or more computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform the operations of the method of claim 1 .