Feature and case importance and confidence for imputation in computer-based reasoning systems

1. A method comprising:

performing the following until there are no more cases in a computer-based reasoning model with missing fields for which imputation is desired:

determining which cases have fields to impute in the computer-based reasoning model;

determining imputation order information for the cases that have data to impute in the computer-based reasoning model based at least in part on numbers of features that need imputed data for each of the cases;

determining for which one or more cases with missing fields to impute data for the missing fields based on the imputation order information, wherein determining for which one or more cases with missing fields to impute data for the missing fields based on the imputation order information comprises:

determining which one or more particular cases of the cases has a lowest number of features that need imputed data; and

determining to impute data for the missing fields for the one or more particular cases that have the lowest number of features that need imputed data among the one or more particular cases;

for each of the determined one or more particular cases with the lowest number of missing fields to impute:

determining imputed data for a missing field of the missing fields based on the case, and an imputation model, and the missing fields in the case;

modifying the case with the imputed data, wherein the modified case becomes part of the computer-based reasoning model in place of the original case to create an updated computer-based reasoning model,

wherein the method is performed by one or more computing devices,

causing, with a control system, control of a system with the updated computer-based reasoning model.

2. The method of claim 1 , wherein causing control of the system comprises: receiving a request for an action to take in the system, including a context for the system; determining the action to take based at least in part on the context for the system and the updated computer-based reasoning model; causing the control system to perform the determined action in the system.

3. The method of claim 1 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with a certainty that a particular set of data fits a model.

4. The method of claim 1 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with how much information a point distorts the model.

5. The method of claim 1 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with information required to describe the position of the point in question relative to existing points.

6. The method of claim 1 , wherein determining imputed data for the missing field comprises:

determining the imputed data based on a machine learning model for the computer-based reasoning model's data, wherein the machine learning model for the computer-based reasoning model's data has been trained using the data in the computer-based reasoning model;

and the method further comprises:

determining an update to the machine learning model based on the updated computer-based reasoning model.

7. One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause performance of a method of:

performing the following until there are no more cases in a computer-based reasoning model with missing fields for which imputation is desired:

determining which cases have fields to impute in the computer-based reasoning model;

determining imputation order information for the cases that have data to impute in the computer-based reasoning model based at least in part on numbers of features that need imputed data for each of the cases;

determining for which one or more cases with missing fields to impute data for the missing fields based on the imputation order information, wherein determining for which one or more cases with missing fields to impute data for the missing fields based on the imputation order information comprises:

determining which one or more particular cases of the cases has a lowest number of features that need imputed data, and

determining to impute data for the missing fields for the one or more particular cases that have the lowest number of features that need imputed data among the one or more particular cases;

for each of the determined one or more particular cases with the lowest number of missing fields to impute:

determining imputed data for a missing field of the missing fields based on the case, and an imputation model, and the missing fields in the case;

modifying the case with the imputed data, wherein the modified case becomes part of the computer-based reasoning model in place of the original case to create an updated computer-based reasoning model,

causing, with a control system, control of a system with the updated computer-based reasoning model.

8. The one or more non-transitory storage media of claim 7 , wherein causing control of the system comprises: receiving a request for an action to take in the system, including a context for the system; determining the action to take based at least in part on the context for the system and the updated computer-based reasoning model; causing the control system to perform the determined action in the system.

9. The one or more non-transitory storage media of claim 7 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with a certainty that a particular set of data fits a model.

10. The one or more non-transitory storage media of claim 7 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with how much information a point distorts the model.

11. The one or more non-transitory storage media of claim 7 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with information required to describe the position of the point in question relative to existing points.

12. The one or more non-transitory storage media of claim 7 , wherein determining imputed data for the missing field comprises:

determining the imputed data based on a machine learning model for the computer-based reasoning model's data, wherein the machine learning model for the computer-based reasoning model's data has been trained using the data in the computer-based reasoning model;

and the method further comprises:

determining an update to the machine learning model based on the updated computer-based reasoning model.

13. A system comprising one or more computing devices, which one or more computing devices are configured to perform a method of:

performing the following until there are no more cases in a computer-based reasoning model with missing fields for which imputation is desired:

determining which cases have fields to impute in the computer-based reasoning model;

determining imputation order information for the cases that have data to impute in the computer-based reasoning model based at least in part on numbers of features that need imputed data for each of the cases;

determining for which one or more cases with missing fields to impute data for the missing fields based on the imputation order information, wherein determining for which one or more cases with missing fields to impute data for the missing fields based on the imputation order information comprises:

determining which one or more particular cases of the cases has a lowest number of features that need imputed data, and

determining to impute data for the missing fields for the one or more particular cases that have the lowest number of features that need imputed data among the one or more particular cases;

for each of the determined one or more particular cases with the lowest number of missing fields to impute:

determining imputed data for a missing field of the missing fields based on the case, and an imputation model, and the missing fields in the case;

modifying the case with the imputed data, wherein the modified case becomes part of the computer-based reasoning model in place of the original case to create an updated computer-based reasoning model,

causing, with a control system, control of a system with the updated computer-based reasoning model.

14. The system of claim 13 , wherein determining imputed data for the missing field comprises:

determining the imputed data based on a machine learning model for the computer-based reasoning model's data, wherein the machine learning model for the computer-based reasoning model's data has been trained using the data in the computer-based reasoning model.

15. The system of claim 13 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with a certainty that a particular set of data fits a model.

16. The system of claim 13 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with how much information a point distorts the model.

17. The system of claim 13 , wherein

the one or more particular cases with the highest number of missing fields to impute comprise two or more particular cases with the highest number of missing fields to impute;

determining for which one or more cases with the missing fields to impute data for the missing fields based on the imputation order information further comprises:

determining which of the two or more particular cases has a highest certainty score among the two or more particular cases, wherein the certainty scores is determined by a certainty function associated with:

removing the case from the computer-based reasoning model;

adding the case back into the computer-based reasoning model,

wherein the certainty function is associated with information required to describe the position of the point in question relative to existing points.