Methods and apparatus for data analysis
A method and apparatus for data analysis according to various aspects of the present invention is configured to automatically select one or more outlier identification algorithms for identifying statistical outliers in test data for components.
1. A test data analysis system for analyzing test data for a set of components fabricated and tested using a fabrication process, comprising:
a memory for storing the test data; and
an outlier identification system having access to the memory and configured to identify outliers in the test data according to a selected outlier identification algorithm, wherein the outlier identification system is configured to automatically select the selected outlier identification algorithm from a plurality of outlier identification algorithms according to a data population type of the test data.
2. A computer-implemented method for testing components comprising:
obtaining test data for the components;
automatically selecting an outlier identification algorithm from a plurality of outlier identification algorithms according to a data population type of the test data; and
automatically identifying outliers in the test data according to the selected outlier identification algorithm.
3. A medium storing instructions executable by a machine, wherein the instructions cause the machine to execute a method for analyzing test data comprising:
obtaining test data for the components;
automatically selecting an outlier identification algorithm from a plurality of outlier identification algorithms according to a data population type of the test data; and
automatically identifying an outlier in the test data using the selected outlier identification algorithm.
4. A test data analysis system according to claim 1 , wherein the outlier identification system is configured to select the selected outlier identification algorithm according to at least one descriptive statistic derived from the test data.
5. A test data analysis system according to claim 1 , wherein the outlier identification system is configured to forego automatically identifying the outliers in the test data if the test data is a selected data population type.
6. A test data analysis system according to claim 1 , wherein the outlier identification system comprises a pre-processing engine configured to identify the outliers in the test data using more of the plurality of outlier identification algorithms than the selected outlier identification algorithm.
7. A test data analysis system according to claim 1 , wherein the outlier identification system is configured to select the selected outlier identification algorithm according to user-defined rules.
8. A test data analysis system according to claim 1 , wherein the outlier identification system is configured to select the selected outlier identification algorithm according to a classification hierarchy based on characteristics of the test data.
9. A test data analysis system according to claim 1 , wherein the outlier identification system is configured to automatically identify outliers in the test data according to multiple outlier identification algorithms and discard results generated by outlier identification algorithms other than the selected outlier identification algorithm.
10. A test data analysis system according to claim 1 , wherein the outlier identification system comprises:
a pre-processing engine configured to analyze the test data according to the plurality of outlier identification algorithms to generate pre-processing data; and
a classification engine configured to select the selected outlier identification algorithm based on the pre-processing data.
11. A test data analysis system according to claim 1 , wherein the test data for the set of components includes tests data from multiple tests for the components, and wherein the outlier identification system is configured to automatically identify outliers in the test data for each test according to a test-specific selected outlier identification algorithm selected for the test data for each test, wherein the test-specific selected outlier identification algorithm for each test is automatically selected for each test from the plurality of outlier identification algorithms.
12. A test data analysis system according to claim 1 , wherein the plurality of outlier identification algorithms is configured to be modified by a user.
13. A test data analysis system according to claim 1 , wherein the test data includes raw test data and filtered test data, wherein the filtered test data comprises the raw data without test data corresponding to failures.
14. A test data analysis system according to claim 1 , wherein the outlier identification system is configured to:
forego automatically identifying the outliers in the test data according to the selected outlier identification algorithm if the test data is generated by a user-specified test; and
identify outliers in the test data for the selected test using a user-specified outlier identification algorithm.
15. A test data analysis system according to claim 1 , wherein:
the selected outlier identification compares the test data to an upper threshold and a lower threshold;
the upper threshold and the lower threshold are defined according to different distances from a mean between an upper test limit and a lower test limit.
16. A test data analysis system according to claim 1 , wherein the outlier identification system is configured to:
dynamically adjust a threshold according to the test data; and
compare the test data to the threshold to identify the outliers.
17. A test data analysis system according to claim 16 , wherein the outlier identification system is configured to forego dynamically adjusting the upper threshold and the lower threshold and assign selected values to the upper threshold and the lower threshold if a selected portion of the tests data is at least one of (a) within a selected range of the upper test limit or the lower test limit, and (b) beyond a selected range of the upper test limit or the lower test limit.
18. A computer-implemented method according to claim 2 , wherein selecting the outlier identification algorithm includes selecting the selected outlier identification algorithm according to at least one descriptive statistic derived from the test data.
19. A computer-implemented method according to claim 2 , further comprising foregoing automatically identifying the outliers in the test data if the test data is a selected data population type.
20. A computer-implemented method according to claim 2 , wherein selecting the outlier identification algorithm includes identifying the outliers in the test data using more of the plurality of outlier identification algorithms than the selected outlier identification algorithm.
21. A computer-implemented method according to claim 2 , wherein selecting the outlier identification algorithm includes selecting the selected outlier identification algorithm according to user-defined rules.
22. A computer-implemented method according to claim 2 , wherein selecting the outlier identification algorithm includes selecting the selected outlier identification algorithm according to a classification hierarchy based on characteristics of the test data.
23. A computer-implemented method according to claim 2 , wherein selecting the outlier identification algorithm includes automatically identifying outliers in the test data according to multiple outlier identification algorithms and discarding results generated by outlier identification algorithms other than the selected outlier identification algorithm.
24. A computer-implemented method according to claim 2 , wherein selecting the outlier identification algorithm includes:
analyzing the test data according to the plurality of outlier identification algorithms to generate pre-processing data; and
selecting the selected outlier identification algorithm based on the pre-processing data.
25. A computer-implemented method according to claim 2 , wherein the test data for the set of components includes tests data from multiple tests for the components, and wherein selecting the outlier identification algorithm includes selecting a test-specific selected outlier identification algorithm for each test from the plurality of outlier identification algorithms.
26. A computer-implemented method according to claim 2 , wherein the plurality of outlier identification algorithms is configured to be modified by a user.
27. A computer-implemented method according to claim 2 , wherein the test data includes raw test data and filtered test data, wherein the filtered test data comprises the raw data without test data corresponding to failures.
28. A computer-implemented method according to claim 2 , wherein automatically identifying outliers comprises:
foregoing automatically identifying the outliers in the test data according to the selected outlier identification algorithm if the test data is generated by a user-specified test; and
identifying outliers in the test data for the selected test using a user-specified outlier identification algorithm.
29. A computer-implemented method according to claim 2 , wherein automatically identifying outliers comprises:
comparing the test data to an upper threshold and a lower threshold, wherein the upper threshold and the lower threshold are defined according to different distances from a mean between an upper test limit and a lower test limit.
30. A computer-implemented method according to claim 2 , wherein automatically identifying outliers comprises:
dynamically adjusting a threshold according to the test data; and
comparing the test data to the threshold to identify the outliers.
31. A computer-implemented method according to claim 30 , wherein automatically identifying outliers further comprises foregoing dynamically adjusting the upper threshold and the lower threshold and assigning selected values to the upper threshold and the lower threshold if a selected portion of the tests data is at least one of (a) within a selected range of the upper test limit or the lower test limit and (b) beyond a selected range of the upper test limit or the lower test limit.
32. A medium according to claim 3 , wherein selecting the outlier identification algorithm includes selecting the selected outlier identification algorithm according to at least one descriptive statistic derived from the test data.
33. A medium according to claim 3 , further comprising foregoing automatically identifying the outliers in the test data if the test data is a selected data population type.
34. A medium according to claim 3 , wherein selecting the outlier identification algorithm includes identifying the outliers in the test data using more of the plurality of outlier identification algorithms than the selected outlier identification algorithm.
35. A medium according to claim 3 , wherein selecting the outlier identification algorithm includes selecting the selected outlier identification algorithm according to user-defined rules.
36. A medium according to claim 3 , wherein selecting the outlier identification algorithm includes selecting the selected outlier identification algorithm according to a classification hierarchy based on characteristics of the test data.
37. A medium according to claim 3 , wherein selecting the outlier identification algorithm includes automatically identifying outliers in the test data according to multiple outlier identification algorithms and discarding results generated by outlier identification algorithms other than the selected outlier identification algorithm.
38. A medium according to claim 3 , wherein selecting the outlier identification algorithm includes:
analyzing the test data according to the plurality of outlier identification algorithms to generate pre-processing data; and
selecting the selected outlier identification algorithm based on the pre-processing data.
39. A medium according to claim 3 , wherein the test data for the set of components includes tests data from multiple tests for the components, and wherein selecting the outlier identification algorithm includes selecting a test-specific selected outlier identification algorithm for each test from the plurality of outlier identification algorithms.
40. A medium according to claim 3 , wherein the plurality of outlier identification algorithms is configured to be modified by a user.
41. A medium according to claim 3 , wherein the test data includes raw test data and filtered test data, wherein the filtered test data comprises the raw data without test data corresponding to failures.
42. A medium according to claim 3 , wherein automatically identifying outliers comprises:
foregoing automatically identifying the outliers in the test data according to the selected outlier identification algorithm if the test data is generated by a user-specified test; and
identifying outliers in the test data for the selected test using a user-specified outlier identification algorithm.
43. A medium according to claim 3 , wherein automatically identifying outliers comprises:
comparing the test data to an upper threshold and a lower threshold, wherein the upper threshold and the lower threshold are defined according to different distances from a mean between an upper test limit and a lower test limit.
44. A medium according to claim 3 , wherein automatically identifying outliers comprises:
dynamically adjusting a threshold according to the test data; and
comparing the test data to the threshold to identify the outliers.
45. A medium according to claim 44 , wherein automatically identifying outliers further comprises foregoing dynamically adjusting the upper threshold and the lower threshold and assigning selected values to the upper threshold and the lower threshold if a selected portion of the tests data is at least one of (a) within a selected range of the upper test limit or the lower test limit and (b) beyond a selected range of the upper test limit or the lower test limit.