Hierarchically organizing data using a partial least squares analysis (PLS-trees)
View Patent ↗A method and system for partitioning (clustering) large amounts of data in a relatively short processing time. The method involves providing a first data matrix and a second data matrix where each of the first and second data matrices includes one or more variables, and a plurality of data points. The method also involves determining a first score from the first data matrix using a partial least squares (PLS) analysis or orthogonal PLS (OPLS) analysis and partitioning the first and second data matrices (e.g., row-wise) into a first group and a second group based on the sorted first score, the variance of the first data matrix, and a variance of the first and second groups relative to the variances of the first and second data matrices.
1. A computer-implemented method comprising:
providing, by a computer, a first data matrix and a second data matrix, each of the first and second data matrices including one or more variables, and a plurality of data points;
determining, by the computer, a first score from the first data matrix using a partial least squares (PLS) analysis or orthogonal PLS (OPLS) analysis; and
partitioning, by the computer, the first and second data matrices row-wise into a first group and a second group based on the first score of the first data matrix, and a penalty function that evaluates the first group and the second group based on (a) the variance of the first data matrix, and (b) a variance in the first and second groups relative to the variances of the first and second data matrices.
2. The method of claim 1 , wherein partitioning comprises minimizing a parameter representative of the relationship between the variance of the first PLS or OPLS score, and the variation of the second data matrix.
3. The method of claim 1 , wherein partitioning comprises maximizing a statistical difference between the first and second groups, the statistical difference calculated based on the variance of the first PLS or OPLS score of the first data matrix, the variance of the second data matrix of each group, and a function relating the sizes of the data matrices remaining in the first and the second groups after partitioning.
4. The method of claim 1 , wherein the first data matrix contains data representative of process data.
5. The method of claim 1 , wherein the second data matrix contains data representative of yield data, quality data, or a combination thereof.
6. The method of claim 1 , wherein the first data matrix contains data representative of measured or calculated data associated with the structural variation of molecules or macromolecules of interest.
7. The method of claim 1 , wherein the second data matrix contains data representative of biological data of the same molecules or macromolecules.
8. The method of claim 1 , wherein the first group comprises a third data matrix and a fourth data matrix each resulting from row-wise partitioning the first and second data matrices into the first and second groups, further comprising: determining a second score from the third data matrix using a second partial least squares (PLS) analysis or OPLS analysis; and partitioning the third and fourth data matrices row-wise into a third group and a fourth group based on the second score of the third data matrix, the variance of the third data matrix, and a variance in the third and fourth groups relative to the variances of the third and fourth data matrices.
9. The method of claim 8 , wherein the second group comprises a fifth data matrix and a sixth data matrix, further comprising: determining a third score from the fifth matrix using a third partial least squares (PLS) analysis or OPLS analysis when the second group includes more than a threshold number of data points; and partitioning the fifth and sixth data matrices row-wise into a fifth group and a sixth group based on the third score of the fifth data matrix, the variance of the third data matrix, and a variance in the fifth and sixth groups relative to the variances in the fifth and sixth data matrices.
10. The method of claim 9 , further comprising hierarchically displaying the first, second, third, fourth, fifth, or sixth groups.
11. The method of claim 9 , further comprising terminating partitioning of the second group when the second group includes less than a threshold number of data points.
12. The method of claim 9 , further comprising terminating partitioning of the second group when the combined variances of the second score and the second data matrix do not decrease upon partitioning the second group into the fifth and six groups.
13. The method of claim 9 , further comprising terminating partitioning of the second group when the number of previous partitions associated with the first and second data matrices equals or exceeds a predetermined threshold.
14. The method of claim 9 wherein the predetermined threshold is a limiting value representative of a maximum number of hierarchical levels in a dendrogram.
15. The method of claim 1 , further comprising identifying the first group or the second group on a graph displaying the first data matrix and the second data matrix.
16. The method of claim 1 , wherein the variances are calculated using inter-quartiles.
17. The method of claim 1 , further comprising partitioning the first and second data matrices row-wise into a first group comprising a first set of data rows and a second group comprising a second set of data rows.
18. A computer program product, tangibly embodied in a non-transitory computer readable medium, the computer program product including instructions being operable to cause data processing apparatus to:
receive a first data matrix and a second data matrix, each of the first and second data matrices including one or more data points;
determine a first score from the first data matrix using a partial least squares (PLS) analysis or OPLS analysis of the first and second data matrices; and
partition the first and second data matrices row-wise into a first group and a second group based on the first score of the first data matrix, and a penalty function the evaluates the first group and the second group based on (a) the variance of the first data matrix, and (b) a variance in the first and second groups relative to the variances of the first and second data matrices.
19. A system for hierarchically organizing data, the system comprising:
(a) a memory including:
(a1) a data structure including a first data matrix and a second data matrix;
(b) a processor operatively coupled to the memory, the processor comprising;
(b1) a module for determining a first score based in part on a partial least squares analysis or OPLS analysis of the first data matrix;
(b2) a module for partitioning the first and second data matrices to generate a first group and a second group based in part on the first score of the first data matrix, and a penalty function that evaluates the first group and the second group based on (a) the variance of the first data matrix, and (b) a variance in the first and second groups relative to the first and second data matrices; and
(c) a display operatively coupled to the processor to display the first and second groups and an association of the first and second groups to the first and second data matrices.
20. A system for analyzing data, the system comprising:
a data retrieval means for retrieving a first data matrix and a second data matrix from a memory, each of the first and second data matrices including one or more data points;
a data analysis means to determine a first score from the first data matrix using a partial least squares (PLS) analysis or OPLS analysis; and
a data partitioning means to divide the first and second data matrices into a first group and a second group based on the first score of the first data matrix, and a penalty function that evaluates the first group and the second group based on (a) the variance of the first data matrix, and (b) a variance in the first and second groups relative to the variances of the first and second data matrices.