Decorrelating effects in multiple linear regression to decompose and attribute risk to common and proper effects
Effects in multiple linear regression may be decorrelated to decompose and attribute risk to common and proper effects. In other words, an attribute risk may be decomposed to two or more causes, where each cause is characterized by multiple attributes. The risk decomposition may decompose risk into a first residual part associated with a first set of risk factors, a second residual part associated with a second set of risk factors, and a common part associated with a set of common hidden variables that minimize a correlation between the first set of factors and the second set of factors. The common hidden variables may be modeled using a hidden factor model. An effect of the correlation may be minimized on the first set of risk factors and the second set of risk factors, and how correlated the terms of the risk decomposition are may be quantified.
1. A computer program for controlling an amount of unexplained correlation that remains in data after accounting for common hidden variables, the program embodied on a non-transitory computer-readable storage medium, the program configured to cause at least one processor to:
determine residual matrices R 1 and R 2 comprising a first residual part and a second residual part for a first set of risk factors and a second set of risk factors, respectively;
when R 1 t R 2 =0:
perform a three-way risk decomposition approach enforcing orthogonality of the first residual part and the second residual part that also decomposes risk into a common part associated with a set of common hidden variables common to R 1 and R 2 that minimize a correlation between the first set of risk factors and the second set of risk factors, the common hidden variables modeled using a hidden factor model, and
generate a computer-based data structure corresponding to linear vector spaces of unobserved latent variables, wherein the unobserved latent variables are represented as two matrices A and B whose inner-product A T B=0;
when R 1 t R 2 ≠0, perform a generalized risk decomposition approach without enforcing orthogonality of the first residual part and the second residual part;
quantify how correlated the terms of the risk decomposition are based on the performed risk decomposition approach; and
output the quantification, wherein
a maximum correlation is used between linear combinations of explanatory variables for each of the first set of risk factors and the second set of risk factors, given by:
sup
{
x
1
t
x
2
x
1
x
2
|
x
1
∈
X
1
,
x
2
∈
X
2
}
where X 1 and X 2 are linear spaces spanned by columns of two design matrices X 1 and X 2 associated with the explanatory variables of the first set of risk factors and the second set of risk factors, respectively.
2. The computer program of claim 1 , wherein the generalized risk decomposition approach comprises:
selecting a sufficiently small number ε such that correlation effects are minimized to a predetermined tolerance; and
computing contributions from each risk factor.
3. The computer program of claim 2 , wherein ε is 10 −3 or less.
4. The computer program of claim 2 , wherein given E, contributions to total risk is decomposed according to:
γ 1 =X 1 β 1 =f 1 +r 1 and γ 2 =X 2 β 2 =f 2 +r 2
where f 1 t r 1 =f 2 t r 2 =0 and r 1 t r 2 ≤ε∥r 1 ∥∥r 2 ∥.
5. The computer program of claim 4 , wherein a squared norm of r 1 , once a linear effect from r 2 is removed, is greater of equal to (1−ε 2 )∥r 1 ∥ 2 .
6. The computer program of claim 4 , wherein X 1 and X 2 are design matrices of dimensions n× t and n× 2 , respectively, with probabilities t + 2 = and both 1 , 2 ≥2.
7. The computer program of claim 1 , wherein when R 1 t R 2 =0, the three-way risk decomposition enforces orthogonality of R 1 and R 2 by setting a risk equal to [f(X 1 )C 1 +f(X 2 )C 2 ]+g(X 1 )C 1 +h(X 2 )C 2 , where f, g, and h are functions that are constructed such that each of the terms [f(X 1 )C 1 +f(X 2 )C 2 ], g(X 1 )C 1 , and h(X 2 )C 2 are uncorrelated.
8. The computer program of claim 1 , wherein when R 1 t R 2 ≠0, contributions from each risk factor are computed by:
setting 0<ε<1 to a sufficiently small value such that correlation effects are minimized to a predetermined tolerance;
performing an eigen decomposition of projection matrices P 1 =(X 1 t X 1 ) −1 X 1 t and P 2 =X 2 (X 2 t X 2 ) −1 X 2 t to identify an orthonormal basis U 1* and U 2* for 1 and 2 , a linear span of columns of design matrices X 1 and X 2 , respectively;
performing an eigen decomposition of Z 1 =U 1* t P 2 U 1* and Z 2 =U 2* t P 1 U 2* ;
setting I 1 and I 2 to indices of the eigenvalues of Z 1 and Z 2 , respectively, that are less than ε 2 ;
setting Q 1 ={U 1* w 1j :j∈I 1 }, Q 2 ={U 2* w 2j :j∈I 2 }; and
computing C 1 =X 1 −Q 1 Q 1 t X 1 and C 2 =X 2 −Q 2 Q 2 t X 2 .
9. A computer-implemented method for controlling an amount of unexplained correlation that remains in data after accounting for common hidden variables, comprising:
determining, by a computing system, residual matrices R 1 and R 2 comprising a first residual part and a second residual part for a first set of risk factors and a second set of risk factors, respectively;
performing a three-way risk decomposition approach, by the computing system, enforcing orthogonality of the first residual part and the second residual part that also decomposes risk into a common part associated with a set of common hidden variables common to R 1 and R 2 that minimize a correlation between the first set of risk factors and the second set of risk factors, the common hidden variables modeled using a hidden factor model;
generating, by the computing system, a computer-based data structure corresponding to linear vector spaces of unobserved latent variables, wherein the unobserved latent variables are represented as two matrices A and B whose inner-product A T B=0;
quantifying, by the computing system, how correlated the terms of the risk decomposition are based on the performed three-way risk decomposition approach; and
outputting the quantification, by the computing system, wherein
a maximum correlation is used between linear combinations of explanatory variables for each of the first set of risk factors and the second set of risk factors, given by:
sup
{
x
1
t
x
2
x
1
x
2
|
x
1
∈
X
1
,
x
2
∈
X
2
}
where X 1 and X 2 are linear spaces spanned by columns of two design matrices X 1 and X 2 associated with the explanatory variables of the first set of risk factors and the second set of risk factors, respectively.
10. The computer-implemented method of claim 9 , wherein the three-way risk decomposition enforces orthogonality of R 1 and R 2 by setting a risk equal to [f(X 1 )C 1 +f(X 2 )C 2 ]+g(X 1 )C 1 +h(X 2 )C 2 , where f, g, and h are functions that are constructed such that each of the terms [f(X 1 )C 1 +f(X 2 )C 2 ], g(X 1 )C 1 , and h(X 2 )C 2 are uncorrelated.
11. The computer-implemented method of claim 9 , wherein the method is performed when R 1 t R 2 =0.
12. A computer-implemented method for controlling an amount of unexplained correlation that remains in data after accounting for common hidden variables, comprising:
determining, by a computing system, residual matrices R 1 and R 2 comprising a first residual part and a second residual part for a first set of risk factors and a second set of risk factors, respectively;
performing a generalized risk decomposition approach, by the computing system, without enforcing orthogonality of the first residual part and the second residual part;
generating, by the computing system, a computer-based data structure representing the generalized risk decomposition;
quantifying, by the computing system, how correlated the terms of the risk decomposition are based on the performed generalized risk decomposition approach; and
outputting the quantification as a data structure, by the computing system, wherein
contributions from each risk factor are computed by:
setting 0<ε<1 to a sufficiently small value such that correlation effects are minimized to a predetermined tolerance, by the computing system;
performing, by the computing system, an eigen decomposition of projection matrices P 1 =(X 1 t X 1 ) −1 X 1 t and P 2 =X 2 (X 2 t X 2 ) −1 X 2 t to identify an orthonormal basis U 1* and U 2* for 1 and 2 , a linear span of columns of design matrices X 1 and X 2 , respectively;
performing, by the computing system, an eigen decomposition of Z 1 =U 1* t P 2 U 1* and Z 2 =U 2* t P 1 U 2* ;
setting I 1 and I 2 to indices of the eigenvalues of Z 1 and Z 2 , respectively, that are less than ε 2 , by the computing system;
setting Q 1 ={U 1* w 1j :j∈I 1 }, Q 2 ={U 2* w 2j :j∈I 2 }, by the computing system; and
computing C 1 =X 1 −Q 1 Q 1 t X 1 and C 2 =X 2 −Q 2 Q 2 t X 2 , by the computing system.
13. The computer-implemented method of claim 12 , wherein the generalized risk decomposition approach comprises:
selecting a sufficiently small number ε such that correlation effects are minimized to a predetermined tolerance, by the computing system; and
computing contributions from each risk factor, by the computing system.
14. The computer-implemented method of claim 13 , wherein ε is 10 −3 or less.
15. The computer-implemented method of claim 13 , wherein given E,
contributions to total risk is decomposed according to:
γ 1 =X 1 β 1 =f 1 +r 1 and γ 2 =X 2 β 2 =f 2 +r 2
where f 1 t r 1 =f 2 t r 2 =0 and r 1 t r 2 ≤ε∥r 1 ∥∥r 2 ∥.
16. The computer-implemented method of claim 13 , wherein a squared norm of r 1 , once a linear effect from r 2 is removed, is greater of equal to (1−ε 2 )∥r 1 ∥ 2 .
17. The computer-implemented method of claim 12 , wherein the method is performed when R 1 t R 2 ≠0.