Preferential system identification (PSID) for joint dynamic modeling of signals with dissociation and prioritization of their shared dynamics, with applicability to modeling brain and behavior data
A method for preferential system identification (PSID) for modeling neural dynamics of a brain includes extracting behaviorally relevant latent brain states via a projection of future behavior onto past neural activity. The method further includes identifying, based on the extracted latent brain states, model parameters for a linear state space dynamic model of neural activity. An extension of PSID includes accounting for an effect of external inputs to the brain by using oblique projections along the external inputs instead of orthogonal projections. PSID may be extended to apply to any primary signals and secondary signals in place of the neural activity and the behavior. In some embodiments, the extracting and the identifying may be performed using a numerical optimization of recurrent neural networks (RNN) instead of the projection.
1. A method for preferential system identification (PSID) for modeling neural dynamics of a brain by a recurrent neural network (RNN), the method comprising:
extracting, by the recurrent neural network, behaviorally relevant latent brain states via a projection of future behavior onto past neural activity, wherein the future behavior includes a movement kinematic and wherein the behaviorally relevant latent brain states are brain states associated with the movement kinematic; and
identifying, by the recurrent neural network, based on the extracted latent brain states, model parameters for a linear state space dynamic model of neural activity.
2. The method of claim 1 wherein extracting the behaviorally relevant latent brain states includes:
forming examples of the future behavior and associated examples of the past neural activity; and
projecting the examples of the future behavior onto the associated past neural activity.
3. The method of claim 2 wherein extracting the behaviorally relevant latent brain states further includes computing a singular value decomposition (SVD) of the projection of the future behavior onto the past neural activity.
4. The method of claim 3 wherein identifying the model parameters includes identifying a full latent state consisting of the extracted behaviorally relevant latent brain states at each time-step.
5. The method of claim 1 further comprising extracting behaviorally irrelevant latent brain states via a projection of residual future neural activity onto the past neural activity, the residual future neural activity not being described by the behaviorally relevant latent brain states, wherein the behaviorally irrelevant latent brain states are brain states that are not the behaviorally relevant latent brain states.
6. The method of claim 5 wherein the method is further for dissociating neural dynamics that are behaviorally relevant from neural dynamics that are behaviorally irrelevant while prioritizing the neural dynamics that are behaviorally relevant.
7. The method of claim 5 wherein extracting the behaviorally irrelevant latent brain states further includes:
computing a singular value decomposition (SVD) of the result of projecting the residual future neural activity onto the past neural activity; and
computing the behaviorally irrelevant latent brain states based on the SVD.
8. The method of claim 7 wherein identifying the model parameters includes identifying a full latent state consisting of a concatenation of the extracted behaviorally relevant latent brain states and the behaviorally irrelevant latent brain states at each time-step.
9. The method of claim 8 wherein identifying the model parameters further includes identifying additional full latent states each having a shift of a single step in time.
10. The method of claim 9 wherein identifying the model parameters further includes computing a least squares solution for the model parameters using a pseudoinverse calculation.
11. The method of claim 10 wherein identifying the model parameters further includes computing a covariance of residuals in the least squares solution to get noise statistics.
12. The method of claim 11 wherein the model parameters are used to construct a Kalman filter to extract the behaviorally relevant latent brain states and the behaviorally irrelevant latent brain states from new neural data and to decode behavior from the extracted states.
13. The method of claim 1 wherein an effect of an external input to the brain can be accounted for by using oblique projections along an external input instead of orthogonal projections.
14. A method for preferential system identification (PSID) for modeling neural dynamics of a brain, the method comprising:
extracting behaviorally relevant latent brain states, wherein the behaviorally relevant latent brain states are brain states associated with a movement kinematic; and
identifying, based on the extracted latent brain states, model parameters for a state space dynamic model of neural activity,
wherein the extracting and the identifying are performed using a numerical optimization of recurrent neural networks (RNN).
15. The method of claim 14 wherein extracting the behaviorally relevant latent brain states is performed using the RNN that minimizes an error of decoding behavior from the neural activity.
16. The method of claim 14 wherein behavior measurements include any known distribution including categorical distribution and the empirical optimization minimizes a negative log-likelihood of the behavior measurements given past neural activity.
17. The method of claim 14 further comprising extracting behaviorally irrelevant latent brain states using the RNN that minimizes a neural reconstruction loss, wherein the behaviorally irrelevant latent brain states are brain states that are not the behaviorally relevant latent brain states.
18. The method of claim 14 wherein neural measurements include any known distribution including a Poisson distribution and the optimization minimizes a negative log-likelihood of the neural activity given past neural activity.
19. The method of claim 14 wherein behavioral and neural measurements can be intermittently available in time rather than continuously available, and numerical optimization is solved only based on the intermittently available behavioral and neural measurements.
20. The method of claim 14 wherein the neural activity has nonlinear dynamics and a state-space dynamic model is made nonlinear by making components of the RNN be nonlinear multilayer neural networks.
21. A system for preferential system identification (PSID) for modeling neural dynamics of a brain, the system comprising:
an input device configured to detect or receive at least one of past neural activity data or behavior; and
a machine learning processor coupled to the input device and configured to:
extract behaviorally relevant latent brain states via a projection of future behavior onto the at least one of the past neural activity data or the behavior, wherein the future behavior includes a movement kinematic and wherein behaviorally relevant latent brain states are brain states associated with the movement kinematic; and
identify, based on the extracted latent brain states, model parameters for a liner state space dynamic model of neural activity, and
wherein the machine learning processor utilizes recurrent neural networks (RNN) to extract the behaviorally relevant latent brain states and to identify the model parameters for the state space dynamic model of neural activity.
22. The system of claim 21 wherein the input device is configured to additionally detect or receive external inputs to the brain, and the machine learning processor is further configured to utilize oblique projections along the external input to extract the behaviorally relevant latent brain states.