Granted Patent B1

US 12,431,235 · App. 17/558,312 · Granted Sep 30, 2025

Inventors: Nolan Serrao (Plano, TX); Keegan Patrick Hayes (Whitestown, IN); Arthur Quentin Smith (Fredericksburg, TX); John Andrew Weems (San Antonio, TX); Salvador Adrian Bretado (San Antonio, TX); Justin Royell Nash (Little Elm, TX); David Joaquin Harris (San Antonio, TX); Carol Lyn Lawrence (Fair Oaks Ranch, TX); Theresa Marie Matowitz (San Antonio, TX); Courtney St. Martin (Duluth, GA)

Assignee: United Services Automobile Association (USAA)

G16H20/70G06F9/453G16H50/20

Abstract

Claims (102)

1. A method for detecting cognitive impairment in users of a computing system, the method comprising:

obtaining multiple sets of inputs, each associated with a different user, of multiple users, and each set of inputs, of the multiple sets of inputs, having multiple dimensions, wherein the multiple dimensions include:

transaction history inputs, and

computer user interface (UI) interaction pattern inputs;

labeling multiple of the users associated with the sets of inputs as either having or not having a cognitive impairment by applying an unsupervised learning classifier to the sets of inputs;

training a machine learning model to identify users with cognitive impairments through a supervised learning process that iterates through the multiple sets of inputs, each set of inputs being associated with one different user, of the multiple users, and each different user being assigned a label as either having or not having a cognitive impairment, wherein each iteration includes updating aspects of the machine learning model by:

applying the machine learning model to a current set of inputs, of the multiple sets of inputs, for a current iteration to produce results by taking the current set of inputs through one or more nodes, each node:

transforming an input, of the current set of inputs, according to a node coefficient, and passing the results, with a particular edge weight, to a next layer until the results are mapped to a cognitive impairment prediction result;

comparing the cognitive impairment prediction result to the label assigned to the different user associated with the current set of inputs; and

updating the machine learning model by updating the node coefficient and/or the particular edge weight of at least one node, of the one or more nodes, based on the comparison;

labeling a new user as having a cognitive impairment based on an application of the trained machine learning model to a new set of inputs corresponding to the new user, the new set of inputs including one or more interactions of the user with a keyboard, of a computing system, as an input device,

wherein the labeled cognitive impairment is at least one of a brain injury, a learning disability, a stroke, post-traumatic stress disorder, depression, schizophrenia, or any combination thereof; and

based on the labeling of the new user as having the cognitive impairment, applying one or more adaptive measures to the computing system, the one or more adaptive measures including automatically switching, by the computing system, to a microphone as the input device of the computing system, through which one or more further interactions of the user are received.

2. The method of claim 1 , wherein the applying the unsupervised learning classifier to the sets of inputs includes:

iteratively applying the unsupervised learning classifier,

wherein each iteration includes applying the unsupervised learning classifier to each of the sets of inputs to get a classification, and

wherein the iterations end when the classifications between iterations do not vary above a threshold amount.

3. The method of claim 2 ,

wherein at least some of the multiple sets of inputs are each associated with one different user who has a pre-classification of having or not having a cognitive impairment; and

wherein the applying the unsupervised learning classifier to the sets of inputs further includes:

analyzing, for the inputs with pre-classifications, ratios of correctly classified inputs to incorrectly classified inputs in each of multiple clusters of classifications resulting when the classification iterations end;

based on the analyzing, re-performing the classification iterations with different ones of the multiple dimensions until the ratios of correctly classified inputs to incorrectly classified inputs in each of the multiple clusters are above a threshold level of cluster homogeneousness; and

in response to the ratios being above the threshold level of cluster homogeneousness, performing the labeling of the multiple of the users associated with the sets of inputs by labeling each different user in a cluster with the pre-classification most prevalent in that cluster.

4. The method of claim 1 , wherein the multiple dimensions further include communication history inputs and errors or lost data history inputs, and wherein the applying the machine learning model to the current set of inputs includes one or more of:

formulating the communication history inputs as codes corresponding to identified emotional state changes, language structure changes, word choice changes, or any combination thereof;

formulating the transaction history inputs as one or more values representing a number of late payments, a number of repeated payments, or ratios of recent late payments or repeated payments to a previous amount of late payments or repeated payments for the one different user, of the multiple users, or for a typical user;

formulating the errors or lost data history inputs as one or more values indicating a frequency of such losses as compared to a previous loss rate for the one different user, of the multiple users, or for a typical user; or

any combination thereof.

5. The method of claim 1 , wherein the multiple dimensions further include location or body state history inputs, and wherein the applying the machine learning model to the current set of inputs includes one or more of:

formulating the location or body state history inputs as a frequency of identified abnormalities in recorded location or body state for the one different user of the multiple users;

formulating the computer user interface (UI) interaction pattern inputs as a frequency of identified abnormalities in UI interactions performed by the one different user of the multiple users; or

any combination thereof.

6. The method of claim 1 ,

wherein the machine learning model is a type of a neural network; and

wherein the updating the machine learning model, based on the comparison, includes applying a loss function to update parameters of the neural network such that the cognitive impairment prediction result is closer to the label assigned to the one different user associated with the current set of inputs.

7. The method of claim 1 ,

wherein the machine learning model is trained to identify types of cognitive impairment; and

wherein the labeling the new user as having the cognitive impairment includes labeling the new user with a type of cognitive impairment indicated by the machine learning model.

8. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for detecting cognitive impairment in users, the process comprising:

obtaining multiple sets of inputs, each associated with a different user, of multiple users, and each set of inputs, of the multiple sets of inputs, having multiple dimensions;

labeling multiple of the users associated with the sets of inputs as either having or not having a cognitive impairment by iteratively applying an unsupervised learning classifier to the set of inputs,

wherein each iteration includes applying the unsupervised learning classifier to each of the sets of inputs to get a classification, and

wherein the iterations end when the classifications between iterations do not vary above a threshold amount;

training a machine learning model to identify users with cognitive impairments through a supervised learning process that iterates through the multiple sets of inputs, each set of inputs being associated with one different user, of the multiple users, and each different user being assigned a label as either having or not having a cognitive impairment, wherein each iteration includes updating aspects of the machine learning model by:

applying the machine learning model to a current set of inputs, of the multiple sets of inputs, for a current iteration to produce results by taking the current set of inputs through one or more nodes, each node:

transforming an input, of the current set of inputs, according to a node coefficient, and passing the results, with a particular edge weight, to a next layer until the results are mapped to a cognitive impairment prediction result;

comparing the cognitive impairment prediction result to the label assigned to the different user associated with the current set of inputs; and

updating the machine learning model by updating the node coefficient and/or the particular edge weight of at least one node, of the one or more nodes, based on the comparison;

labeling a new user as having a cognitive impairment based on an application of the trained machine learning model to a new set of inputs corresponding to the new user, the new set of inputs including one or more interactions of the user with a keyboard, of a computing device, as an input device,

wherein the labeled cognitive impairment is at least one of a brain injury, a learning disability, a stroke, post-traumatic stress disorder, depression, schizophrenia, or any combination thereof; and

based on the labeling of the new user as having the cognitive impairment, applying one or more adaptive measures to the computing system, the one or more adaptive measures including automatically switching, by the computing device, to a microphone as the input device of the computing device, through which one or more further interactions of the user are received.

9. The non-transitory computer-readable storage medium of claim 8 ,

wherein at least some of the multiple sets of inputs are each associated with one different user, of the multiple users, who has a pre-classification of having or not having a cognitive impairment; and

wherein the applying the unsupervised learning classifier to the sets of inputs further includes:

analyzing, for the inputs with pre-classifications, ratios of correctly classified inputs to incorrectly classified inputs in each of multiple clusters of classifications resulting when the classification iterations end;

based on the analyzing, re-performing a set of classification iterations with different ones of the multiple dimensions until the ratios of correctly classified inputs to incorrectly classified inputs in each of the multiple clusters are above a threshold level of cluster homogeneousness; and

in response to the ratios being above the threshold level of cluster homogeneousness, performing the labeling of the multiple of the users associated with the sets of inputs by labeling each user in a cluster with the pre-classification most prevalent in that cluster.

10. The non-transitory computer-readable storage medium of claim 8 , wherein the applying the machine learning model to the current set of inputs includes one or more of:

formulating the location or body state history inputs as a frequency of identified abnormalities in recorded location or body state for the one different user of the multiple users;

formulating the computer user interface (UI) interaction pattern inputs as a frequency of identified abnormalities in UI interactions performed by the one different user of the multiple users; or

any combination thereof.

11. The non-transitory computer-readable storage medium of claim 8 ,

wherein the machine learning model is a type of a neural network; and

wherein the updating the machine learning model, based on the comparison, includes applying a loss function to update parameters of the neural network.

12. A computing system for detecting cognitive impairment in users, the computing system comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:

obtaining multiple sets of inputs, each associated with a different user, of multiple users, and each set of inputs, of the multiple sets of inputs, having multiple dimensions;

labeling multiple of the users associated with the sets of inputs as either having or not having a cognitive impairment by applying an unsupervised learning classifier to the set of inputs;

training a machine learning model to identify users with cognitive impairments through a supervised learning process that updates aspects of the machine learning model using the ones of the multiple sets of inputs that are associated with one different user, of the multiple users, who has been labeled as either having or not having a cognitive impairment, by performing multiple iterations,

wherein the machine learning model is a type of neural network,

wherein each iteration includes updating aspects of the machine learning model by:

applying the machine learning model to a current set of inputs, of the multiple sets of inputs, for a current iteration to produce results by taking the current set of inputs through one or more nodes, each node:

transforming an input, of the current set of inputs, according to a node coefficient, and passing the results, with a particular edge weight, to a next layer until the results are mapped to a cognitive impairment prediction result;

comparing the cognitive impairment prediction result to the label assigned to the different user associated with the current set of inputs; and

updating the machine learning model by:

updating the node coefficient and/or the particular edge weight of at least one node, of the one or more nodes, based on the comparison; and

applying a loss function to update parameters of the neural network such that the cognitive impairment prediction result is closer to the label assigned to the one different user associated with the current set of inputs;

labeling a new user as having a cognitive impairment based on an application of the trained machine learning model to a new set of inputs corresponding to the new user, the new set of inputs including one or more interactions of the user with a keyboard, of a computing device, as an input device,

wherein the labeled cognitive impairment is at least one of a brain injury, a learning disability, a stroke, post-traumatic stress disorder, depression, schizophrenia, or any combination thereof; and

based on the labeling of the new user as having the cognitive impairment, applying one or more adaptive measures to the computing device, the one or more adaptive measures including automatically switching, by the computing device, to a microphone as the input device of the computing device, through which one or more further interactions of the user.

13. The computing system of claim 12 ,

wherein at least some of the multiple sets of inputs are each associated with one different user who has a pre-classification of having or not having a cognitive impairment; and

wherein the applying the unsupervised learning classifier to the sets of inputs further includes:

analyzing, for the inputs with pre-classifications, ratios of correctly classified inputs to incorrectly classified inputs in each of multiple clusters of classifications resulting when the classification iterations end;

based on the analyzing, re-performing a set of classification iterations with different ones of the multiple dimensions until the ratios of correctly classified inputs to incorrectly classified inputs in each of the multiple clusters are above a threshold level of cluster homogeneousness; and

in response to the ratios being above the threshold level of cluster homogeneousness, performing the labeling of the multiple of the users associated with the sets of inputs by labeling each user in a cluster with the pre-classification most prevalent in that cluster.

14. The method of claim 1 , wherein one or more iterations further include:

identifying a false positive or a false negative cognitive impairment determination result; and

excluding the current set of inputs from further training of the machine learning model.

15. The method of claim 14 , wherein the false positive or the false negative cognitive impairment determination result is identified by one or both of:

comparing a confidence value, generated by the machine learning model, to a threshold value; and/or

comparing a progression of the cognitive impairment determination result, over multiple iterations, to one or more historical patterns of cognitive impairment progression.

16. The method of claim 1 , further comprising:

wherein the one or more automated adaptive measures further include updating a user interface for the new user.

17. The method of claim 1 ,

wherein the computer UI interaction pattern inputs include user interactions, with a user device, shown on a display of the user device.

18. The method of claim 1 , wherein the one or more adaptive measures includes at least one of:

disabling user interfaces and functionality tagged as being complicated or dense,

adjusting criteria for recognition of haptic motions,

enabling simpler user interfaces, or

any combination thereof.

Assignments (2)

References Cited (45)