Granted Patent B1

US 12,555,033 · App. 17/558,333 · Granted Feb 17, 2026

Inventors: Nolan Serrao (Plano, TX); Keegan Patrick Hayes (Whitestown, IN); Arthur Quentin Smith (Fredericksburg, TX); John Andrew Weems (San Antonio, TX); Salvador Adrian Bretado (San Antonio, TX); Justin Royell Nash (Little Elm, TX); David Joaquin Harris (San Antonio, TX); Carol Lyn Lawrence (Fair Oaks Ranch, TX); Theresa Marie Matowitz (San Antonio, TX); Courtney St. Martin (Duluth, GA)

Assignee: United Services Automobile Association (USAA)

G06N20/00A61B5/165G06F18/2113

Abstract

Claims (78)

1 . A method for analyzing cognitive impairment determinations, in users of a computing system, for false positives and/or false negatives, the method comprising:

training a machine learning model to identify users with cognitive impairments by A) classifying user data instances each as being either associated with or not associated with a cognitive impairment, B) validating at least some of the classified user data instances by comparing the at least some of the classified user data instances with known cognitive impairment or non-cognitive impairment labels for the at least some of the classified user data instances, and C) based on the validating the at least some of the classified user data instances, labeling each user data instance as being associated with a corresponding user having or not having a cognitive impairment;

generating a cognitive impairment label for a user based on a cognitive impairment prediction from applying the trained machine learning model to a set of inputs corresponding to the user, the set of inputs including representations of content of communication histories associated with the user;

determining a weighted combination of values produced by analyzing modifying factors for the cognitive impairment prediction, the analyzing including:

comparing a confidence score, produced for the cognitive impairment prediction generated by the trained machine learning model, to a first threshold;

determining whether the cognitive impairment prediction aligns with a determined pattern of cognitive impairment predictions; and

determining whether any of the set of inputs, used in generating the cognitive impairment prediction, correspond to extenuating external factors determined to render the use of that input unsuitable;

determining that the cognitive impairment prediction, generated by the trained machine learning model, is a result of one or more deviations based on the weighted combination not exceeding a second threshold; and

in response to the analyzing having found that the cognitive impairment label is the result of the one or more deviations:

causing the machine learning model to re-execute, excluding or reweighting some of the inputs used by the machine learning model for the previous cognitive impairment prediction.

2 . The method of claim 1 ,

wherein the determined pattern of cognitive impairment is a pattern determined from a typical progression of cognition impairment predictions for users specified as having been accurately estimated to have cognitive impairments; and

wherein the determining whether the cognitive impairment prediction aligns with the determined pattern of cognitive impairment predictions includes comparing multiple cognitive impairment predictions for the user, made over a period of at least multiple months, to the typical progression of cognition impairment predictions.

3 . The method of claim 1 ,

wherein the determined pattern of cognitive impairment is a function fit to a progression of cognition impairments made for the user over a period of at least multiple months; and

wherein the determining whether the cognitive impairment prediction aligns with the determined pattern of cognitive impairment predictions includes determining whether a value of the cognitive impairment prediction is within a threshold distance of the y-value of the function for an x-value time that the prediction was made.

4 . The method of claim 1 ,

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether the content of communication histories corresponds to an extenuating external factor that the user is a non-native English speaker.

5 . The method of claim 1 ,

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether an atypical computer UI interaction pattern input corresponds to a recent update to the computing system UI.

6 . The method of claim 1 ,

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether an input corresponds to an identified user stressor occurring when that input was recorded.

7 . The method of claim 1 ,

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether an input corresponds to an identified user stressor occurring when that input was recorded; and

wherein the identified user stressor includes one or more of: an identified pandemic, an identified stock market change, an identified political upheaval, an identified catastrophe in an area of the user, or any combination thereof.

8 . The method of claim 1 ,

wherein, in response to the analyzing having found that the cognitive impairment label is the result of the one or more deviations, the method includes the causing the machine learning model to re-execute excluding one or more inputs associated with the determined deviations.

9 . The method of claim 1 , wherein weights for the values are determined by a regression analysis using i) predictions of the classified user data instances as being either associated with or not associated with a cognitive impairment and ii) corresponding accuracy determinations.

10 . The method of claim 1 , wherein the content of communication histories includes logs of communications associated with the user.

11 . The method of claim 1 , wherein the set of inputs further includes variations between communication instances associated with the user over time, the variations including one or more of identified emotional state changes, language structure changes, word choice changes, or any combination thereof.

12 . A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for analyzing cognitive impairment determinations for false positives and/or false negatives, the process comprising:

training a machine learning model to identify users with cognitive impairments by A) classifying user data instances each as being either associated with or not associated with a cognitive impairment, B) validating at least some of the classified user data instances by comparing the at least some of the classified user data instances with known cognitive impairment or non-cognitive impairment labels for the at least some of the classified user data instances, and C) based on the validating the at least some of the classified user data instances, labeling each user data instance as being associated with a corresponding user having or not having a cognitive impairment;

generating a cognitive impairment label for a user based on a cognitive impairment prediction from applying the trained machine learning model to a set of inputs corresponding to the user, the set of inputs including representations of content of communication histories associated with the user;

determining a weighted combination of values produced by analyzing modifying factors for the cognitive impairment prediction, the analyzing including one two or more of:

comparing a confidence score, produced for the cognitive impairment prediction generated by the trained machine learning model, to a first threshold;

determining whether the cognitive impairment prediction aligns with a determined pattern of cognitive impairment predictions;

determining whether any of the set of inputs, used in generating the cognitive impairment prediction, correspond to extenuating external factors determined to render the use of that input unsuitable; or

any combination thereof;

determining that the cognitive impairment prediction, generated by the trained machine learning model, is a result of one or more deviations based on the weighted combination not exceeding a second threshold; and

in response to the analyzing having found that the cognitive impairment label is the result of the one or more deviations:

causing the machine learning model to re-execute, excluding or reweighting some of the inputs used by the machine learning model for the previous cognitive impairment prediction.

13 . The computer-readable storage medium of claim 12 ,

wherein the analyzing includes the determining whether the cognitive impairment prediction aligns with the determined pattern of cognitive impairment predictions;

wherein the determined pattern of cognitive impairment is a function fit to a progression of cognition impairments made for the user over a period of at least multiple months; and

wherein the determining whether the cognitive impairment prediction aligns with the determined pattern of cognitive impairment predictions includes determining whether a value of the cognitive impairment prediction is within a threshold distance of the y-value of the function for an x-value time that the prediction was made.

14 . The computer-readable storage medium of claim 12 ,

wherein the analyzing includes the determining whether any of the set of inputs correspond to extenuating external factors; and

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether an atypical computer UI interaction pattern input corresponds to a recent update to the computing system UI.

15 . The computer-readable storage medium of claim 12 ,

wherein the analyzing includes the determining whether any of the set of inputs correspond to extenuating external factors; and

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether an input corresponds to an identified user stressor occurring when that input was recorded.

16 . The computer-readable storage medium of claim 12 ,

wherein the analyzing includes the determining whether any of the set of inputs correspond to extenuating external factors;

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether an input corresponds to an identified user stressor occurring when that input was recorded; and

wherein the identified user stressor includes one or more of: an identified pandemic, an identified stock market change, an identified political upheaval, an identified catastrophe in an area of the user, or any combination thereof.

17 . A computing system for analyzing cognitive impairment determinations for false positives and/or false negatives, the computing system comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:

training a machine learning model to identify users with cognitive impairments by A) classifying user data instances each as being either associated with or not associated with a cognitive impairment, B) validating at least some of the classified user data instances by comparing the at least some of the classified user data instances with known cognitive impairment or non-cognitive impairment labels for the at least some of the classified user data instances, and C) based on the validating the at least some of the classified user data instances, labeling each user data instance as being associated with a corresponding user having or not having a cognitive impairment;

generating a cognitive impairment label for a user based on a cognitive impairment prediction from applying the trained machine learning model to a set of inputs corresponding to the user, the set of inputs including representations of content of communication histories associated with the user;

determining a weighted combination of values produced by analyzing modifying factors for the cognitive impairment prediction, the analyzing including two or more of:

comparing a confidence score, produced for the cognitive impairment prediction generated by the trained machine learning model, to a first threshold;

determining whether the cognitive impairment prediction aligns with a determined pattern of cognitive impairment predictions;

determining whether any of the set of inputs, used in generating the cognitive impairment prediction, correspond to extenuating external factors determined to render the use of that input unsuitable; or

any combination thereof;

determining the cognitive impairment prediction, generated by the trained machine learning model, is a result of one or more deviations based on the weighted combination not exceeding a second threshold; and

in response to the analyzing having found that the cognitive impairment label is the result of the one or more deviations:

causing the machine learning model to re-execute, excluding or reweighting some of the inputs used by the machine learning model for the previous cognitive impairment prediction.

18 . The computing system of claim 17 ,

wherein the analyzing includes the determining whether the cognitive impairment prediction aligns with the determined pattern of cognitive impairment predictions;

wherein the determined pattern of cognitive impairment is a pattern determined from a typical progression of cognition impairment predictions for users specified as having been accurately estimated to have cognitive impairments; and

wherein the determining whether the cognitive impairment prediction aligns with the determined pattern of cognitive impairment predictions includes comparing multiple cognitive impairment predictions for the user, made over a period of at least multiple months, to the typical progression of cognition impairment predictions.

19 . The computing system of claim 17 ,

wherein the analyzing includes the determining whether any of the set of inputs correspond to extenuating external factors; and

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether the content of the communication histories corresponds to an extenuating external factor that the user is a non-native English speaker.

20 . The computing system of claim 17 ,

wherein the analyzing includes the determining whether any of the set of inputs correspond to extenuating external factors; and

wherein the determining whether any of the set of inputs used in generating the cognitive impairment prediction correspond to extenuating external factors includes identifying whether an atypical computer UI interaction pattern input corresponds to a recent update to the computing system UI.

Assignments (2)

References Cited (67)