Granted Patent B2

US 12,737,433 · App. 18/941,802 · Granted Sep 15, 2026

Inventors: Sheng Ren (Irvine, CA); Michael P. Lahm (Minneapolis, MN); Lilian Z. Lingcaro (Mandaue, PH)

Assignee: Optum, Inc.

G06F18/23G06F18/213

Abstract

Claims (92)

1 . A computer-implemented method comprising:

generating, by one or more processors, one or more states based on a candidate set of states corresponding to a highest silhouette score and a minimum quantity of samples with a corresponding event combination failing to satisfy a combination frequency threshold, the one or more states representing event combinations associated with one or more samples;

applying, by the one or more processors, a feature generation algorithm to the one or more states, wherein applying the feature generation algorithm includes:

determining a state sequence associated with a sample of the one or more samples that satisfies an occurrence threshold, the state sequence including a state from the one or more states, and

generating one or more timing features for the state sequence;

clustering, by the one or more processors executing a machine-learned model, the sample associated with the state sequence into a cluster of a set of clusters based on the state sequence and the one or more timing features; and

generating, by the one or more processors for the sample, a data object indicating one or more outcomes associated with the cluster that includes the sample.

2 . The computer-implemented method of claim 1 , wherein the machine-learned model is an unsupervised decision tree model, wherein the state sequence is included within a set of state sequences determined by the feature generation algorithm as satisfying the occurrence threshold, and wherein clustering the sample further comprises:

iteratively determining, by the one or more processors, whether the sample includes one or more state sequences that (i) include a larger number of states than at least one other state sequence included in the set of state sequences and (ii) occur more frequently within a set of states of the one or more states than at least one other state sequence included in the set of state sequences;

determining, by the one or more processors, that:

a node of the unsupervised decision tree model comprising the sample satisfies a size threshold, or

an occurrence frequency, within the set of states, of remaining state sequences in the set of state sequences included in at least one sample within the node that have not been evaluated as part of the iterative determining fails to satisfy a sequence frequency threshold; and

clustering, by the one or more processors and based on the determining, the sample into the cluster of the set of clusters that corresponds with the node.

3 . The computer-implemented method of claim 2 , further comprising:

iteratively applying, by the one or more processors, a density-based clustering algorithm to (i) samples comprising the node and (ii) timing features of the one or more state sequences evaluated as part of the iterative determining to generate one or more sets of subclusters, wherein iterative applications of the density-based clustering algorithm utilize one or more unique hyperparameter combinations;

determining, by the one or more processors for the one or more sets of subclusters, a silhouette score indicating a match quality of the sample to one or more subclusters; and

clustering, by the one or more processors, the sample into a subcluster of the one or more subclusters based on a set of subclusters corresponding to a highest silhouette score.

4 . The computer-implemented method of claim 1 , wherein the machine-learned model is a gradient boosting machine (GBM) model, wherein the state sequence is included within a set of state sequences determined by the feature generation algorithm as satisfying the occurrence threshold, wherein the one or more timing features are included within a set of timing features generated by the feature generation algorithm, and wherein clustering the sample further comprises:

training, by the one or more processors, the GBM using (i) the set of state sequences, (ii) the set of timing features, and (iii) one or more labels associated with historical outcomes, wherein the GBM is trained to output an outcome of interest likelihood for the sample; and

stratifying, by the one or more processors based on the outcome of interest likelihood, the sample into a bin of a set of bins associated with outcome of interest likelihoods, the bin corresponding with the cluster of the set of clusters, wherein boundaries of the set of bins are based on:

a predetermined range of outcome of interest likelihood values, or

one or more local minima of a distribution of the outcome of interest likelihood values.

5 . The computer-implemented method of claim 4 , wherein the GBM comprises one or more decision trees, wherein the one or more samples have associated decision paths along the one or more decision trees, and wherein the computer-implemented method further comprises:

determining, by the one or more processors, a listing of terminal nodes for the one or more decision trees;

determining, by the one or more processors, a set of pairwise distances between terminal nodes of the listing of terminal nodes, wherein a pairwise distance of the set of pairwise distances represents a fraction of decision paths along a decision tree of the one or more decision trees not shared by a first decision path leading to a first terminal node and a second decision path leading to a second terminal node; and

storing, by the one or more processors, the listing of terminal nodes and the set of pairwise distances in a storage location.

6 . The computer-implemented method of claim 5 , further comprising:

querying, by the one or more processors, the storage location to determine pairwise distances between the sample and one or more other samples included in the bin of the set of bins;

iteratively applying, by the one or more processors, a density-based clustering algorithm to (i) samples included in the bin of the set of bins and (ii) the pairwise distances to generate one or more sets of subclusters, wherein iterative applications of the density-based clustering algorithm utilize one or more unique hyperparameter combinations;

determining, by the one or more processors for the one or more sets of subclusters, a silhouette score indicating a match quality of the sample to one or more subclusters; and

clustering, by the one or more processors, the sample into a subcluster of the one or more subclusters based on a set of subclusters corresponding to a highest silhouette score.

7 . The computer-implemented method of claim 1 , wherein the feature generation algorithm comprises a pattern mining method, and wherein applying the feature generation algorithm further comprises:

applying, by the one or more processors, the pattern mining method to a set of states of the one or more states to determine whether one or more state sequences from the set of states satisfies the occurrence threshold, the set of states corresponding to the sample;

determining, by the one or more processors, that the state sequence satisfies the occurrence threshold based on the state sequence occurring within one or more other sets of states corresponding to one or more other samples of the one or more samples; and

generating, by the one or more processors, the one or more timing features associated with the state sequence based on timestamp data of event combinations associated with the state sequence.

8 . The computer-implemented method of claim 7 , wherein the one or more timing features includes at least one of: (i) an initial appearance value, (ii) a frequency value, or (iii) an average duration value.

9 . The computer-implemented method of claim 1 , wherein the event combinations associated with the one or more samples comprise one or more events, and wherein the computer-implemented method further comprises:

determining, by the one or more processors based on the event combinations, a unique event combination of the one or more events;

generating, by the one or more processors executing an encoder, an embedding for the unique event combination;

applying, by the one or more processors, a dimension reduction algorithm to the embedding to generate a reduced-dimension embedding representing the unique event combination; and

applying, by the one or more processors, a density-based clustering algorithm to the reduced-dimension embedding and one or more other reduced-dimension embeddings representing other unique event combinations to generate the one or more states.

10 . The computer-implemented method of claim 9 , further comprising:

determining, by the one or more processors, a cosine distance of the reduced-dimension embedding from the one or more other reduced-dimension embeddings;

iteratively applying, by the one or more processors, the density-based clustering algorithm to (i) the reduced-dimension embedding and (ii) the one or more other reduced-dimension embeddings to generate one or more candidate sets of states, wherein iterative applications of the density-based clustering algorithm utilize one or more unique hyperparameter combinations; and

determining, by the one or more processors for the one or more candidate sets of states, (i) a silhouette score indicating a match quality of the reduced-dimension embedding to a candidate state and (ii) a quantity of samples with a corresponding event combination failing to satisfy a combination frequency threshold.

11 . The computer-implemented method of claim 1 , further comprising:

receiving, at the one or more processors, sample data comprising a first sample formatted in accordance with a first format and a second sample formatted in accordance with a second format that is different from the first format;

extracting, by the one or more processors executing a standardization algorithm, (i) an identification number, (ii) timestamp data, or (iii) event data from the first sample and the second sample;

generating, by the one or more processors, a set of standardized samples comprising the extracted data from the first sample and the second sample, the set of standardized samples being formatted in accordance with a standardized format; and

determining, by the one or more processors, the event combinations based on the set of standardized samples.

12 . The computer-implemented method of claim 1 , wherein the machine-learned model includes an unsupervised decision tree model and a trained GBM.

13 . The computer-implemented method of claim 1 , wherein the data object includes (i) a longest shared state sequence between the sample and at least one other sample within the cluster, (ii) metrics corresponding to the cluster, or (iii) a sample size associated with the cluster.

14 . A system comprising:

one or more processors; and

at least one memory storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

generating one or more states representing event combinations associated with one or more samples based on a candidate set of states corresponding to a highest silhouette score and a minimum quantity of samples with a corresponding event combination failing to satisfy a combination frequency threshold;

applying a feature generation algorithm to the one or more states, wherein applying the feature generation algorithm includes:

determining a state sequence associated with a sample of the one or more samples that satisfies an occurrence threshold, the state sequence including a state from the one or more states, and

generating one or more timing features for the state sequence;

clustering, by executing a machine-learned model, the sample associated with the state sequence into a cluster of a set of clusters based on the state sequence and the one or more timing features; and

generating a data object indicating one or more outcomes associated with the cluster that includes the sample.

15 . The system of claim 14 , wherein the machine-learned model is an unsupervised decision tree model, wherein the state sequence is included within a set of state sequences determined by the feature generation algorithm as satisfying the occurrence threshold, and wherein clustering the sample further comprises:

iteratively determining whether the sample includes one or more state sequences that (i) include a larger number of states than at least one other state sequence included in the set of state sequences and (ii) occur more frequently within a set of states of the one or more states than at least one other state sequence included in the set of state sequences;

determining that:

a node of the unsupervised decision tree model comprising the sample satisfies a size threshold, or

an occurrence frequency, within the set of states, of remaining state sequences in the set of state sequences included in at least one sample within the node that have not been evaluated as part of the iterative determining fails to satisfy a sequence frequency threshold; and

clustering, based on the determining, the sample into the cluster of the set of clusters that corresponds with the node.

16 . The system of claim 15 , wherein the processor-executable instructions, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

iteratively applying a density-based clustering algorithm to (i) samples comprising the node and (ii) timing features of the one or more state sequences evaluated as part of the iterative determining to generate one or more sets of subclusters, wherein iterative applications of the density-based clustering algorithm utilize one or more unique hyperparameter combinations;

determining, for the one or more sets of subclusters, a silhouette score indicating a match quality of the sample to one or more subclusters; and

clustering the sample into a subcluster of the one or more subclusters based on a set of subclusters corresponding to a highest silhouette score.

17 . The system of claim 14 , wherein the machine-learned model is a gradient boosting machine (GBM) model, wherein the state sequence is included within a set of state sequences determined by the feature generation algorithm as satisfying the occurrence threshold, wherein the one or more timing features are included within a set of timing features generated by the feature generation algorithm, and wherein clustering the sample further comprises:

training the GBM using (i) the set of state sequences, (ii) the set of timing features, and (iii) one or more labels associated with historical outcomes, wherein the GBM is trained to output an outcome of interest likelihood for the sample; and

stratifying, based on the outcome of interest likelihood, the sample into a bin of a set of bins associated with outcome of interest likelihoods, the bin corresponding with the cluster of the set of clusters, wherein boundaries of the set of bins are based on:

a predetermined range of outcome of interest likelihood values, or

one or more local minima of a distribution of the outcome of interest likelihood values.

18 . The system of claim 17 , wherein the GBM comprises one or more decision trees, wherein the one or more samples have associated decision paths along the one or more decision trees, and wherein the processor-executable instructions, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

determining a listing of terminal nodes for the one or more decision trees;

determining a set of pairwise distances between terminal nodes of the listing of terminal nodes, wherein a pairwise distance of the set of pairwise distances represents a fraction of decision paths along a decision tree of the one or more decision trees not shared by a first decision path leading to a first terminal node and a second decision path leading to a second terminal node; and

storing the listing of terminal nodes and the set of pairwise distances in a storage location.

19 . The system of claim 18 , wherein the processor-executable instructions, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

querying the storage location to determine pairwise distances between the sample and one or more other samples included in the bin of the set of bins;

iteratively applying a density-based clustering algorithm to (i) samples included in the bin of the set of bins and (ii) the pairwise distances to generate one or more sets of subclusters, wherein iterative applications of the density-based clustering algorithm utilize one or more unique hyperparameter combinations;

determining, for the one or more sets of subclusters, a silhouette score indicating a match quality of the sample to one or more subclusters; and

clustering the sample into a subcluster of the one or more subclusters based on a set of subclusters corresponding to a highest silhouette score.

20 . One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

generating one or more states representing event combinations associated with one or more samples based on a candidate set of states corresponding to a highest silhouette score and a minimum quantity of samples with a corresponding event combination failing to satisfy a combination frequency threshold;

applying a feature generation algorithm to the one or more states, wherein applying the feature generation algorithm includes:

determining a state sequence associated with a sample of the one or more samples that satisfies an occurrence threshold, the state sequence including a state from the one or more states, and

generating one or more timing features for the state sequence;

clustering, by executing a machine-learned model, the sample associated with the state sequence into a cluster of a set of clusters based on the state sequence and the one or more timing features; and

generating a data object indicating one or more outcomes associated with the cluster that includes the sample.

Assignments (1)

Continuity (1)

References Cited (29)