IP Library › Granted Patent US 12,591,802
Granted Patent B2
US 12,591,802 · App. 17/491,240 · Granted Mar 31, 2026

Generating estimates by combining unsupervised and supervised machine learning

Inventors: Jingxian Liao (San Jose, CA); Wei Wang (San Jose, CA); Zhicheng Xue (Union City, CA)
Assignee: Intuit Inc.
G06N20/00G06F18/2148G06F18/24137
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,591,802
App. No.
17/491,240
Granted
Mar 31, 2026
Kind
B2
Abstract

A method may include obtaining a cluster. The cluster may include a subset of reference entities. The method may further include calculating distances between features of a target entity and features of the subset of reference entities, selecting, based on the distances, peer entities from the subset, and generating an estimated value of a metric. The generating may include applying, to the features of the target entity, a machine learning model trained using training data including values of the features for the peer entities labeled with a value of the metric. The method may further include presenting the estimated value of the metric.

Claims (102)

1 . A method comprising:

obtaining a first cluster of a first plurality of clusters, wherein the first cluster comprises a first subset of a plurality of reference entities;

calculating a first plurality of distances between a first plurality of features of a target entity and the first plurality of features of the first subset of the plurality of reference entities;

selecting, from the first subset, a first plurality of peer entities that are closest, within the first subset, to the target entity according to the first plurality of distances, each peer entity in the first plurality of peer entities being labeled with a corresponding value of a metric of the peer entity;

training, after selecting, a machine learning model specifically for the target entity, wherein the machine learning model is trained to estimate, as output, the corresponding value of the metric of the peer entity using, as input, values of the first plurality of features from only the first plurality of peer entities;

generating, by applying the machine learning model to the first plurality of features of the target entity, and after training the machine learning model specifically for the target entity, a first estimated value of the metric for the target entity; and

presenting the first estimated value of the metric.

2 . The method of claim 1 , wherein the first plurality of clusters comprises a plurality of centroids, and wherein the target entity and the plurality of reference entities are comprised by a plurality of entities, the method further comprising:

generating the first plurality of features for the plurality of entities;

dividing, using the first plurality of features, the plurality of reference entities into the first plurality of clusters;

calculating a second plurality of distances between the first plurality of features of the target entity and the plurality of centroids; and

selecting, using the second plurality of distances, the first cluster.

3 . The method of claim 2 , wherein the first plurality of features comprises a plurality of transaction aggregations and a plurality of operational features, the method further comprising:

aggregating the plurality of transaction aggregations and the plurality of operational features to obtain plurality of aggregated features,

wherein the dividing is based on the plurality of aggregated features.

4 . The method of claim 3 , wherein the plurality of entities is associated with a plurality of transactions, the method further comprising:

dividing the plurality of transactions into a plurality of categories;

selecting a subset of the plurality of categories that are relevant to the metric; and

generating the plurality of transaction aggregations by aggregating the plurality of transactions in the subset of the plurality of categories.

5 . The method of claim 2 ,

wherein the first plurality of features comprises a plurality of entity attributes, and

wherein the dividing is based on a subset of the plurality of entity attributes.

6 . The method of claim 2 , wherein the first plurality of features is generated for a first time window, the method further comprising:

generating a second plurality of features for the plurality of entities for a second time window;

dividing, using the second plurality of features, the plurality of reference entities into a second plurality of clusters, wherein the second plurality of clusters comprises a second cluster comprising a second subset of the plurality of reference entities;

calculating a third plurality of distances between a second plurality of features of the target entity and the second plurality of features of the second subset;

selecting, from the second subset and using the third plurality of distances, a second plurality of peer entities;

generating, by applying the machine learning model to the second plurality of features of the target entity, a second estimated value of the metric; and

combining the first estimated value and the second estimated value.

7 . The method of claim 6 , further comprising:

further training the machine learning model using second training data comprising a plurality of values of the second plurality of features for the second plurality of peer entities labeled with a value of the metric.

8 . The method of claim 1 , further comprising:

receiving a target entity request, wherein the first cluster is obtained in response to receiving the target entity request; and

recommending a response to the target entity request using the first estimated value of the metric.

9 . A system comprising:

a computer processor;

a repository configured to store:

a first plurality of clusters comprising a first cluster comprising a first subset of a plurality of reference entities; and

a metric estimation manager executing on the computer processor and configured to:

calculate a first plurality of distances between a first plurality of features of a target entity and the first plurality of features of the first subset,

select, from the first subset, a first plurality of peer entities that are closest, within the first subset, to the target entity according to the first plurality of distances, each peer entity in the first plurality of peer entities being labeled with a corresponding value of a metric of the peer entity,

train, after selecting, a machine learning model specifically for the target entity, wherein the machine learning model is trained to estimate, as output, the corresponding value of the metric of the peer entity using, as input, values of the first plurality of features labeled from only the first plurality of peer entities,

generate, by applying the machine learning model to the first plurality of features of the target entity, and after training the machine learning model specifically for the target entity, a first estimated value of the metric for the target entity, and

present the first estimated value of the metric.

10 . The system of claim 9 , further comprising a feature generator executing on the computer processor,

wherein the first plurality of clusters comprises a plurality of centroids,

wherein the target entity and the plurality of reference entities are comprised by a plurality of entities,

wherein the feature generator is configured to generate the first plurality of features for the plurality of entities, and

wherein the metric estimation manager is further configured to:

divide, using the first plurality of features for the plurality of entities, the plurality of reference entities into the first plurality of clusters,

calculate a second plurality of distances between the first plurality of features of the target entity and the plurality of centroids, and

select, using the second plurality of distances, the first cluster.

11 . The system of claim 10 , wherein the first plurality of features comprises a plurality of transaction aggregations and a plurality of operational features, and wherein the feature generator is further configured to:

aggregate the plurality of transaction aggregations and the plurality of operational features to obtain a plurality of aggregated features, wherein the plurality of entities is divided into the first plurality of clusters based on the plurality of aggregated features.

12 . The system of claim 11 , further comprising a transaction categorizer executing on the computer processor,

wherein the plurality of entities is associated with a plurality of transactions,

wherein the transaction categorizer is configured to:

divide the plurality of transactions into a plurality of categories, and

select a subset of the plurality of categories that are relevant to the metric, and

wherein the metric estimation manager is further configured to generate the plurality of transaction aggregations by aggregating transactions in the subset of the categories.

13 . The system of claim 10 ,

wherein the first plurality of features comprises a plurality of entity attributes, and

wherein the plurality of entities is divided into the first plurality of clusters based on a subset of the plurality of entity attributes.

14 . The system of claim 10 ,

wherein the first plurality of features is generated for a first time window,

wherein the feature generator is further configured to generate a second plurality of features for the plurality of entities for a second time window, and

wherein the metric estimation manager is further configured to:

divide, using the second plurality of features, the plurality of reference entities into a second plurality of clusters, wherein the second plurality of clusters comprises a second cluster comprising a second subset of the plurality of reference entities;

calculate a third plurality of distances between a second plurality of features of the target entity and the second plurality of features of the second subset;

select, from the second subset and using the third plurality of distances, a second plurality of peer entities;

generate, by applying the machine learning model to the second plurality of features of the target entity, a second estimated value of the metric; and

combine the first estimated value and the second estimated value.

15 . The system of claim 14 , wherein the machine learning model is further trained using second training data comprising a plurality of values of the second plurality of features for the second plurality of peer entities labeled with a value of the metric.

16 . The system of claim 9 , wherein the metric estimation manager is further configured to:

receive a target entity request, wherein the first cluster is obtained in response to receiving the target entity request; and

recommend a response to the target entity request using the first estimated value of the metric.

17 . A method comprising:

sending a target entity request to a metric estimation manager configured to perform:

obtaining a first cluster of a first plurality of clusters, wherein the first cluster comprises a first subset of a plurality of reference entities,

calculating a first plurality of distances between a first plurality of features of a target entity and the first plurality of features of the first subset of the plurality of reference entities,

selecting, from the first subset, a first plurality of peer entities that are closest, within the first subset, to the target entity according to the first plurality of distances, each peer entity in the first plurality of peer entities being labeled with a corresponding value of a metric of the peer entity,

training, after selecting, a machine learning model specifically for the target entity, wherein the machine learning model is trained to estimate, as output, the corresponding value of the metric of the peer entity using, as input, values of the first plurality of features from only the first plurality of peer entities;

generating, by applying the machine learning model to the first plurality of features of the target entity, and after training the machine learning model specifically for the target entity, a first estimated value of the metric for the target entity, and

transmitting the first estimated value of the metric;

receiving, from the metric estimation manager, the first estimated value of the metric; and

presenting the first estimated value of the metric.

18 . The method of claim 17 , wherein the first plurality of clusters comprises a plurality of centroids, wherein the target entity and the plurality of reference entities are comprised by a plurality of entities, and wherein the metric estimation manager is further configured to perform:

generating a first plurality of transaction aggregations and a first plurality of operational features for the plurality of entities;

aggregating the first plurality of transaction aggregations and the first plurality of operational features to obtain a first plurality of aggregated features;

dividing, using the first plurality of aggregated features, the plurality of reference entities into the first plurality of clusters;

calculating a second plurality of distances between the first plurality of features of the target entity and the plurality of centroids; and

selecting, using the second plurality of distances, the first cluster.

19 . The method of claim 18 , wherein the first plurality of aggregated features is generated for a first time window, and wherein the metric estimation manager is further configured to perform:

generating a second plurality of transaction aggregations and a second plurality of operational features for the plurality of entities;

aggregating the second plurality of transaction aggregations and the second plurality of operational features to obtain a second plurality of aggregated features;

dividing, using the second plurality of aggregated features, the plurality of reference entities into a second plurality of clusters, wherein the second plurality of clusters comprises a second cluster comprising a second subset of the plurality of reference entities;

calculating a third plurality of distances between the second plurality of aggregated features of the target entity and the second plurality of aggregated features of the second subset;

selecting, from the second subset and using the third plurality of distances, a second plurality of peer entities;

generating, by applying the machine learning model to a second plurality of features of the target entity, a second estimated value of the metric; and

combining the first estimated value and the second estimated value.

20 . The method of claim 19 , wherein the metric estimation manager is further configured to perform:

further training the machine learning model using second training data comprising values of the second plurality of features for the second plurality of peer entities labeled with a value of the metric.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2024
From: LIAO, JINGXIAN; WANG, WEI; XUE, ZHICHENG
To: INTUIT INC.
Reel/Frame 067293/0210 →
Continuity (1)
Related Publication 20230101182A1 · Mar 30, 2023
References Cited (26)
US 10565471B1 · Abdi Taghi Abad · 2020 [cited by examiner]
US 20170228615A1 · Chavez · 2017 [cited by examiner]
US 20190354850A1 · Watson · 2019 [cited by examiner]
US 20210049428A1 · Huang · 2021 [cited by examiner]
US 20220092214A1 · Brannon · 2022 [cited by examiner]
US 20220108137A1 · Margolin · 2022 [cited by examiner]
CN 102254028A · 2011 [cited by examiner]
CN 108369659A · 2018 [cited by examiner]
CN 111079830A · 2020 [cited by examiner]
WO WO2021034932A1 · 2021 [cited by examiner]
Jason Brownlee, “K-Nearest Neighbors for Machine Learning”, published on Aug. 15, 2020 to https://machinelearningmastery.com/k-nearest-neighbors-for-machine-learning, retrieved Mar. 26, 2025. (Year: 2020). [cited by examiner]
Hauwen Liu, etc., “A New Feature Selection Method based on Clustering”, published via 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD) (vol. 2, 2011, pp. 965-969), retrieved Mar. 26, … [cited by examiner]
Johnnie C-N Chang, etc., “Clustering System Data using Aggregate Measures”, published via SysML18, Feb. 2018, Stanford, CA USA, 2018., retrieved on Mar. 26, 2025 to https://mlsys.org/Conferences/doc/2018/201.pdf. (Year:… [cited by examiner]
Antony Christopher, “K-Nearest Neighbor”, published Feb. 2, 2021 to https://medium.com/swlh/k-nearest-neighbor-ca2593d7a3c4, retrieved Mar. 26, 2025. (Year: 2021). [cited by examiner]
Aristides Gionis, etc., “Clustering Aggregation”, published via ACM Transactions on Knowledge Discovery from Data, vol. 1, No. 1, Article 4, Publication date: Mar. 2007, retrieved Mar. 26, 2025. (Year: 2007). [cited by examiner]
Hands on Machine Learning, “Chapter 21 Hierarchical Clustering”, published on Jul. 28, 2019 to https://bradleyboehmke.github.io/HOML/hierarchical.html, retrieved Mar. 26, 2025. (Year: 2019). [cited by examiner]
Ben Postance, “A guide to clustering large datasets with mixed data-types [updated]”, published on Mar. 25, 2021 to https://bpostance.github.io/posts/clustering-mixed-data, retrieved Mar. 26, 2025. (Year: 2021). [cited by examiner]
Edwin M. Knorr, etc., “Finding Aggregate Proximity Relationships and Commonalities in Spatial Data Mining”, published via IEEE Transactions on Knowledge and Data Engineering, vol. 8, No. 6, Dec. 1996, retrieved Mar. 26,… [cited by examiner]
AJ Maschinot, “Extending Contrastive Learning to the Supervised Setting”, published on Jun. 4, 2021 to https://research.google/blog/extending-contrastive-learning-to-the-supervised-setting, retrieved Sep. 26, 2025. (Yea… [cited by examiner]
Conglong Li, etc., “Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination”, published via SIGMOD '20: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data,… [cited by examiner]
Yibang Ruan, etc., “A nearest-neighbor search model for distance metric learning”, published via Information Sciences 552 (2021), pp. 261-277, retrieved Jan. 24, 2026. (Year: 2021). [cited by examiner]
Dezhong Yao, etc., “Sparse Online Relative Similarity Learning”, published on Apr. 15, 2021 to arXiv, retrieved Jan. 24, 2026. (Year: 2021). [cited by examiner]
“1.6 Nearest Neighbors”, retrieved via the wayback machine as of Aug. 12, 2021 for https://scikit-learn.org/stable/modules/neighbors.html, retrieved Jan. 24, 2026. (Year: 2021). [cited by examiner]
Marius Muja, etc., “Scalable Nearest Neighbor Algorithms for High Dimensional Data”, published via IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, No. 11, (Nov. 2014), retrieved Jan. 24, 2026. (… [cited by examiner]
Purushottam Kar, etc., “Similarity-based Learning via Data Driven Embeddings”, published on Dec. 22, 2011 to arXiv, retrieved Jan. 24, 2026. (Year: 2011). [cited by examiner]
Alexandr Andoni, etc., “Approximate Nearest Neighbor Search in High Dimensions”, published on Jun. 26, 2018 to arXiv, retrieved Jan. 24, 2026. (Year: 2018). [cited by examiner]