IP Library Granted Patent US 12,450,508
Granted Patent B2
US 12,450,508 · App. 16/938,416 · Granted Oct 21, 2025

Categorical input machine learning models

Inventors: Lorcan B. Mac Manus (Kildare, IE); Peter Cogan (Dublin, IE); Conor Breen (Dublin, IE)
Assignee: Optum Services (Ireland) Limited
G06N5/045G06N5/02G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,450,508
App. No.
16/938,416
Granted
Oct 21, 2025
Kind
B2
Abstract

There is a need for more effective and efficient predictive data analysis based at least in part on categorical input data. This need can be addressed by, for example, solutions for performing predictive data analysis that utilize at least one of categorical level merging, mutual-information-based feature filtering, feature-correlation-based feature filtering to generate training data feature value arrangements, as well as training and using categorical input machine learning models trained using the training data feature value arrangements.

Claims (53)

1. A computer-implemented method comprising:

receiving, by one or more processors, a request comprising a prediction input data entry;

inputting, by the one or more processors, the prediction input data entry to a categorical input machine learning model to receive an inferred prediction for the prediction input data entry as an output of the categorical input machine learning model, wherein:

(i) the categorical input machine learning model defines a plurality of training data feature value arrangements, and

(ii) the inferred prediction is generated based on a matching subset of the plurality of training data feature value arrangements that correspond to the prediction input data entry;

identifying, by the one or more processors, a selected training data feature value arrangement from the matching subset of the plurality of training data feature value arrangements that correspond to the prediction input data entry based on a comparison between (a) a mean-wide cross-entropy measure corresponding to the selected training data feature value arrangement and (b) a plurality of mean-wide cross-entropy measures respectively corresponding to the plurality of training data feature value arrangements within the matching subset; and

in response to receipt of the request, providing, by the one or more processors, the inferred prediction with explanatory metadata descriptive of the selected training data feature value arrangement.

2. The computer-implemented method of claim 1 , further comprising:

generating the plurality of mean-wide cross-entropy measures that respectively correspond to the plurality of training data feature value arrangements within the matching subset.

3. The computer-implemented method of claim 1 , further comprising:

generating a plurality of weighted point probability estimation values that respectively correspond to the plurality of training data feature value arrangements within the matching subset; and

generating the inferred prediction by combining the plurality of weighted point probability estimation values.

4. The computer-implemented method of claim 1 , wherein:

the inferred prediction is determined based at least in part on the selected training data feature value arrangement of the plurality of training data feature value arrangements within the matching subset, and

generating the explanatory metadata is performed based at least in part on the selected training data feature value arrangement.

5. The computer-implemented method of claim 1 , wherein the explanatory metadata comprises a model complexity measure for the selected training data feature value arrangement.

6. The computer-implemented method of claim 1 , further comprising enabling display of a prediction output user interface.

7. The computer-implemented method of claim 6 , wherein the prediction output user interface is configured to display the inferred prediction and the explanatory metadata.

8. The computer-implemented method of claim 1 , wherein the selected training data feature value arrangement is identified based on the plurality of mean-wide cross-entropy measures respectively corresponding to the matching subset of the plurality of training data feature value arrangements.

9. A system comprising memory and one or more processors communicatively coupled to the memory, the one or more processors configured to:

receive a request comprising a prediction input data entry;

input the prediction input data entry to a categorical input machine learning model to receive an inferred prediction for the prediction input data entry as an output of the categorical input machine learning model, wherein:

(i) the categorical input machine learning model defines a plurality of training data feature value arrangements, and

(ii) the inferred prediction is generated based on a matching subset of the plurality of training data feature value arrangements that correspond to the prediction input data entry;

identify a selected training data feature value arrangement from the matching subset of the plurality of training data feature value arrangements that correspond to the prediction input data entry based on a comparison between (a) a mean-wide cross-entropy measure corresponding to the selected training data feature value arrangement and (b) a plurality of mean-wide cross-entropy measures respectively corresponding to the plurality of training data feature value arrangements within the matching subset; and

in response to receipt of the request, provide the inferred prediction with explanatory metadata descriptive of the selected training data feature value arrangement.

10. The system of claim 9 , wherein the one or more processors are further configured to:

generate the plurality of mean-wide cross-entropy measures that respectively correspond to the plurality of training data feature value arrangements within the matching subset.

11. The system of claim 9 , wherein the one or more processors are further configured to:

generate a plurality of weighted point probability estimation values that respectively correspond to the plurality of training data feature value arrangements within the matching subset; and

generate the inferred prediction by combining the plurality of weighted point probability estimation values.

12. The system of claim 9 , wherein:

the inferred prediction is determined based at least in part on the selected training data feature value arrangement of the plurality of training data feature value arrangements within the matching subset, and

the one or more processors are further configured to generate the explanatory metadata based at least in part on the selected training data feature value arrangement.

13. The system of claim 9 , wherein the explanatory metadata comprises a model complexity measure for the selected training data feature value arrangement.

14. The system of claim 9 , wherein the one or more processors are further configured to enable display of a prediction output user interface.

15. The system of claim 14 , wherein the prediction output user interface is configured to display the inferred prediction and the explanatory metadata.

16. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to:

receive a request comprising a prediction input data entry;

input the prediction input data entry to a categorical input machine learning model to receive an inferred prediction for the prediction input data entry as an output of the categorical input machine learning model, wherein:

(i) the categorical input machine learning model defines a plurality of training data feature value arrangements, and

(ii) the inferred prediction is generated based on a matching subset of the plurality of training data feature value arrangements that correspond to the prediction input data entry;

identify a selected training data feature value arrangement from the matching subset of the plurality of training data feature value arrangements that correspond to the prediction input data entry based on a comparison between (a) a mean-wide cross-entropy measure corresponding to the selected training data feature value arrangement and (b) a plurality of mean-wide cross-entropy measures respectively corresponding to the plurality of training data feature value arrangements within the matching subset; and

in response to receipt of the request, provide the inferred prediction with explanatory metadata descriptive of the selected training data feature value arrangement.

17. The one or more non-transitory computer-readable storage media of claim 16 , wherein the one or more processors are further configured to:

generate the plurality of mean-wide cross-entropy measures that respectively correspond to the plurality of training data feature value arrangements within the matching subset.

18. The one or more non-transitory computer-readable storage media of claim 16 , wherein the one or more processors are further configured to:

generate a plurality of weighted point probability estimation values that respectively correspond to the plurality of training data feature value arrangements within the matching subset; and

generate the inferred prediction by combining the plurality of weighted point probability estimation values.

19. The one or more non-transitory computer-readable storage media of claim 16 , wherein:

the inferred prediction is determined based at least in part on the selected training data feature value arrangement of the plurality of training data feature value arrangements within the matching subset, and

the one or more processors are further configured to generate the explanatory metadata based at least in part on the selected training data feature value arrangement.

20. The one or more non-transitory computer-readable storage media of claim 16 , wherein the explanatory metadata comprises a model complexity measure for the selected training data feature value arrangement.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 24, 2020
From: MAC MANUS, LORCAN B.; COGAN, PETER; BREEN, CONOR
To: OPTUM SERVICES (IRELAND) LIMITED
Reel/Frame 053306/0800 →
Continuity (1)
Related Publication 20220027769A1 · Jan 27, 2022
References Cited (39)
US 6275806B1 · Petrushin · 2001 [cited by applicant]
US 7318051B2 · Weston et al. · 2008 [cited by applicant]
US 7933851B2 · Tuv · 2011 [cited by applicant]
US 11481647B1 · Bataineh et al. · 2022 [cited by applicant]
US 20060161403A1 · Jiang et al. · 2006 [cited by applicant]
US 20100049538A1 · Frazer et al. · 2010 [cited by applicant]
US 20120035859A1 · Thomas · 2012 [cited by applicant]
US 20130262140A1 · Friedlander · 2013 [cited by examiner]
US 20150332157A1 · Baughman et al. · 2015 [cited by applicant]
US 20160078361A1 · Brueckner et al. · 2016 [cited by applicant]
US 20190221311A1 · Takeuchi · 2019 [cited by examiner]
US 20190286724A1 · Kudo · 2019 [cited by examiner]
US 20190303994A1 · Sesia et al. · 2019 [cited by applicant]
US 20190311301A1 · Pyati · 2019 [cited by applicant]
US 20190392268A1 · Tariq · 2019 [cited by examiner]
US 20210174922A1 · Noro · 2021 [cited by examiner]
US 20230005574A1 · Sparks · 2023 [cited by examiner]
Holzinger, Andreas, et al. “What do we need to build explainable AI systems for the medical domain?.” arXiv preprint arXiv:1712.09923 (2017). (Year: 2017). [cited by examiner]
Wang, Yisen, et al. “Symmetric cross entropy for robust learning with noisy labels.” Proceedings of the IEEE/CVF international conference on computer vision. 2019. (Year: 2019). [cited by examiner]
NonFinal Office Action for U.S. Appl. No. 16/938,380, dated Aug. 16, 2023, (15 pages), Unite States Patent and Trademark Office, US. [cited by applicant]
NonFinal Office Action for U.S. Appl. No. 16/938,436, dated Aug. 22, 2023, (60 pages), United States Patent and Trademark Office, US. [cited by applicant]
NonFinal Office Action for U.S. Appl. No. 16/938,461, dated Aug. 22, 2023, (43 pages), United States Patent and Trademark Office, US. [cited by applicant]
NonFinal Office Action for U.S. Appl. No. 16/938,480, dated Aug. 16, 2023, (16 pages), United States Patent and Trademark Office, US. [cited by applicant]
Bock, Tim. “Feature Engineering For Categorical Variables,” DisplayR Blog Machine Learning, (11 pages), [article, online], [retrieved from the Internet Aug. 3, 2020] <URL: https://www.displayr.com/feature-engineering-fo… [cited by applicant]
Brownlee, Jason. “How To Perform Feature Selection With Categorical Data,” Machine Learning Mastery, Nov. 25, 2019, (33 pages), [article, online], [retrieved from the Internet Aug. 3, 2020] <URL: https://machinelearning… [cited by applicant]
Guo, Cheng et al. “Entity Embeddings of Categorical Variables,” arXiv preprint arXiv:1604.06737v1 [cs.LG] Apr. 22, 2016. (9 pages). [cited by applicant]
Final Rejection Mailed on Dec. 7, 2023 for U.S. Appl. No. 16/938,380, 17 page(s). [cited by applicant]
Final Rejection Mailed on Dec. 7, 2023 for U.S. Appl. No. 16/938,480, 18 page(s). [cited by applicant]
Final Rejection Mailed on Feb. 29, 2024 for U.S. Appl. No. 16/938,436, 28 page(s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Feb. 9, 2024 for U.S. Appl. No. 16/938,480, 8 page(s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Feb. 13, 2024 for U.S. Appl. No. 16/938,380, 8 page(s). [cited by applicant]
Final Rejection Mailed on Jun. 28, 2024 for U.S. Appl. No. 16/938,461, 20 page(s). [cited by applicant]
Frees, Edward W. Regression modeling with actuarial and financial applications. Cambridge University Press, 2009. (Year: 2009). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Jun. 26, 2024 for U.S. Appl. No. 16/938,436, 5 page(s). [cited by applicant]
Advisory Action (PTOL-303) Mailed on Sep. 5, 2024 for U.S. Appl. No. 16/938,461, 2 page(s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on Sep. 9, 2024 for U.S. Appl. No. 16/938,436, 2 page(s). [cited by applicant]
Knuth, Donald E. “The Art of Computer Programming: Fundamental Algorithms”, vol. 1. Addison-Wesley Professional, 1997. pp. 124-143. (Year: 1997). [cited by applicant]
Non-Final Rejection Mailed on Dec. 4, 2024 for U.S. Appl. No. 16/938,461, 25 page(s). [cited by applicant]
Notice of Allowance and Fees Due (PTOL-85) Mailed on May 20, 2025 for U.S. Appl. No. 16/938,461, 7 page(s). [cited by applicant]