IP Library Granted Patent US 11,263,003
Granted Patent B1
US 11,263,003 · App. 17/122,415 · Granted Mar 1, 2022

Intelligent versioning of machine learning models

Inventors: Seng Chai Gan (Ashburn, VA); Shikhar Kwatra (San Jose, CA); Gandhi Sivakumar (Bentleigh, AU); Vinod A. Valecha (Pune, IN); Thomas Joseph Eunice (Studio City, CA)
Assignee: KYNDRYL, INC.
G06F8/71G06F16/219G06F16/285G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,263,003
App. No.
17/122,415
Granted
Mar 1, 2022
Kind
B1
Abstract

An embodiment includes specifying an original model to be trained by a machine learning platform using training data and an algorithm, wherein the training data includes a plurality of training input values upon which to make a prediction and a plurality of training output values representing target prediction values. The embodiment generates an updated model by training the original model to algorithmically approach the training output values as provided with the training data based on the plurality of input values. The embodiment generates an updated function map representative of output values of the updated model and versions the updated model according to a classification rule for assigning a version classification based on a comparison of the updated function map to an original function map representative of output values of the original model. The embodiment deploys the updated model into a production environment. to replace the original model.

Claims (37)

1. A computer-implemented method comprising:

specifying an original model to be trained by a machine learning platform using training data and an algorithm, wherein the training data includes a plurality of training input values upon which to make a prediction and a plurality of training output values representing target prediction values for respective input values;

generating an updated model by training the original model to algorithmically approach the training output values as provided with the training data based on the plurality of input values;

generating an updated function map representative of output values of the updated model;

versioning the updated model according to a classification rule for assigning a version classification based on a comparison of the updated function map to an original function map representative of output values of the original model; and

deploying the updated model into a production environment, to replace the original model.

2. The computer-implemented method of claim 1 , wherein the original model is a linear regression model.

3. The computer-implemented method of claim 1 , further comprising editing metadata of the updated model to include the version classification.

4. The computer-implemented method of claim 1 , further comprising detecting that the updated model is not a first generated model.

5. The computer-implemented method of claim 4 , further comprising, responsive to determining that the updated model is not a first generated model, retrieving the original function map from memory for the comparison to the updated function map.

6. The computer-implemented method of claim 5 , further comprising, responsive to the versioning of the updated model, storing the version classification of the updated model in the memory.

7. The computer-implemented method of claim 1 , wherein the generating of the updated function map includes performing an iterative process such that the iterative process determines input test values that cause the updated model to output the updated function map.

8. The computer-implemented method of claim 7 , wherein the iterative process comprises iteratively inputting input test values selected from a range of values until a boundary of the function map is detected.

9. The computer-implemented method of claim 1 , wherein the classification rule includes assigning the version classification based on whether the comparison of the updated function map to the original function map results in a difference that exceeds a specified threshold value.

10. The computer-implemented method of claim 9 , wherein the classification rule includes assigning the version classification of the updated model as a major revision responsive to detecting that the difference exceeds the specified threshold value.

11. A computer program product comprising one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by a processor to cause the processor to perform operations comprising:

specifying an original model to be trained by a machine learning platform using training data and an algorithm, wherein the training data includes a plurality of training input values upon which to make a prediction and a plurality of training output values representing target prediction values for respective input values;

generating an updated model by training the original model to algorithmically approach the training output values as provided with the training data based on the plurality of input values;

generating an updated function map representative of output values of the updated model;

versioning the updated model according to a classification rule for assigning a version classification based on a comparison of the updated function map to an original function map representative of output values of the original model; and

deploying the updated model into a production environment, to replace the original model.

12. The computer program product of claim 11 , wherein the stored program instructions are stored in a computer readable storage device in a data processing system, and wherein the stored program instructions are transferred over a network from a remote data processing system.

13. The computer program product of claim 11 , wherein the stored program instructions are stored in a computer readable storage device in a server data processing system, and wherein the stored program instructions are downloaded in response to a request over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system, further comprising:

program instructions to meter use of the program instructions associated with the request; and

program instructions to generate an invoice based on the metered use.

14. The computer program product of claim 11 , wherein the generating of the updated function map includes performing an iterative process such that the iterative process determines input test values that cause the updated model to output the updated function map.

15. The computer program product of claim 14 , wherein the iterative process comprises iteratively inputting input test values selected from a range of values until a boundary of the function map is detected.

16. The computer program product of claim 11 , wherein the classification rule includes assigning the version classification based on whether the comparison of the updated function map to the original function map results in a difference that exceeds a specified threshold value.

17. A computer system comprising a processor and one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by the processor to cause the processor to perform operations comprising:

specifying an original model to be trained by a machine learning platform using training data and an algorithm, wherein the training data includes a plurality of training input values upon which to make a prediction and a plurality of training output values representing target prediction values for respective input values;

generating an updated model by training the original model to algorithmically approach the training output values as provided with the training data based on the plurality of input values;

generating an updated function map representative of output values of the updated model;

versioning the updated model according to a classification rule for assigning a version classification based on a comparison of the updated function map to an original function map representative of output values of the original model; and

deploying the updated model into a production environment, to replace the original model.

18. The computer system of claim 17 , wherein the generating of the updated function map includes performing an iterative process such that the iterative process determines input test values that cause the updated model to output the updated function map.

19. The computer system of claim 18 , wherein the iterative process comprises iteratively inputting input test values selected from a range of values until a boundary of the function map is detected.

20. The computer system of claim 17 , wherein the classification rule includes assigning the version classification based on whether the comparison of the updated function map to the original function map results in a difference that exceeds a specified threshold value.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2021
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: KYNDRYL, INC.
Reel/Frame 058213/0912 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2020
From: GAN, SENG CHAI; KWATRA, SHIKHAR; SIVAKUMAR, GANDHI; VALECHA, VINOD A.; EUNICE, THOMAS JOSEPH
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 054652/0781 →
Cited By (2)
US 12,248,853 US 12,332,771