IP Library › Granted Patent US 11,500,671
Granted Patent B2
US 11,500,671 · App. 16/510,534 · Granted Nov 15, 2022

System for operationalizing high-level machine learning training enhancements from low-level primitives

Inventors: Evelyn Duesterwald (Millwood, NY); Anupama Murthi (Pleasantville, NY); Deepak Vijaykeerthy (Bangalore, IN); Vijay Arya (Bangalore, IN); Ganesh Venkataraman (White Plains, NY)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06F9/4843G06K9/6256G06K9/6262G06K9/6267G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,500,671
App. No.
16/510,534
Granted
Nov 15, 2022
Kind
B2
Abstract

In an embodiment, a method for inspecting and transforming a machine learning model includes receiving a request that includes the machine learning model and a configuration object that provides an indication of a selected strategy. In the embodiment, the method includes creating a partially specified task graph that includes a first placeholder node for a future expanded task node. In the embodiment, the method includes performing a dynamic expansion and execution phase that includes, repeatedly (a) using a cognitive engine to evaluate whether to revise the partially specified task graph based at least in part on the selected strategy, and (b) using a processor-based execution engine to perform an action specified by the complete node. In an embodiment, the dynamic expansion and execution phase repeats until after the cognitive engine adds a consolidated results node.

Claims (47)

1. A computer implemented method for inspecting and transforming a machine learning model, the computer implemented method comprising:

receiving, by one or more processors, a request that includes the machine learning model and a configuration object, wherein the configuration object provides an indication of a selected strategy;

creating, by one or more processors, a partially specified task graph based on the selected strategy, wherein the partially specified task graph is formed of a subset of nodes of a final version of the partially specified task graph, wherein the final version of the partially specified task graph is a final task graph that completes the selected strategy, wherein the partially specified task graph includes a first placeholder node for a future expanded task node; and

performing, by one or more processors, a dynamic expansion and execution phase that includes, repeatedly:

if at least one placeholder node is present in the partially specified task graph, using a cognitive engine to evaluate whether to revise the partially specified task graph based at least in part on the selected strategy; and

if the cognitive engine adds a complete node to the partially specified task graph, using a processor-based execution engine to perform an action specified by the complete node;

wherein the dynamic expansion and execution phase repeats until after the cognitive engine adds a consolidated results node that completes the final task graph.

2. The computer implemented method of claim 1 , further comprising:

receiving, via a graphical user interface (GUI) generated by one or more processors, user input data indicative of the selected strategy, the selected strategy corresponding to a displayed strategy option; and

retrieving, from a computer-readable storage device by one or more processors, an algorithm library associated with the selected strategy.

3. The computer implemented method of claim 2 , further comprising:

creating, by one or more processors, the configuration object based at least in part on the selected strategy and the algorithm library associated therewith.

4. The computer implemented method of claim 3 , wherein the creating, by the one or more processors, of the partially specified task graph includes initializing the partially specified task graph to include a number of nodes, wherein the number of nodes is based on the algorithm library associated with the selected strategy.

5. The computer implemented method of claim 1 , wherein the dynamic expansion and execution phase further comprises storing, by one or more processors in a computer-readable memory, historical data representative of a result of an action previously performed by the execution engine as specified by a previous node of the partially specified task graph.

6. The computer implemented method of claim 5 , wherein the cognitive engine evaluates whether to revise the partially specified task graph based at least in part on the historical data and based on the configuration object.

7. The computer implemented method of claim 1 , wherein the cognitive engine revises the partially specified task graph based on a revised hyperparameter.

8. The computer implemented method of claim 1 , wherein the cognitive engine evaluates how to revise the partially specified task graph by querying a composition planning module to determine a next step in a composition plan for executing the request.

9. The computer implemented method of claim 1 , wherein the cognitive engine revises the partially specified task graph according to a selected strategy for improving a robustness of the machine learning model.

10. The computer implemented method of claim 1 , wherein the performing of the dynamic and execution phase includes a plurality of iterations of the using of the execution engine to perform respective actions, wherein the respective actions collectively modify the machine learning module and thereby repair model revisions previously made by malicious training data.

11. A computer usable program product for inspecting and transforming a machine learning model, the computer usable program product comprising a computer-readable storage device, and program instructions stored on the storage device, the stored program instructions comprising:

program instructions to receive, by one or more processors, a request that includes the machine learning model and a configuration object, wherein the configuration object provides an indication of a selected strategy;

program instructions to create, by one or more processors, a partially specified task graph based on the selected strategy, wherein the partially specified task graph is formed of a subset of nodes of a final version of the partially specified task graph, wherein the final version of the partially specified task graph is a final task graph that completes the selected strategy, wherein the partially specified task graph includes a first placeholder node for a future expanded task node; and

program instructions to perform, by one or more processors, a dynamic expansion and execution phase that includes, repeatedly:

if at least one placeholder node is present in the partially specified task graph, using a cognitive engine to evaluate whether to revise the partially specified task graph based at least in part on the selected strategy; and

if the cognitive engine adds a complete node to the partially specified task graph, using a processor-based execution engine to perform an action specified by the complete node;

wherein the dynamic expansion and execution phase repeats until after the cognitive engine adds a consolidated results node that completes the final task graph.

12. The computer usable program product of claim 11 , further comprising:

program instructions to receive, via a graphical user interface (GUI) generated by one or more processors, user input data indicative of the selected strategy, the selected strategy corresponding to a displayed strategy option; and

program instructions to retrieve, from a computer-readable storage device by one or more processors, an algorithm library associated with the selected strategy.

13. The computer implemented method of claim 12 , further comprising:

program instructions to create, by one or more processors, the configuration object based at least in part on the selected strategy and the algorithm library associated therewith.

14. The computer implemented method of claim 13 , wherein the creating, by the one or more processors, of the partially specified task graph includes initializing the partially specified task graph to include a number of nodes, wherein the number of nodes is based on the algorithm library associated with the selected strategy.

15. The computer implemented method of claim 11 , wherein the dynamic expansion and execution phase further comprises storing, by one or more processors in a computer-readable memory, historical data representative of a result of an action previously performed by the execution engine as specified by a previous node of the partially specified task graph.

16. A computer system comprising a processor, a computer-readable memory, and a computer-readable storage device, and program instructions stored on the storage device for execution by the processor via the memory, the stored program instructions comprising:

program instructions to receive, by one or more processors, a request that includes the machine learning model and a configuration object, wherein the configuration object provides an indication of a selected strategy;

program instructions to create, by one or more processors, a partially specified task graph based on the selected strategy, wherein the partially specified task graph is formed of a subset of nodes of a final version of the partially specified task graph, wherein the final version of the partially specified task graph is a final task graph that completes the selected strategy, wherein the partially specified task graph includes a first placeholder node for a future expanded task node; and

program instructions to perform, by one or more processors, a dynamic expansion and execution phase that includes, repeatedly:

if at least one placeholder node is present in the partially specified task graph, using a cognitive engine to evaluate whether to revise the partially specified task graph based at least in part on the selected strategy; and

if the cognitive engine adds a complete node to the partially specified task graph, using a processor-based execution engine to perform an action specified by the complete node;

wherein the dynamic expansion and execution phase repeats until after the cognitive engine adds a consolidated results node that completes the final task graph.

17. The computer implemented method of claim 16 , further comprising:

program instructions to receive, via a graphical user interface (GUI) generated by one or more processors, user input data indicative of the selected strategy, the selected strategy corresponding to a displayed strategy option;

program instructions to retrieve, from a computer-readable storage device by one or more processors, an algorithm library associated with the selected strategy; and

program instructions to create, by one or more processors, the configuration object based at least in part on the selected strategy and the algorithm library associated therewith.

18. The computer implemented method of claim 17 , wherein the creating, by the one or more processors, of the partially specified task graph includes initializing the partially specified task graph to include a number of nodes, wherein the number of nodes is based on the algorithm library associated with the selected strategy.

19. The computer implemented method of claim 16 , wherein the dynamic expansion and execution phase further comprises storing, by one or more processors in a computer-readable memory, historical data representative of a result of an action previously performed by the execution engine as specified by a previous node of the partially specified task graph.

20. The computer implemented method of claim 19 , wherein the cognitive engine evaluates whether to revise the partially specified task graph based at least in part on the historical data and based on the configuration object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 12, 2019
From: DUESTERWALD, EVELYN; MURTHI, ANUPAMA; VIJAYKEERTHY, DEEPAK; ARYA, VIJAY; VENKATARAMAN, GANESH
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 049743/0080 →
Continuity (1)
Related Publication 20210011757A1 · Jan 14, 2021