IP Library › Granted Patent US 12,572,819
Granted Patent B2
US 12,572,819 · App. 17/805,572 · Granted Mar 10, 2026

Generating artificial intelligence plans of high diversity

Inventors: Michael Katz (Goldens Bridge, NY); Shirin Sohrabi Araghi (Briarcliff Manor, NY)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06N3/126
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,572,819
App. No.
17/805,572
Granted
Mar 10, 2026
Kind
B2
Abstract

In an approach for improved artificial intelligence planning in an automated machine learning pipeline, a processor formulates an artificial intelligence planning problem. A processor receives a pre-defined stopping criterion for generating one or more plans for the artificial intelligence planning problem. A processor generates the one or more plans by executing a planning algorithm. A processor reformulates the artificial intelligence planning problem into a new artificial intelligence planning problem by forbidding plans that correspond to super-sets of the one or more plans. A processor generates one or more new plans based on the reformulation until the pre-defined stopping criterion is reached.

Claims (60)

1 . A computer-implemented method comprising:

formulating, by one or more processors, an artificial intelligence (AI) planning problem represented as a computer-readable model comprising a set of operators;

executing a planning algorithm, by the one or more processors, to generate a solution set to the AI planning problem comprising one or more plans;

iteratively reformulating, by the one or more processors, the computer-readable model of the AI planning problem by encoding exclusion conditions such that the planning algorithm cannot generate:

a previously generated plan from a previous iteration; and

any plan that, for each operator occurring a given number of times in the previously generated plan, includes that operator at least that given number of times; and

executing the planning algorithm, by the one or more processors, on the iteratively reformulated computer-readable model to generate additional plans until a pre-defined stopping criterion is reached, wherein executing the planning algorithm on the iteratively reformulated computer-readable model reduces a size of the solution set, thereby improving computational efficiency.

2 . The computer-implemented method of claim 1 , wherein the encoding exclusion conditions further comprises:

extending to track one or more operators on the previously generated plan;

modifying and extending the one or more operators to change a value of an extra variable;

extending an initial state to allocate an initial value to the extra variable; and

extending a goal to allocate a goal value to the one or more operators and to check the extra variable.

3 . The computer-implemented method of claim 1 , wherein the iteratively reformulating the computer-readable model of the AI planning problem further comprises:

forbidding plans that correspond to super-multisets of the one or more plans.

4 . The computer-implemented method of claim 1 , wherein the pre-defined stopping criterion is selected from the group consisting of: a plan of worse quality than a quality bound is found, enough plans are found, and no more plans exist.

5 . The computer-implemented method of claim 1 , wherein the planning algorithm is selected from the group consisting of: a cost-optimal planner, an agile planner, a satisficing planner, a top-k planner, a top-quality planner, and a diverse planner.

6 . The computer-implemented method of claim 1 , further comprising:

generating a subset top-quality solution in domain with infinite size unordered top-quality solutions.

7 . The computer-implemented method of claim 1 , further comprising:

utilizing a cost-optimal planner to generate an updated solution to the AI planning problem based on the pre-defined stopping criterion.

8 . A computer program product comprising:

one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising:

program instructions to formulate an artificial intelligence (AI) planning problem represented as a computer-readable model comprising a set of operators;

program instructions to execute a planning algorithm to generate a solution set to the AI planning problem comprising one or more plans;

program instructions to iteratively reformulate the computer-readable model of the AI planning problem by encoding exclusion conditions such that the planning algorithm cannot generate:

a previously generated plan from a previous iteration; and

any plan that, for each operator occurring a given number of times in the previously generated plan, includes that operator at least that given number of times; and

program instructions to execute the planning algorithm on the iteratively reformulated computer-readable model to generate additional plans until a pre-defined stopping criterion is reached, wherein executing the planning algorithm on the iteratively reformulated computer-readable model reduces a size of the solution set, thereby improving computational efficiency.

9 . The computer program product of claim 8 , wherein the encoding exclusion conditions further comprises:

program instructions to extend to track one or more operators on the previously generated plan;

program instructions to modify and extend the one or more operators to change a value of an extra variable;

program instructions to extend an initial state to allocate an initial value to the extra variable; and

program instructions to extend a goal to allocate a goal value to the one or more operators and to check the extra variable.

10 . The computer program product of claim 8 , wherein the program instructions to iteratively reformulate the computer-readable model of the AI planning problem further comprises:

program instructions to forbid plans that correspond to super-multisets of the one or more plans.

11 . The computer program product of claim 8 , wherein the pre-defined stopping criterion is selected from the group consisting of: a plan of worse quality than a quality bound is found, enough plans are found, and no more plans exist.

12 . The computer program product of claim 8 , wherein the planning algorithm is selected from the group consisting of: a cost-optimal planner, an agile planner, a satisficing planner, a top-k planner, a top-quality planner, and a diverse planner.

13 . The computer program product of claim 8 , further comprising:

program instructions to generate a subset top-quality solution in domain with infinite size unordered top-quality solutions.

14 . The computer program product of claim 8 , further comprising:

program instructions to utilize a cost-optimal planner to generate an updated solution to the AI planning problem based on the pre-defined stopping criterion.

15 . A computer system comprising:

one or more computer processors, one or more computer readable storage media, and program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising:

program instructions to formulate an artificial intelligence (AI) planning problem represented as a computer-readable model comprising a set of operators;

program instructions to execute a planning algorithm to generate a solution set to the AI planning problem comprising one or more plans;

program instructions to iteratively reformulate the computer-readable model of the AI planning problem by encoding exclusion conditions such that the planning algorithm cannot generate:

a previously generated plan from a previous iteration; and

any plan that, for each operator occurring a given number of times in the previously generated plan, includes that operator at least that given number of times; and

program instructions to execute the planning algorithm on the iteratively reformulated computer-readable model to generate additional plans until a pre-defined stopping criterion is reached, wherein executing the planning algorithm on the iteratively reformulated computer-readable model reduces a size of the solution set, thereby improving computational efficiency.

16 . The computer system of claim 15 , wherein the encoding exclusion conditions further comprises:

program instructions to extend to track one or more operators on the previously generated plan;

program instructions to modify and extend the one or more operators to change a value of an extra variable;

program instructions to extend an initial state to allocate an initial value to the extra variable; and

program instructions to extend a goal to allocate a goal value to the one or more operators and to check the extra variable.

17 . The computer system of claim 15 , wherein the program instructions to iteratively reformulate the computer-readable model of the AI planning problem further comprises:

program instructions to forbid plans that correspond to super-multisets of the one or more plans.

18 . The computer system of claim 15 , wherein the pre-defined stopping criterion is selected from the group consisting of: a plan of worse quality than a quality bound is found, enough plans are found, and no more plans exist.

19 . The computer system of claim 15 , wherein the planning algorithm is selected from the group consisting of: a cost-optimal planner, an agile planner, a satisficing planner, a top-k planner, a top-quality planner, and a diverse planner.

20 . The computer system of claim 15 , further comprising:

program instructions to generate a subset top-quality solution in domain with infinite size unordered top-quality solutions.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2022
From: KATZ, MICHAEL; SOHRABI ARAGHI, SHIRIN
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 060111/0327 →
Continuity (1)
Related Publication 20230394325A1 · Dec 7, 2023
References Cited (31)
US 9740978B2 · Hassanzadeh · 2017 [cited by applicant]
US 10831629B2 · Riabov · 2020 [cited by applicant]
US 20130132042A1 · Chan · 2013 [cited by examiner]
US 20160321544A1 · Hassanzadeh · 2016 [cited by examiner]
US 20180005079A1 · Tosic · 2018 [cited by examiner]
US 20190340525A1 · Katz · 2019 [cited by applicant]
US 20200074315A1 · Katz · 2020 [cited by applicant]
US 20200210532A1 · Matei · 2020 [cited by examiner]
US 20200265267A1 · Galuten · 2020 [cited by applicant]
US 20210004741A1 · Katz · 2021 [cited by applicant]
US 20210020297A1 · Adler · 2021 [cited by examiner]
US 20210241866A1 · Bhattacharya · 2021 [cited by examiner]
“Autonomous Model Learning System for Model Based Solutions in Industrial Plants”, An IP.com Prior Art Database Technical Disclosure, IP.com No. IPCOM000264030D, IP.com Electronic Publication Date: Nov. 2, 2020, 5 pages. [cited by applicant]
“DevOps Robotic Process Automation and Artificial Intelligence Remediation”, An IP.com Prior Art Database Technical Disclosure, IP.com No. IPCOM000264994D, IP.com Electronic Publication Date: Feb. 16, 2021, 7 pages. [cited by applicant]
“L&T blog: About learning, training and technology at ITC-ILO, L&T blog learning about scenario planning and thinking”, Apr. 9, 2012, 6 pages, <https://itcilo.wordpress.com/2012/04/09/learning-about-scenario-planning-an… [cited by applicant]
“System and Method of Goal-Oriented Optimization for Recommendation and Interpretability Based on Process-Aware Automated Machine Learning Pipeline”, An IP.com Prior Art Database Technical Disclosure, IP.com No. IPCOM00… [cited by applicant]
Berman, Emily, “A Government of Laws and Not of Machines”, Boston University Law Review, vol. 98:1277, 2018, 70 pages. [cited by applicant]
Feblowitz et al., “IBM Scenario Planning Advisor: A Neuro-Symbolic ERM Solution”, Association for the Advancement of Artificial Intelligence, 2021, 3 pages, <https://www.aaai.org/AAAI21Papers/DEMO-252.FeblowitzM.pdf>. [cited by applicant]
Katz et al. “Bounding Quality in Diverse Planning”, Proceedings of the 36th AAAI Conference on Artificial Intelligence, Virtual Conference, Feb. 22-Mar. 2, 2022, 8 pages, < https://www.aaai.org/AAAI22Papers/AAAI-9930.Ka… [cited by applicant]
Katz et al., “A Novel Iterative Approach to Top-κ Planning”, Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling (ICAPS 2018), Jun. 24-29, 2018, Delft, Netherlands, 9 pages, <h… [cited by applicant]
Katz et al., “Machine Learning Pipeline Generation”, U.S. Appl. No. 17/119,134, filed Dec. 11, 2020, 50 pages. [cited by applicant]
Katz et al., “Reshaping Diverse Planning”, Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20), Feb. 7-12, 2020, New York, New York, 8 pages, <https://ojs.aaai.org/index.php/AAAI/articl… [cited by applicant]
Katz et al., “Top-Quality Planning: Finding Practically Useful Sets of Best Plans”, Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20), Feb. 7-12, 2020, New York, New York, 8 pages, <h… [cited by applicant]
Katz et al., “Who Needs These Operators Anyway: Top Quality Planning with Operator Subset Criteria”, Proceedings of the 32nd International Conference on Automated Planning and Scheduling, Jun. 13-24, 2022, Virtual Confe… [cited by applicant]
Naser, Samy S. Abu, “Predicting Learners Performance Using Artificial Neural Networks in Linear Programming Intelligent Tutoring System”, International Journal of Artificial Intelligence & Applications (IJAIA), vol. 3, … [cited by applicant]
Riabov et al., “Planning-Based Reasoning for Automated Large-Scale Data Analysis”, Proceedings of the Twenty-Fifth International Conference on Automated Planning and Scheduling, Jun. 7-11, 2015, Jerusalem, Israel, 9 pag… [cited by applicant]
Sohrabi et al. “Scenario Planning for Enterprise Risk Management”, Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS 2017), Jun. 18-23, 2017, Pittsburgh, PA, 3 pages. [cited by applicant]
Sohrabi et al., “Hypothesis Exploration for Malware Detection using Planning”, Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, Jul. 14-18, 2013, Bellevue, WA, 7 pages, <https://www.aaai.org… [cited by applicant]
Sohrabi et al., “State Projection via AI Planning”, Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), Feb. 4-9, 2017, San Francisco, CA, 7 pages, <https://www.aaai.org/ocs/index.php/A… [cited by applicant]
Spies et al., “Domain-Independent Cost-Optimal Planning in ASP”, Cornell University, arXiv, Jul. 31, 2019, 33 pages. [cited by applicant]
Yu et al., “Learning Search-Space Specific Heuristics Using Neural Network”, Proceedings of the 30th International Conference on Automated Planning and Scheduling, Oct. 19-30, 2020, online, 144 pages, <https://icaps20su… [cited by applicant]