Large language model-based method for translating a prompt into a planning problem
Certain aspects of the disclosure provide techniques for translating a prompt into a structured input to resolve the natural langue query as a planning problem. A method generally includes identifying and classifying tokens in a prompt using a large language model (LLM); extracting from a domain description in a planning domain definition language (PDDL): object types used to categorize objects; and predicates identifying relationships between the objects that may be true or false; categorizing at least one token in the prompt as one or more of the objects, one or more of the object types, or one or more of the predicates based on the classification of the at least one token determined by the LLM; and generating a task description in the PDDL based on the categorization, the task description comprising a translation of the prompt into a structured input for a planner.
1 . A method of prompt translation, comprising:
identifying and classifying a plurality of tokens in a prompt using a large language model (LLM), the prompt requesting a state change from an initial state to a desired goal state;
extracting from a domain description in a planning domain definition language (PDDL):
object types used to categorize objects; and
predicates identifying relationships between the objects that may be true or false;
categorizing at least one token of the plurality of tokens in the prompt as one or more of the objects, one or more of the object types, or one or more of the predicates based on a classification of the at least one token determined by the LLM;
generating a task description in the PDDL based on categorization of the at least one token of the plurality of tokens, the task description comprising a translation of the prompt into a structured input for a planner;
determining one or more validation errors exist in the task description in the PDDL;
determining whether a termination condition has been met;
when the termination condition has been met, generating an error message; and
when the termination condition has not been met, generating a new task description in the PDDL to fix the one or more validation errors.
2 . The method of claim 1 , wherein determining whether the termination condition has been met comprises determining the termination condition has been met based on a number of task description generation attempts exceeding a task description generation threshold number.
3 . The method of claim 1 , wherein determining whether the one or more validation errors exist in the task description in the PDDL comprises performing at least one of:
syntactic validation to check whether the task description conforms to a general task description template syntactically;
generic constraint validation to check whether the task description includes elements declared in the domain description in the PDDL; or
domain constraint validation to check whether the task description is consistent with constraints defined in the domain description in the PDDL.
4 . The method of claim 3 , wherein:
determining whether the one or more validation errors exist in the task description in the PDDL comprises performing the syntactic validation, without performing the generic constraint validation and the domain constraint validation, and
the one or more validation errors are determined when performing the syntactic validation.
5 . The method of claim 3 , wherein:
determining whether the one or more validation errors exist in the task description in the PDDL comprises performing the syntactic validation and the generic constraint validation, without performing the domain constraint validation,
the one or more validation errors are determined when performing the generic constraint validation, and
the generic constraint validation is performed after performing the syntactic validation.
6 . The method of claim 1 , wherein the desired goal state requested by the prompt comprises:
a possession of information;
a completion of one or more tasks excluding information retrieval; or
a completion of one or more tasks including the information retrieval.
7 . The method of claim 1 , wherein identifying and classifying the plurality of tokens in the prompt using the LLM comprises prompting, via a set of prompts, the LLM to perform a series of natural language processing (NLP) tasks.
8 . A processing system, comprising:
one or more memories comprising processor-executable instructions; and
one or more processors configured to execute the processor-executable instructions and cause the processing system to:
identify and classify a plurality of tokens in a prompt using a large language model (LLM), the prompt requesting a state change from an initial state to a desired goal state;
extract from a domain description in a planning domain definition language (PDDL):
object types used to categorize objects; and
predicates identifying relationships between the objects that may be true or false;
categorize at least one token of the plurality of tokens in the prompt as one or more of the objects, one or more of the object types, or one or more of the predicates based on a classification of the at least one token determined by the LLM;
generate a task description in the PDDL based on categorization of the at least one token of the plurality of tokens, the task description comprising a translation of the prompt into a structured input for a planner;
determine one or more validation errors exist in the task description in the PDDL;
determine whether a termination condition has been met;
when the termination condition has been met generate an error message; and
when the termination condition has not been met, generate a new task description in the PDDL to fix the one or more validation errors.
9 . The processing system of claim 8 , wherein to determine whether the termination condition has been met, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to determine the termination condition has been met based on a number of task description generation attempts exceeding a task description generation threshold number.
10 . The processing system of claim 8 , wherein to determine whether the one or more validation errors exist in the task description in the PDDL, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to perform at least one of:
syntactic validation to check whether the task description conforms to a general task description template syntactically;
generic constraint validation to check whether the task description includes elements declared in the domain description in the PDDL; or
domain constraint validation to check whether the task description is consistent with constraints defined in the domain description in the PDDL.
11 . The processing system of claim 10 , wherein:
to determine whether the one or more validation errors exist in the task description in the PDDL, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to perform the syntactic validation, without performing the generic constraint validation and the domain constraint validation, and
the one or more validation errors are determined when performing the syntactic validation.
12 . The processing system of claim 10 , wherein:
to determine whether the one or more validation errors exist in the task description in the PDDL, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to perform the syntactic validation and the generic constraint validation, without performing the domain constraint validation,
the one or more validation errors are determined when performing the generic constraint validation, and
the generic constraint validation is performed after performing the syntactic validation.
13 . The processing system of claim 8 , wherein the desired goal state requested by the prompt comprises:
a possession of information;
a completion of one or more tasks excluding information retrieval; or
a completion of one or more tasks including the information retrieval.
14 . The processing system of claim 8 , wherein to identify and classify the plurality of tokens in the prompt using the LLM, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to prompt, via a set of prompts, the LLM to perform a series of natural language processing (NLP) tasks.