Responding to a user query using machine learning
A method, apparatus, non-transitory computer readable medium, and system for data processing include obtaining a query relating to a document and identifying metadata for the document based on the query, where the metadata describes a structure including a plurality of portions of the document. Some embodiments including generating, using a machine learning model, a retrieval command based on the query and the metadata, selectively retrieving at least one of the plurality of portions of the document based on the retrieval command, and generating, using the machine learning model, a response to the query based on the at least one of the plurality of portions of the document.
1 . A method for data processing, comprising:
obtaining, from a computing device, a query relating to a document; and
in response to the obtaining of the query from the computing device:
identifying, by a metadata component executed on a data processing system, metadata for the document based on the query, wherein the metadata describes a structure of the document including a plurality of portions of the document;
generating, by a query component executed on the data processing system, a retrieval command prompt, wherein the retrieval command prompt includes the query, the metadata, a plurality of executable functions, and an instruction to generate, based on the query and the metadata, a retrieval command including at least one executable function of the plurality of executable functions and an argument for the at least one executable function;
generating, by a language generation machine learning model of the data processing system executing an attention mechanism, the retrieval command based on the instruction by computing a first set of attention weights corresponding to the instruction, wherein the retrieval command is based on the first set of attention weights and includes an executable function of the plurality of executable functions and an argument generated by the language generation machine learning model for the executable function;
selectively retrieving at least one portion of the plurality of portions of the document based on a context window size of the language generation machine learning model of the data processing system executing the attention mechanism by executing the executable function according to the argument for the executable function;
generating, by the language generation machine learning model of the data processing system executing the attention mechanism, a response to the query based on the retrieved portion of the document by computing a second set of attention weights corresponding to the retrieved portion of the document, wherein the second set of attention weights is different from the first set of attention weights, wherein the retrieval command is based on the second set of attention weights, and wherein the response comprises natural language text; and
displaying the response to a user via the computing device.
2 . The method of claim 1 , wherein: the query specifies the portion of the document.
3 . The method of claim 1 , wherein: the metadata comprises a hierarchical tree of structural elements included in the document.
4 . The method of claim 3 , wherein obtaining the metadata comprises: generating the hierarchical tree of structural elements based on text of the document.
5 . The method of claim 1 , wherein: the language generation machine learning model is trained to generate text in response to a natural language query.
6 . A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to:
obtain, from a computing device, a query relating to a document; and
in response to the obtaining of the query from the computing device:
identify, by a metadata component executed on a data processing system, metadata for the document based on the query, wherein the metadata describes a structure of the document including a plurality of portions of the document;
generate, by a query component executed on the data processing system, a retrieval command prompt, wherein the retrieval command prompt includes the query, the metadata, a plurality of executable functions, and an instruction to generate, based on the query and the metadata, a retrieval command including at least one executable function of the plurality of executable functions and an argument for the at least one executable function;
generate, by a language generation machine learning model of the data processing system executing an attention mechanism, the retrieval command based on the instruction by computing a first set of attention weights corresponding to the instruction, wherein the retrieval command is based on the first set of attention weights and includes an executable function of the plurality of executable functions and an argument generated by the language generation machine learning model for the executable function;
selectively retrieve at least one portion of the plurality of portions of the document based on a context window size of the language generation machine learning model of the data processing system executing the attention mechanism by executing the executable function according to the argument for the executable function;
generate, by the language generation machine learning model of the data processing system executing the attention mechanism, a response to the query based on the retrieved portion of the document by computing a second set of attention weights corresponding to the retrieved portion of the document and different from the first set of attention weights, wherein the retrieval command is based on the second set of attention weights and wherein the response comprises natural language text; and
display the response to a user via the computing device.
7 . The non-transitory computer readable medium of claim 6 , wherein: the query specifies the portion of the document.
8 . The non-transitory computer readable medium of claim 6 , wherein: the metadata comprises a hierarchical tree of structural elements included in the document.
9 . The non-transitory computer readable medium of claim 8 , wherein the instructions further cause the processor to: generate the hierarchical tree of structural elements based on text of the document.
10 . The non-transitory computer readable medium of claim 6 , wherein: the language generation machine learning model is trained to generate text in response to a natural language query.
11 . A data processing system, comprising:
a memory; and
a processing device coupled to the memory, the processing device configured to perform operations comprising:
obtaining, from a computing device, a query relating to a document; and
in response to the obtaining of the query from the computing device:
identifying, by a metadata component executed on a data processing system, metadata for the document based on the query, wherein the metadata describes a structure of the document including a plurality of portions of the document;
generating, by a query component executed on the data processing system, a retrieval command prompt, wherein the retrieval command prompt includes the query, the metadata, a plurality of executable functions, and an instruction to generate, based on the query and the metadata, a retrieval command including at least one executable function of the plurality of executable functions and an argument for the at least one executable function;
generating, by a language generation machine learning model of the data processing system executing an attention mechanism, the retrieval command based on the instruction by computing a first set of attention weights corresponding to the instruction, wherein the retrieval command is based on the first set of attention weights and includes an executable function of the plurality of executable functions and an argument generated by the language generation machine learning model for the executable function;
selectively retrieving at least one portion of the plurality of portions of the document based on a context window size of the language generation machine learning model of the data processing system executing the attention mechanism by executing the executable function according to the argument for the executable function;
generating, by the language generation machine learning model of the data processing system executing the attention mechanism, a response to the query based on the retrieved portion of the document by computing a second set of attention weights corresponding to the retrieved portion of the document, wherein the second set of attention weights is different from the first set of attention weights, wherein the retrieval command is based on the second set of attention weights, and wherein the response comprises natural language text; and
displaying the response to a user via the computing device.
12 . The system of claim 11 , further comprising: a metadata component configured to identify the metadata for the document based on the query.
13 . The system of claim 12 , wherein the metadata component is further configured to: generate a hierarchical tree of structural elements based on text of the document.