Natural language to API conversion
Representative embodiments disclose mechanisms to map natural language input to an application programming interface (API) call. The natural language input is first mapped to an API frame, which is a representation of the API call without any API call formatting. The mapping from natural language input to API frame is performed using a trained sequence to sequence neural model. The sequence to sequence neural model is decomposed into small prediction units called modules. Each module is highly specialized at predicting a pre-defined kind of sequence output. The output of the modules can be displayed in an interactive user interface that allows the user to add, remove, and/or modify the output of the individual modules. The user input can be used as further training data. The API frame is mapped to an API call using a deterministic mapping.
1. A computer implemented method, comprising:
receiving a natural language utterance;
submitting the natural language utterance to a trained machine learning model comprising a sequence to sequence neural model, the sequence to sequence neural model comprising an encoder and a plurality of decoders, each of the plurality of decoders coupled to the encoder and being trained to recognize one or more tokens output from the encoder and to map the one or more tokens to one or more items of an API frame;
receiving from the trained machine learning model, a plurality of items, each item received from a different decoder;
assembling the plurality of items into the API frame, the API frame representing an intermediate format between the natural language utterance and a final API format;
mapping the API frame to the final API format; and
issuing a call to the API using the final API format.
2. The method of claim 1 wherein the encoder comprises a recurrent neural network.
3. The method of claim 1 wherein each decoder comprises an attentive recurrent neural network.
4. The method of claim 1 further comprising a controller coupled to the encoder, the controller receiving the output of the encoder and producing a layout comprising the plurality of decoders and activating each of the decoders in the layout.
5. The method of claim 1 further comprising:
presenting each item via a user interface, the user interface comprising:
a region to present each item along with an associated indication that indicates what the item means;
a control, which when activated, removes an associated item; and
a control, which when activated, adds a new item; and
receiving input via the user interface, indicating any changes to the items are complete.
6. The method of claim 5 further comprising:
aggregating performance data comprising:
the natural language utterance;
the assembled API frame; and
any changes made to items via the user interface.
7. The method of claim 1 wherein mapping the API frame to the final API format is accomplished using a plurality of rules that deterministically map items of the API frame to items in the final API format.
8. The method of claim 1 wherein the encoder and plurality of decoders are trained using a supervised learning process that utilizes annotated natural language utterance data comprising:
a plurality of training natural language utterances; and
an API frame for each of the training natural language utterances.
9. The method of claim 1 further comprising updating the training of the sequence to sequence neural model using data gathered using an interactive user interface that presents items output from the plurality of decoders and allows:
an item to be removed prior to assembling items into the API frame;
an item to be added prior to assembling items into the API frame; and
an item to be modified prior to assembling items into the API frame.
10. A system comprising:
a processor and device-storage media having executable instructions which, when executed by the processor, cause the system to perform operations comprising:
receiving a natural language utterance;
submitting the natural language utterance to a trained machine learning model comprising a sequence to sequence neural model, the sequence to sequence neural model comprising an encoder and a plurality of decoders, each of the plurality of decoders coupled to the encoder and being trained to recognize one or more tokens output from the encoder and to map the one or more tokens to one or more items of an API frame;
receiving from the trained machine learning model, a plurality of items, each item received from a different decoder;
assembling the plurality of items into the API frame, the API frame representing an intermediate format between the natural language utterance and a final API format;
mapping the API frame to the final API format; and
issuing a call to the API using the final API format.
11. The system of claim 10 wherein the encoder comprises a recurrent neural network.
12. The system of claim 10 wherein each decoder comprises an attentive recurrent neural network.
13. The system of claim 10 further comprising a controller coupled the encoder, the controller:
receiving the output of the encoder;
producing a layout comprising the plurality of decoders; and
activating each of the decoders in the layout.
14. The system of claim 10 , the operations further comprising:
presenting each item via a user interface, the user interface comprising:
a region to present each item along with an associated indication that indicates what the item means;
a control, which when activated, removes an associated item; and
a control, which when activated, adds a new item; and
receiving input via the user interface, indicating any changes to the items are complete.
15. The system of claim 14 , the operations further comprising:
aggregating performance data comprising:
the natural language utterance;
the assembled API frame; and
any changes made to items via the user interface; and
sending the aggregated performance data to a system to be used to update training of the sequence to sequence neural model.
16. The system of claim 10 wherein mapping the API frame to the final API format is accomplished using a plurality of rules that deterministically map items of the API frame to items in the final API format.
17. The system of claim 10 wherein the encoder and plurality of decoders are trained using a supervised learning process that utilizes annotated natural language utterance data comprising:
a plurality of training natural language utterances; and
an API frame for each of the training natural language utterances.
18. The system of claim 10 , the operations further comprising updating the training of the sequence to sequence neural model using data gathered using an interactive user interface that presents items output from the plurality of decoders and allows:
an item to be removed prior to assembling items into the API frame;
an item to be added prior to assembling items into the API frame; and
an item to be modified prior to assembling items into the API frame.
19. A computer storage medium comprising executable instructions that, when executed by a processor of a machine, cause the machine to perform acts comprising:
receiving a natural language utterance;
submitting the natural language utterance to a trained machine learning model comprising a sequence to sequence neural model, the sequence to sequence neural model comprising an encoder and a plurality of decoders, each of the plurality of decoders coupled to the encoder and being trained to recognize one or more tokens output from the encoder and to map the one or more tokens to one or more items of an API frame;
receiving from the trained machine learning model, a plurality of items, each item received from a different decoder;
assembling the plurality of items into the API frame, the API frame representing an intermediate format between the natural language utterance and a final API format;
mapping the API frame to the final API format; and
issuing a call to the API using the final API format.
20. The medium of claim 19 , the acts further comprising:
presenting each item via a user interface, the user interface comprising:
a region to present each item along with an associated indication that indicates what the item means;
a control, which when activated, removes an associated item; and
a control, which when activated, adds a new item; and
receiving input via the user interface, indicating any changes to the items are complete.