Gestural prompting based on conversational artificial intelligence

1. A method comprising:

obtaining training data that describes (a) a situation, (b) a gesture for a response to the situation, (c) a prompt to accompany the response, and (d) a gestural annotation for the response, the training data including a hypothetical user query related to a custom domain,

the situation being a stimulus warranting a response from a natural language understanding (NLU) model,

the gesture for the response being a suggested gesture in response to the situation from the NLU model,

the prompt to accompany the response being a suggested phrase in response to the situation from the NLU model, and

the gestural annotation for the response being a process of labeling the response to show a gestural outcome to be predicted by the NLU model;

the training data comprising an utterance, an intent, an entity, a vocabulary, a gesture, or an action usable to respond to the situation;

tagging the hypothetical user query with the gestural annotation for the response; and

training the NLU model to address the situation, based on the training data, the training comprising:

based on the tagged hypothetical user query, training the NLU model to recognize a plurality of gesture types, and

based on recognizing a gesture type of the plurality of gesture types, train the NLU model to classify and annotate a real user query at a runtime based on the recognized gesture type.

2. The method of claim 1 , wherein the situation comprises a verbal query.

3. The method of claim 1 , wherein the situation comprises a detection of an entity selected from the group consisting of a person, an animal, and an object.

4. The method of claim 1 , wherein the situation comprises a detection of an environmental condition.

5. The method of claim 1 , wherein the gesture comprises a directional gesture.

6. The method of claim 1 , wherein the NLU model is utilized in a process that controls a bot.

7. A system comprising:

a processor; and

a memory that contains instructions that are readable by the processor to cause the processor to perform operations of:

obtaining training data that describes (a) a situation, (b) a gesture for a response to the situation, (c) a prompt to accompany the response, and (d) a gestural annotation for the response, the training data including a hypothetical user query related to a custom domain,

the situation being a stimulus warranting a response from a natural language understanding (NLU) model,

the gesture for the response being a suggested gesture in response to the situation from the NLU model,

the prompt to accompany the response being a suggested phrase in response to the situation from the NLU model, and

the gestural annotation for the response being a process of labeling the response to show a gestural outcome to be predicted by the NLU model;

the training data comprising an utterance, an intent, an entity, a vocabulary, a gesture, or an action usable to respond to the situation;

tagging the hypothetical user query with the gestural annotation for the response; and

training the NLU model to address the situation, based on the training data, the training comprising:

based on the tagged hypothetical user query, training the NLU model to recognize a plurality of gesture types, and

based on recognizing a gesture type of the plurality of gesture types, train the NLU model to classify and annotate a real user query at a runtime based on the recognized gesture type.

8. The system of claim 7 , wherein the situation comprises a verbal query.

9. The system of claim 7 , wherein the situation comprises a detection of an entity selected from the group consisting of a person, an animal, and an object.

10. The system of claim 7 , wherein the situation comprises an environmental condition.

11. The system of claim 7 , wherein the gesture comprises a directional gesture.

12. The system of claim 7 , wherein the NLU model is utilized in a process that controls a bot.

13. A storage device that is non-transitory, comprising instructions that are readable by a processor to cause said processor to perform operations of:

obtaining training data that describes (a) a situation, (b) a gesture for a response to the situation, (c) a prompt to accompany the response, and (d) a gestural annotation for the response, the training data including a hypothetical user query related to a custom domain,

the situation being a stimulus warranting a response from a natural language understanding (NLU) model,

the gesture for the response being a suggested gesture in response to the situation from the NLU model,

the prompt to accompany the response being a suggested phrase in response to the situation from the NLU model, and

the gestural annotation for the response being a process of labeling the response to show a gestural outcome to be predicted by the NLU model;

the training data comprising an utterance, an intent, an entity, a vocabulary, a gesture, or an action usable to respond to the situation;

tagging the hypothetical user query with the gestural annotation for the response; and

training the NLU model to address the situation, based on the training data, the training comprising:

based on the tagged hypothetical user query, training the NLU model to recognize a plurality of gesture types, and

based on recognizing a gesture type of the plurality of gesture types, train the NLU model to classify and annotate a real user query at a runtime based on the recognized gesture type.

14. The storage device of claim 13 , wherein the situation comprises a verbal query.

15. The storage device of claim 13 , wherein the situation comprises a detection of an entity selected from the group consisting of a person, an animal, and an object.

16. The storage device of claim 13 , wherein the situation comprises a detection of an environmental condition.

17. The storage device of claim 13 , wherein the gesture comprises a directional gesture.

18. The storage device of claim 13 , wherein the NLU model is utilized in a process that controls a bot.

19. The method of claim 1 , further comprising:

recognizing a stimulus warranting a response, the stimulus comprising a user query;

based on the stimulus, performing gesture analysis on the stimulus using the trained NLU, the gesture analysis comprising:

classifying the stimulus; and

annotating the stimulus into a gestural prompt, the annotating comprising outputting tags based on the stimulus.

20. The method of claim 1 , wherein training the NLU further comprises using a conversational machine learning technique.