Non-verbal engagement of a virtual assistant
Systems and methods related to engaging with a virtual assistant via ancillary input are provided. Ancillary input may refer to non-verbal, non-tactile input based on eye-gaze data and/or eye-gaze attributes, including but not limited to, facial recognition data, motion or gesture detection, eye-contact data, head-pose or head-position data, and the like. Thus, to initiate and/or maintain interaction with a virtual assistant, a user need not articulate an attention word or words. Rather the user may initiate and/or maintain interaction with a virtual assistant more naturally and may even include the virtual assistant in a human conversation with multiple speakers. The virtual assistant engagement system may utilize at least one machine-learning algorithm to more accurately determine whether a user desires to engage with and/or maintain interaction with a virtual assistant. Various hardware configurations associated with a virtual assistant device may allow for both near-field and/or far-field engagement.
1. A processor-implemented method for engaging a virtual assistant device, comprising:
receiving, by the virtual assistant device, eye-gaze data of a user as input;
upon receiving the eye-gaze data, collecting topology data associated with the electronic device, wherein the topology data is not collected using a head-mounted display;
analyzing one or more attributes of the eye-gaze data to determine whether the user intends to engage the virtual assistant device, wherein the analyzing comprises comparing an alignment of an engagement area associated with the virtual assistant device to an indicator area for detecting at least one of: the eye-gaze data or user proximity to the virtual assistant device, wherein the virtual assistant device is engaged when the engagement area and the indicator area are aligned; and
initiating a conversation with the user based on the one or more attributes of the eye-gaze data and the topology data, wherein initiating the conversation comprises providing a salutation to the user.
2. The processor-implemented method of claim 1 , wherein initiating the conversation comprises:
determining a response, wherein the response comprises at least one of: a verbal response, a transmission of data, and a non-verbal indication; and
providing the response to the user via the virtual assistant.
3. The processor-implemented method of claim 1 , wherein verifying that the user intends to engage the virtual assistant is based on applying at least one machine-learning algorithm.
4. The processor-implemented method of claim 1 , further comprising:
in response to initiating the conversation with the user, receiving, from the user, a response maintaining the conversation.
5. The processor-implemented method of claim 1 , wherein the one or more attributes of the eye-gaze data comprise at least two of: an eye-gaze signal, a facial feature, a facial expression, a gesture, or a body posture of a user.
6. The processor-implemented method of claim 5 , wherein analyzing the facial feature comprises evaluating at least one of: an eye feature, gaze-locking, or infrared eye reflection.
7. The processor-implemented method of claim 1 , further comprising:
determining an engagement confidence level indicated by the eye-gaze data.
8. The processor-implemented method of claim 1 , further comprising:
monitoring multiple users in proximity to the virtual assistant device.
9. The processor-implemented method of claim 8 , wherein monitoring multiple users includes utilizing facial-recognition technology.
10. The processor-implemented method of claim 1 , wherein the virtual assistant is configured to receive near-field user input via a first input detection mechanism of the virtual assistant and far-field user input via a second input detection mechanism of the virtual assistant.
11. The processor-implemented method of claim 10 , wherein the first input detection mechanism is a first hardware device and the second input detection mechanism is a second hardware device that is different from the first hardware device.
12. The processor-implemented method of claim 1 , wherein the user-specific data comprise previously captured images of at least one of: the user's face, the user's head poses, or the user's body position.
13. A system comprising:
a processor; and
memory coupled to the processor, the memory comprising computer executable instructions that, when executed, perform a method comprising:
receiving, by the virtual assistant device, eye-gaze data of a user as input;
upon receiving the eye-gaze data, collecting topology data associated with the electronic device, wherein the topology data is not collected using a head-mounted display;
analyzing one or more attributes of the eye-gaze data to determine whether the user intends to engage the virtual assistant device, wherein the analyzing comprises comparing an alignment of an engagement area associated with the virtual assistant device to an indicator area for detecting at least one of: the eye-gaze data or user proximity to the virtual assistant device, wherein the virtual assistant device is engaged when the engagement area and the indicator area are aligned; and
initiating a conversation with the user based on the one or more attributes of the eye-gaze data and the topology data, wherein initiating the conversation comprises providing a salutation to the user.
14. The system of claim 13 , wherein initiating the conversation comprises:
determining a response, wherein the response comprises at least one of: a verbal response, a transmission of data, and a non-verbal indication; and
providing the response to the user via the virtual assistant.
15. The system of claim 13 , wherein verifying that the user intends to engage the virtual assistant is based on applying at least one machine-learning algorithm.
16. The system of claim 13 , wherein the one or more attributes of the eye-g gaze data comprise at least two of: an eye-gaze signal, a facial feature, a facial expression, a gesture, or a body posture of a user.
17. The system of claim 16 , wherein analyzing the facial feature comprises evaluating at least one of: gaze-locking or infrared eye reflection.
18. A virtual assistant storing instructions that when executed by one or more processors of a computing device perform a method for analyzing eye-gaze input, the method comprising:
receiving, by the virtual assistant device, eye-gaze data of a user as input;
upon receiving the eye-gaze data, collecting topology data associated with the electronic device, wherein the topology data is not collected using a head-mounted display;
analyzing one or more attributes of the eye-gaze data to determine whether the user intends to engage the virtual assistant device, wherein the analyzing comprises comparing an alignment of an engagement area associated with the virtual assistant device to an indicator area for detecting at least one of: the eye-gaze data or user proximity to the virtual assistant device, wherein the virtual assistant device is engaged when the engagement area and the indicator area are aligned; and
initiating a conversation with the user based on the one or more attributes of the eye-gaze data and the topology data, wherein initiating the conversation comprises providing a salutation to the user.
19. The virtual assistant of claim 18 , wherein the virtual assistant is configured to monitor multiple users in proximity to the virtual assistant.
20. The virtual assistant of claim 19 , wherein monitoring the multiple users includes utilizing facial-recognition technology.