Input determination method
View Patent ↗Methods and systems for determining intent in voice and gesture interfaces are described. An example method includes determining that a gaze direction is in a direction of a gaze target, and determining whether a predetermined time period has elapsed while the gaze direction is in the direction of the gaze target. The method may also include providing an indication that the predetermined time period has elapsed when the predetermined time period has elapsed. According to the method, a voice or gesture command that is received after the predetermined time period has elapsed may be determined to be an input for a computing system. Additional example systems and methods are described herein.
1. A method comprising:
de-activating voice control of a voice-controlled device located remotely from a user, wherein the voice-controlled device comprises a gaze tracking component and an indicator component;
determining, using the gaze tracking component of the voice-controlled device, and while the voice control is deactivated, whether a gaze of a user is in a direction of a gaze target, wherein the gaze target comprises a portion of the voice-controlled device in which the indicator component and the gaze tracking component are positioned;
in response to determining that the gaze of the user is not in the direction of the gaze target, and while the voice control is deactivated, providing, using the indicator component and at the gaze target, a first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected and that the voice control is deactivated;
subsequent to providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected, and while the voice control is deactivated, determining, using the gaze tracking component of the voice-controlled device, that the gaze of the user is in the direction of the gaze target;
responsive to the subsequent determination that the gaze of the user is in the direction of the gaze target, and while the voice control is deactivated, switching from providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected to providing, to the user using the indicator component and at the gaze target, a second human-perceivable indication that indicates that: the gaze of the user is in the direction of the gaze target, the voice control is deactivated, and maintaining the gaze in the direction of the gaze target for a predetermined time period will activate the voice control;
subsequent to switching from providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected to providing the second human-perceivable indication that indicates that the gaze of the user is in the direction of the gaze target, and while the voice control is deactivated, determining, using the gaze tracking component of the voice-controlled device, that the predetermined time period has elapsed while the gaze of the user is in the direction of the gaze target;
responsive to determining that the predetermined time period has elapsed while the gaze of the user is in the direction of the gaze target:
switching from providing the second human-perceivable indication to providing, to the user using the indicator component and at the gaze target, a third human-perceivable indication that indicates that the gaze of the user has been maintained in the direction of the gaze target for at least the predetermined time period and that the voice control is activated, wherein the first, second, and third human-perceivable indications are different from one another, and
switching a voice input component from an off-state to an on-state; and
determining that a voice command that is received after switching the voice input component to the on-state is an input for the voice-controlled device.
2. The method of claim 1 , further comprising:
determining, using audio localization, a direction to a source of the voice command; and
determining the voice command is an input for the voice-controlled device when the direction to the source of the voice command matches a direction to an origin of the gaze of the user.
3. The method of claim 1 , further comprising:
determining, using audio localization, a direction to a source of another voice command; and
determining the other voice command is not an input for the voice-controlled device when the direction to the source of the other voice command matches a direction to a position other than the origin of the gaze of the user.
4. The method of claim 1 , further comprising:
determining the voice command is an input for the voice-controlled device when the voice command is received within a second predetermined time period of when the third human-perceivable indication is provided.
5. The method of claim 1 , wherein the indicator component comprises a light source.
6. The method of claim 1 , wherein the indicator component comprises a speaker.
7. The method of claim 1 , wherein determining that the gaze of the user is in the direction of the gaze target comprises:
determining an expected angle from an origin of the gaze of the user to a location of the gaze target; and
determining that the gaze of the user is in the direction of the gaze target based on a comparison of the expected angle and the gaze of the user.
8. The method of claim 7 , further comprising:
determining that the gaze of the user is within a threshold of the expected angle; and
providing feedback indicating a relationship between the gaze of the user and the expected angle, wherein the feedback includes information associated with a change in the gaze of the user to align the gaze of the user with the expected angle.
9. The method of claim 1 , further comprising once provision of the second human-perceivable indication is initiated, continuously providing the second human-perceivable indication while the gaze of the user is in the direction of the gaze target until it is determined that the predetermined time period has elapsed.
10. The method of claim 1 , wherein providing the second human-perceivable indication and providing the third human-perceivable indication comprise illuminating one or more of a plurality of light-emitting diodes.
11. The method of claim 10 :
wherein providing the second human-perceivable indication comprises initiating blinking of one or more of the plurality of light-emitting diodes, and
wherein providing the third human-perceivable indication comprises providing a sustained illumination of the one or more of the plurality of light-emitting diodes.
12. The method of claim 11 , wherein the plurality of light-emitting diodes forms at least a portion of a ring around the gaze target.
13. The method of claim 10 :
wherein providing the second human-perceivable indication comprises initiating blinking of one or more of the plurality of light-emitting diodes, and
wherein providing the third human-perceivable indication comprises increasing a rate of blinking of the one or more of the plurality of light-emitting diodes.
14. The method of claim 1 , further comprising:
obtaining, by the voice-controlled device, one or more images; and
processing, by the voice-controlled device, the one or more images to determine the gaze of the user.
15. A non-transitory computer-readable medium having stored therein instructions executable by a voice-controlled device to cause the voice-controlled device to perform functions comprising:
de-activating voice control of the voice-controlled device, wherein the voice-controlled device is located remotely from a user, and wherein the voice-controlled device comprises a gaze tracking component and an indicator component;
determining, using the gaze tracking component of the voice-controlled device, and while the voice control is deactivated, whether a gaze of a user is in a direction of a gaze target, wherein the gaze target comprises a portion of the voice-controlled device in which the indicator component and the gaze tracking component are positioned;
in response to determining that the gaze of the user is not in the direction of the gaze target, and while the voice control is deactivated, providing using the indicator component and at the gaze target, a first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected and that the voice control is deactivated;
subsequent to providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected, and while the voice control is deactivated, determining, using the gaze tracking component of the voice-controlled device, that a gaze of the user is in the direction of the gaze target;
responsive to the subsequent determination that the gaze of the user is in the direction of the gaze target, and while the voice control of the voice-controlled device is not activated, switching from providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected to providing, to the user using the indicator component and at the gaze target, a second human-perceivable indication that indicates that: the gaze of the user is in the direction of the gaze target, the voice control is deactivated, and that maintaining the gaze in the direction of the gaze target for a predetermined time period will activate the voice control;
subsequent to switching from providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected to providing the second human-perceivable indication that indicates that the gaze from the user is in the direction of the gaze target, and while the voice control is deactivated, determining, using the gaze tracking component of the voice-controlled device, that the predetermined time period has elapsed while the gaze of the user is in the direction of the gaze target;
responsive to determining that the predetermined time period has elapsed while the gaze of the user is in the direction of the gaze target:
switching from providing the second human-perceivable indication to providing, to the user using the indicator component and at the gaze target, a third human-perceivable indication that indicates that the gaze of the user has been maintained in the direction of the gaze target for at least the predetermined time period and that voice control is activated, wherein the first, second, and third human-perceivable indications are different from one another, and
switching a voice input component from an off-state to an on-state; and
determining that a voice command that is received after switching the voice input component to the on-state is an input for the voice-controlled device.
16. The non-transitory computer-readable medium of claim 15 , further comprising instructions executable by the voice-controlled device to perform functions comprising:
determining, using audio localization, a direction to a source of the voice command; and
determining the voice command is an input for the voice-controlled device when the direction to the source of the voice command matches a direction to an origin of the gaze of the user.
17. The non-transitory computer-readable medium of claim 15 , wherein the indicator component comprises a light source.
18. A voice-controlled device comprising:
a gaze tracking component located remotely from a user and configured to determine a gaze of the user;
a voice input component configured to receive a voice command;
an indicator component coupled to the gaze tracking component; and
instructions stored in a memory and executable by a processor to:
de-activate voice control of the voice-controlled device;
determine, using the gaze tracking component and while the voice control is deactivated, whether a gaze of the user is in a direction of a gaze target, wherein the gaze target comprises a portion of the voice-controlled device in which the indicator component and the gaze tracking component are positioned;
in response to determining that the gaze of the user is not in the direction of the gaze target, and while the voice control is deactivated, cause the indicator component to provide at the gaze target a first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected and that the voice control is deactivated;
subsequent to causing the indicator component to provide the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected, and while the voice control is deactivated, determine using the gaze tracking component that the gaze of the user is in the direction of the gaze target;
responsive to the subsequent determination that the gaze of the user is in the direction of the gaze target, and while the voice control is deactivated, cause the indicator component to switch from providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected to providing, to the user and at the gaze target, a second human-perceivable indication that indicates that: the gaze of the user is in the direction of the gaze target, the voice control is deactivated, and that maintaining the gaze in the direction of the gaze target for a predetermined time period will activate the voice control;
subsequent to causing the indicator component to switch from providing the first human-perceivable indication that indicates no gaze of the user in the direction of the gaze target is detected to providing the second human-perceivable indication that indicates that the gaze of the user is in the direction of the gaze target, and while the voice control is deactivated, determine using the gaze tracking component that a predetermined time period has elapsed while the gaze of the user is in the direction of the gaze target;
responsive to determining that the predetermined time period has elapsed while the gaze of the user is in the direction of the gaze target: (i) cause the indicator component to switch from providing the second human-perceivable indication to providing a third human-perceivable indication that indicates that the gaze of the user has been maintained in the direction of the gaze target for at least the predetermined time period and that the voice control is activated, wherein the first, second, and third human-perceivable indications are different from one another, and (ii) cause the voice input component to switch from an off-state to an on-state; and
determine that a voice command that is received after switching the voice input component to the on-state is an input for the voice-controlled device.
19. The voice-controlled device of claim 18 :
wherein the voice input component is further configured to determine a direction to a source of the voice command using audio localization; and
wherein the instructions further comprise instructions executable by the processor to:
determine the received voice command is an input for the voice-controlled device when the voice command is received after the predetermined time period has elapsed and the direction to the source of the voice command matches a direction to an origin of the gaze of the user.
20. The voice-controlled device of claim 19 , wherein the voice input component comprises a microphone array.
21. The voice-controlled device of claim 18 , wherein the indicator component comprises one or more light-emitting diodes coupled to the gaze tracking component.