Methods and Systems for Tracking User Attention in Conversational Agent Systems
A method and system for tracking user attention is disclosed. The method and system receives, by an attention tracking component, a non-audio input from a user. The method and system receives, by a speech processing component, audio input from the user. The method and system determines, by a results processing component, that the attention tracking component received the non-audio input within a first time interval proximate to a second time interval during which the audio input was received. The method and system performs, by the results processing component, at least one actionable command identified within the audio input, based upon the determination that the attention tracking component received the non-audio input within the first time interval and a determination hat the received audio input includes the at least one actionable command.
1 . A method for tracking user attention in a conversational agent system, the method comprising:
receiving, by an attention tracking component, a non-audio input from a user;
receiving, by a speech processing component, audio input from the user;
determining, by a results processing component, that the attention tracking component received the non-audio input within a first time interval proximate to a second time interval during which the audio input was received; and
performing, by the results processing component, at least one actionable command identified within the audio input, based upon the determination that the attention tracking component received the non-audio input within the first time interval and a determination that the received audio input includes the at least one actionable command.
2 . The method of claim 1 , wherein receiving the non-speech input comprises determining that the user gazed at a visual focal point in physical proximity to the user.
3 . The method of claim 1 , wherein receiving, by the attention tracking component, the non-speech input comprises receiving physical input to a user interface element of the attention tracking component.
4 . The method of claim 1 further comprising transmitting, by the attention tracking component, to the results processing component, an indication of the first time interval during which the attention tracking component received the non-audio input.
5 . The method of claim 1 , wherein determining, by the results processing component, that the attention tracking component received the non-audio input within the first time interval proximate to the second time interval further comprises:
determining, by the results processing component, that the first time interval followed the second time interval during which the speech processing component received at least a portion of the audio input; and
storing, by the results processing component, the at least the portion of the audio input received during the second time interval.
6 . The method of claim 4 , wherein determining, by the results processing component, that the attention tracking component received the non-audio input within the first time interval proximate to the second time interval further comprises:
determining, by the results processing component, that the first time interval preceded the second time interval during which the speech processing component received at least a portion of the audio input; and
storing, by the speech processing component, the at least the portion of the audio input received during the second time interval.
7 . The method of claim 4 , wherein performing the at least one actionable command further comprises performing an actionable command included in audio input received at a time that preceded the first time.
8 . The method of claim 4 , wherein performing the at least one actionable command further comprises performing an actionable command included in audio input received at a time subsequent to the first time.
9 . The method of claim 1 further comprising determining, by the speech processing component, that the received audio input includes at least one actionable command.
10 . The method of claim 1 further comprising transmitting, by the attention tracking component, to the speech processing component, an audio signal indicating receipt of the non-audio input.
11 . The method of claim 1 further comprising transmitting, by the attention tracking component, to the speech processing component, an indication of receipt of the non-audio input.
12 . The method of claim 1 further comprising transmitting, by the attention tracking component, to the results processing component, an indication of receipt of the non-speech input.
13 . The method of claim 1 further comprising storing, by the speech processing component, the received audio input for subsequent processing.
14 . The method of claim 1 , wherein determining, by the speech processing component, that the received audio input includes at least one actionable command further comprises processing a subset of the received audio input.
15 . The method of claim 1 , wherein determining, by the speech processing component, that the received audio input includes at least one actionable command further comprises processing all received audio input.
16 . The method of claim 1 further comprises determining, by the speech processing component, that the received audio input includes a keyword.
17 . The method of claim 1 wherein receiving, by the speech processing component, the received audio input comprises receiving the audio input before receiving, by the attention tracking component, the non-audio input.
18 . A system for tracking user attention comprising:
an attention tracking component receiving non-audio input from a user;
a speech processing component receiving audio input from the user; and
a results processing component (i) determining that the attention tracking component received the non-audio input within a first time interval proximate to a second time interval during which the audio input was received, and (ii) performing at least one actionable command identified within the audio input, based on the determination that the attention tracking component received the non-audio input within the first time interval and a determination that the received audio input includes the at least one actionable command.
19 . The system of claim 18 , wherein the attention tracking component comprises a gaze tracker.
20 . The system of claim 18 , wherein the attention tracking component comprises functionality for identifying when the user gazes at a visual focal point.
21 . The system of claim 18 , wherein the attention tracking component comprises a button for receiving non-audio input.
22 . The system of claim 18 , wherein the attention tracking component comprises a foot-pedal for receiving non-audio input.
23 . A non-transitory, computer-readable medium encoded with computer-executable instructions that, when executed on a computing device, cause the computing device to carry out a method for tracking user attention in a conversational agent system, the method comprising:
receiving, by an attention tracking component, a non-audio input from a user;
receiving, by a speech processing component, audio input from the user;
determining, by a results processing component, that the attention tracking component received the non-audio input within a first time interval proximate to a second time interval during which the audio input was received; and
performing, by a results processing component, at least one actionable command identified within the audio input, based on the determination that the attention tracking component received the non-audio input within the first time interval and a determination that the received audio input includes the at least one actionable command.