Reducing latency caused by switching input modalities
Methods, apparatus, and computer-readable media (transitory and non-transitory) are provided herein for reducing latency caused by switching input modalities. In various implementations, a first input such as text input may be received at a first modality of a multimodal interface provided by an electronic device. In response to determination that the first input satisfies one or more criteria, the electronic device may preemptively establish a session between the electronic device and a query processor configured to process input received at a second modality (e.g., voice input) of the multimodal interface. In various implementations, the electronic device may receive a second input (e.g., voice input) at the second modality of the multimodal interface, initiate processing of at least a portion of the second input at the query processor within the session, and build a complete query based on output from the query processor.
1. A method comprising:
receiving a first input at a first modality of a multimodal interface associated with an electronic device;
determining that a context of the electronic device satisfies a criterion; and
in the electronic device, and responsive to receiving the first input and determining that the context of the electronic device satisfies the criterion:
preemptively establishing a session between the electronic device and a query processor configured to process input received at a second modality of the multimodal interface;
receiving a second input at the second modality of the multimodal interface;
initiating processing of at least a portion of the second input at the query processor within the session; and
building a complete query based on output from the query processor.
2. The method of claim 1 , wherein determining that the context of the electronic device satisfies the criterion comprises:
determining the context based on one or more signals from a sensor of the electronic device.
3. The method of claim 2 , wherein the sensor is an accelerometer.
4. The method of claim 1 , wherein building the complete query comprises:
combining the output with at least a portion of the first input.
5. The method of claim 1 , wherein the output from the query processor includes a plurality of candidate interpretations of the second input, and wherein building the complete query comprises:
ranking the plurality of candidate interpretations based at least in part on the first input.
6. The method of claim 1 , wherein the query processor is an online query processor, and wherein the electronic device comprises a mobile device configured to communicate with the online query processor when in communication with a wireless network.
7. The method of claim 1 , wherein initiating processing comprises:
sending data associated with the first input and data associated with the second input to the query processor.
8. The method of claim 1 , further comprising providing output to indicate that the session between the electronic device and the query processor is available.
9. The method of claim 1 , further comprising providing the complete query to a search service.
10. The method of claim 1 , further comprising using the query for one or more non-search actions.
11. The method of claim 10 , wherein the one or more non-search actions include initiating communication with another user.
12. The method of claim 10 , wherein the one or more non-search actions include setting a reminder.
13. An electronic device comprising:
a sensor;
memory storing instructions;
one or more processors configured to execute the instructions to:
receive a first input at a first modality of a multimodal interface associated with the electronic device;
determine, based on one or more signals from the sensor, that a context satisfies a criterion; and
responsive to receiving the first input and determining that the context satisfies the criterion:
preemptively establish a session with a query processor configured to process input received at a second modality of the multimodal interface;
receive a second input at the second modality of the multimodal interface;
initiate processing of at least a portion of the second input at the query processor within the session; and
build a complete query based on output from the query processor.
14. A method comprising:
receiving a first input at a first modality of a multimodal interface associated with an electronic device;
responsive to receiving the first input, determining that a criterion is satisfied;
in response to determining the criterion is satisfied, preemptively establishing a session between the electronic device and a query processor configured to process input received at a second modality of the multimodal interface;
receiving a second input at the second modality of the multimodal interface;
initiating processing of at least a portion of the second input at the query processor within the session; and
building a complete query based on output from the query processor, wherein the output from the query processor includes a plurality of candidate interpretations of the second input, and wherein building the complete query comprises:
ranking the plurality of candidate interpretations based at least in part on the first input.
15. The method of claim 14 , wherein determining that the criterion is satisfied comprises determining that the first input satisfies the criterion.
16. The method of claim 15 , wherein the criterion comprises the first input satisfying a character count threshold.
17. The method of claim 15 , wherein the criterion comprises the first input satisfying a word count threshold.
18. The method of claim 14 , wherein the query processor is an online query processor, and wherein the electronic device comprises a mobile device configured to communicate with the online query processor when in communication with a wireless network.
19. The method of claim 14 , wherein initiating processing comprises:
sending data associated with the first input and data associated with the second input to the query processor.
20. The method of claim 14 , further comprising providing output to indicate that the session between the electronic device and the query processor is available.