Zero latency digital assistant
An electronic device can implement a zero-latency digital assistant by capturing audio input from a microphone and using a first processor to write audio data representing the captured audio input to a memory buffer. In response to detecting a user input while capturing the audio input, the device can determine whether the user input meets a predetermined criteria. If the user input meets the criteria, the device can use a second processor to identify and execute a task based on at least a portion of the contents of the memory buffer.
1 . A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device with a microphone, cause the electronic device to:
capture first audio input from the microphone, wherein the first audio input includes speech from a user of the electronic device;
write data representing the first audio input to a memory buffer, wherein the data includes at least a first portion of data representing at least a portion of the speech from a user of the electronic device;
while capturing the first audio input using the microphone and after writing the first portion of data to the memory buffer, receive a user input invoking a digital assistant, wherein the user input is distinct from the first portion of data; and
in response to receiving the user input invoking the digital assistant, identify a computing task based on at least the first portion of data, wherein identifying the computing task includes identifying, based on at least the first portion of data, an actionable intent corresponding to the computing task.
2 . The non-transitory computer-readable storage medium according to claim 1 , wherein identifying the computing task is further based on a second portion of data written to the memory buffer during detection of the user input.
3 . The non-transitory computer-readable storage medium according to claim 1 , further comprising instructions to cause the electronic device to:
further response to receiving the user input invoking the digital assistant, execute the identified computing task.
4 . The non-transitory computer-readable storage medium according to claim 1 , wherein identifying the computing task includes launching a digital assistant session.
5 . The non-transitory computer-readable storage medium according to claim 4 , wherein the digital assistant session identifies and executes the computing task.
6 . The non-transitory computer-readable storage medium according to claim 4 , wherein launching the digital assistant session includes displaying a user interface associated with the digital assistant session.
7 . The non-transitory computer-readable storage medium according to claim 6 , wherein the user interface associated with the digital assistant session is displayed in a full-screen view.
8 . The non-transitory computer-readable storage medium according to claim 4 , wherein launching the digital assistant session includes activating one or more audio components on the electronic device.
9 . The non-transitory computer-readable storage medium according to claim 4 , wherein the first portion of data is provided to a remote server associated with the digital assistant session.
10 . The non-transitory computer-readable storage medium according to claim 4 , further comprising instructions to cause the electronic device to:
further in response to receiving the user input invoking the digital assistant:
activate a second microphone on the electronic device, and
stream audio detected by the second microphone to the digital assistant session.
11 . The non-transitory computer-readable storage medium according to claim 1 , wherein the user input invoking the digital assistant is a button press.
12 . The non-transitory computer-readable storage medium according to claim 1 , wherein the user input invoking the digital assistant is audio data captured by the microphone and written to the buffer.
13 . The non-transitory computer-readable storage medium according to claim 1 , wherein the user input invoking the digital assistant is a user input determined to meet predetermined criteria.
14 . The non-transitory computer-readable storage medium according to claim 13 , wherein the predetermined criteria include a criterion that is met when the electronic device determines that at least a second portion of the memory buffer includes audio data representing a predetermined trigger.
15 . The non-transitory computer-readable storage medium according to claim 13 , wherein the predetermined criteria include a criterion that is met when the electronic device determines that the first audio input corresponds to an authorized user.
16 . The non-transitory computer-readable storage medium according to claim 1 , wherein identifying the computing task further includes identifying one or more parameters corresponding to the actionable intent.
17 . A method, comprising:
at an electronic device including a microphone and one or more processors:
capturing first audio input from the microphone, wherein the first audio input includes speech from a user of the electronic device;
writing data representing the first audio input to a memory buffer, wherein the data includes at least a first portion of data representing at least a portion of the speech from a user of the electronic device;
while capturing the first audio input using the microphone and after writing the first portion of data to the memory buffer, receive a user input invoking a digital assistant, wherein the user input is distinct from the first portion of data; and
in response to receiving the user input invoking the digital assistant, identifying a computing task based on at least the first portion of data, wherein identifying the computing task includes identifying, based on at least the first portion of data, an actionable intent corresponding to the computing task.
18 . The method according to claim 17 , wherein identifying the computing task is further based on a second portion of data written to the memory buffer during detection of the user input.
19 . The method according to claim 17 , further comprising:
further response to receiving the user input invoking the digital assistant, executing the identified computing task.
20 . The method according to claim 17 , wherein identifying the computing task includes launching a digital assistant session.
21 . The method according to claim 20 , wherein the digital assistant session identifies and executes the computing task.
22 . The method according to claim 20 , wherein launching the digital assistant session includes displaying a user interface associated with the digital assistant session.
23 . The method according to claim 22 , wherein the user interface associated with the digital assistant session is displayed in a full-screen view.
24 . The method according to claim 20 , wherein launching the digital assistant session includes activating one or more audio components on the electronic device.
25 . The method according to claim 20 , wherein the first portion of data is provided to a remote server associated with the digital assistant session.
26 . The method according to claim 20 , further comprising:
further in response to receiving the user input invoking the digital assistant:
activating a second microphone on the electronic device, and
streaming audio detected by the second microphone to the digital assistant session.
27 . The method according to claim 17 , wherein the user input invoking the digital assistant is a button press.
28 . The method according to claim 17 , wherein the user input invoking the digital assistant is audio data captured by the microphone and written to the buffer.
29 . The method according to claim 17 , wherein the user input invoking the digital assistant is a user input determined to meet predetermined criteria.
30 . The method according to claim 29 , wherein the predetermined criteria include a criterion that is met when the electronic device determines that at least a second portion of the memory buffer includes audio data representing a predetermined trigger.
31 . The method according to claim 29 , wherein the predetermined criteria include a criterion that is met when the electronic device determines that the first audio input corresponds to an authorized user.
32 . The method according to claim 17 , wherein identifying the computing task further includes identifying one or more parameters corresponding to the actionable intent.
33 . An electronic device, comprising:
a microphone;
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
capturing first audio input from the microphone, wherein the first audio input includes speech from a user of the electronic device;
writing data representing the first audio input to a memory buffer in the memory, wherein the data includes at least a first portion of data representing at least a portion of the speech from a user of the electronic device;
while capturing the first audio input using the microphone and after writing the first portion of data to the memory buffer, receive a user input invoking a digital assistant, wherein the user input is distinct from the first portion of data; and
in response to receiving the user input invoking the digital assistant, identifying a computing task based on at least the first portion of data, wherein identifying the computing task includes identifying, based on at least the first portion of data, an actionable intent corresponding to the computing task.
34 . The electronic device according to claim 33 , wherein identifying the computing task is further based on a second portion of data written to the memory buffer during detection of the user input.
35 . The electronic device according to claim 33 , the one or more programs further including instructions for:
further response to receiving the user input invoking the digital assistant, executing the identified computing task.
36 . The electronic device according to claim 33 , wherein identifying the computing task includes launching a digital assistant session.
37 . The electronic device according to claim 36 , wherein the digital assistant session identifies and executes the computing task.
38 . The electronic device according to claim 36 , wherein launching the digital assistant session includes displaying a user interface associated with the digital assistant session.
39 . The electronic device according to claim 38 , wherein the user interface associated with the digital assistant session is displayed in a full-screen view.
40 . The electronic device according to claim 36 , wherein launching the digital assistant session includes activating one or more audio components on the electronic device.
41 . The electronic device according to claim 36 , wherein the first portion of data is provided to a remote server associated with the digital assistant session.
42 . The electronic device according to claim 36 , the one or more programs further including instructions for:
further in response to receiving the user input invoking the digital assistant:
activating a second microphone on the electronic device, and
streaming audio detected by the second microphone to the digital assistant session.
43 . The electronic device according to claim 33 , wherein the user input invoking the digital assistant is a button press.
44 . The electronic device according to claim 33 , wherein the user input invoking the digital assistant is audio data captured by the microphone and written to the buffer.
45 . The electronic device according to claim 33 , wherein the user input invoking the digital assistant is a user input determined to meet predetermined criteria.
46 . The electronic device according to claim 45 , wherein the predetermined criteria include a criterion that is met when the electronic device determines that at least a second portion of the memory buffer includes audio data representing a predetermined trigger.
47 . The electronic device according to claim 45 , wherein the predetermined criteria include a criterion that is met when the electronic device determines that the first audio input corresponds to an authorized user.
48 . The electronic device according to claim 33 , wherein identifying the computing task further includes identifying one or more parameters corresponding to the actionable intent.