Voice to text conversion based on third-party agent content
Implementations relate to dynamically, and in a context-sensitive manner, biasing voice to text conversion. In some implementations, the biasing of voice to text conversions is performed by a voice to text engine of a local agent, and the biasing is based at least in part on content provided to the local agent by a third-party (3P) agent that is in network communication with the local agent. In some of those implementations, the content includes contextual parameters that are provided by the 3P agent in combination with responsive content generated by the 3P agent during a dialog that: is between the 3P agent, and a user of a voice-enabled electronic device; and is facilitated by the local agent. The contextual parameters indicate potential feature(s) of further voice input that is to be provided in response to the responsive content generated by the 3P agent.
1. A voice-enabled electronic device comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the at least one processor to be operable to:
receive, from a third-party agent, one or more contextual parameters associated with the third-party agent, wherein the third-party agent is managed by an additional party that is distinct from the party that manages a local agent of the voice-enabled electronic device;
receive, from a user of the voice-enabled electronic device, a voice input provided by the user;
in response to receiving the voice input:
convert, using a voice to text model, the voice input to text, wherein the instructions to convert the voice input to text comprise instructions to use one or more of the contextual parameters to bias the voice to text model in converting at least one segment of the voice input to the text; and
transmit at least a portion of the text to the third-party agent.
2. The voice-enabled electronic device of claim 1 , wherein one or more of the contextual parameters comprise one or more particular tokens.
3. The voice-enabled electronic device of claim 1 , wherein one or more of the contextual parameters comprise one or more semantic types of tokens.
4. The voice-enabled electronic device of claim 1 , wherein one or more of the semantic types of tokens identify one or more of a time semantic type and a date semantic type.
5. The voice-enabled electronic device of claim 1 , wherein one or more of the contextual parameters comprise one or more semantic types of tokens and comprise one or more particular tokens.
6. The voice-enabled electronic device of claim 1 , wherein the instructions further cause the voice-enabled electronic device to:
receive, from the third-party agent and responsive to transmitting at least the portion of the text to the third-party agent, content that includes responsive content that is to be provided in response to the voice input; and
provide the responsive content as output for presentation to the user via the voice-enabled electronic device, the output being provided in response to the voice input.
7. The voice-enabled electronic device of claim 6 , wherein the instructions further cause the voice-enabled electronic device to:
receive, from the user of the voice-enabled electronic device, an additional voice input provided by the user, the additional voice input being provided in response to the output; and
use the content received from the third-party agent to convert the additional voice input to additional text.
8. The voice-enabled electronic device of claim 7 , wherein the content received from the third-party agent comprises one or more additional contextual parameters associated with the third-party agent, and wherein using the content received from the third-party agent to convert the additional voice input to the additional text comprises using one or more of the additional contextual parameter to bias the voice to text model in converting at least one segment of the additional voice input to the additional text.
9. The voice-enabled electronic device of claim 1 , wherein receiving one or more of the contextual parameters associated with the third-party agent is in response to an invocation of the third-party agent.
10. The voice-enabled electronic device of claim 1 , wherein the voice to text model is a streaming voice to text model.
11. A system comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the at least one processor to be operable to:
receive, from a third-party agent, one or more contextual parameters associated with the third-party agent, wherein the third-party agent is managed by an additional party that is distinct from the party that manages a local agent of a voice-enabled electronic device;
receive, from a user of the voice-enabled electronic device, a voice input provided by the user;
in response to receiving the voice input:
convert, using a voice to text model, the voice input to text, wherein the instructions to convert the voice input to text comprise instructions to use one or more of the contextual parameters to bias the voice to text model in converting at least one segment of the voice input to the text; and
transmit at least a portion of the text to the third-party agent.
12. The system of claim 11 , wherein one or more of the contextual parameters comprise one or more particular tokens.
13. The system of claim 11 , wherein one or more of the contextual parameters comprise one or more semantic types of tokens.
14. The system of claim 11 , wherein one or more of the semantic types of tokens identify one or more of a time semantic type and a date semantic type.
15. The system of claim 11 , wherein one or more of the contextual parameters comprise one or more semantic types of tokens and comprise one or more particular tokens.
16. The system of claim 11 , wherein the instructions further cause the voice-enabled electronic device to:
receive, from the third-party agent and responsive to transmitting at least the portion of the text to the third-party agent, content that includes responsive content that is to be provided in response to the voice input; and
provide the responsive content as output for presentation to the user via the voice-enabled electronic device, the output being provided in response to the voice input.
17. The system of claim 16 , wherein the instructions further cause the voice-enabled electronic device to:
receive, from the user of the voice-enabled electronic device, an additional voice input provided by the user, the additional voice input being provided in response to the output; and
use the content received from the third-party agent to convert the additional voice input to additional text.
18. The system of claim 17 , wherein the content received from the third-party agent comprises one or more additional contextual parameters associated with the third-party agent, and wherein using the content received from the third-party agent to convert the additional voice input to the additional text comprises using one or more of the additional contextual parameter to bias the voice to text model in converting at least one segment of the additional voice input to the additional text.
19. The system of claim 11 , wherein receiving one or more of the contextual parameters associated with the third-party agent is in response to an invocation of the third-party agent.
20. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to be operable to perform operations, the operations comprising:
receiving, from a third-party agent, one or more contextual parameters associated with the third-party agent, wherein the third-party agent is managed by an additional party that is distinct from the party that manages a local agent of a voice-enabled electronic device;
receiving, from a user of the voice-enabled electronic device, a voice input provided by the user;
in response to receiving the voice input:
converting, using a voice to text model, the voice input to text, wherein the instructions to convert the voice input to text comprise instructions to use one or more of the contextual parameters to bias the voice to text model in converting at least one segment of the voice input to the text; and
transmitting at least a portion of the text to the third-party agent.