Systems and methods for processing voice queries of different duration

1 . A method comprising:

receiving data associated with a first voice query, wherein the first voice query has a first time duration;

based on the first time duration falling within a first range of time durations, selecting a first server, of a plurality of servers, configured to process voice queries associated with the first range of time durations, wherein each of the plurality of servers is configured to process voice queries associated with a different range of time durations than other servers of the plurality of servers; and

sending the data associated with the first voice query for speech-to-text processing by the first server to determine text associated with the first voice query.

2 . The method of claim 1 , wherein at least one processing function is executing on the first server, and wherein the at least one processing function comprises one or more kernels configured to process voice queries associated with the first range of time durations.

3 . The method of claim 1 , wherein sending the data associated with the first voice query for speech-to-text processing by the first server is based on determining that at least one processing function executing on the first server is available to process the first voice query.

4 . The method of claim 1 , further comprising:

receiving data associated with a second voice query, wherein the second voice query has a second time duration;

based on the second time duration falling within a second range of time durations, determining a second server, of the plurality of servers, configured to process voice queries associated with the second range of time durations; and

sending the data associated with the second voice query for speech-to-text processing by the second server to determine text associated with the second voice query.

5 . The method of claim 1 , further comprising:

receiving data associated with a second voice query, wherein the second voice query has a second time duration that falls within the first range of time durations and is less than the first time duration;

padding the data associated with the second voice query to have a revised time duration that is equal to the first time duration; and

sending the data associated with the second voice query for speech-to-text processing by the first server to determine text associated with the second voice query.

6 . The method of claim 1 , further comprising:

receiving data associated with a second voice query, wherein the second voice query has a second time duration;

based on none of the plurality of servers being configured to process voice queries having the second time duration, determining a second server, of the plurality of servers, configured to process voice queries having a third time duration, wherein the second time duration is less than the third time duration and greater than the first time duration;

padding the data associated with the second voice query to have a revised time duration that is equal to the third time duration; and

sending the data associated with the second voice query for speech-to-text processing by the second server to determine text associated with the second voice query.

7 . The method of claim 1 , wherein at least one compute unified device architecture (CUDA) graph is executing on the first server, wherein the at least one CUDA graph comprises one or more kernels configured to process voice queries associated with the first range of time durations.

8 . A method comprising:

determining, by a server, a first length of a first voice query;

selecting, based on the first length falling within a first range of lengths, a first processing function of a plurality of processing functions configured to process voice queries associated with the first range of lengths;

sending data associated with the first voice query for speech-to-text processing by the first processing function to determine text associated with the first voice query;

determining, by the server, a second length of a second voice query;

selecting, based on the second length falling within a second range of lengths, a second processing function of the plurality of processing functions configured to process voice queries associated with the second range of lengths; and

sending data associated with the second voice query for speech-to-text processing by the second processing function to determine text associated with the second voice query.

9 . The method of claim 8 , wherein the first processing function comprises one or more kernels executing on the server and configured to process voice queries associated with the first range of lengths, and wherein the second processing function comprises one or more kernels executing on the server and configured to process voice queries associated with the second range of lengths.

10 . The method of claim 8 , further comprising:

receiving data associated with a third voice query, wherein the third voice query has a third length that falls within the first range of time durations, and wherein the third length is greater than the second length and less than the first length;

padding the data associated with the third voice query to have a revised length that is equal to the first length; and

sending the data associated with the third voice query for speech-to-text processing by the first processing function to determine text associated with the third voice query.

11 . The method of claim 8 , further comprising:

receiving data associated with a third voice query, wherein the third voice query has a third length;

based on none of the plurality of processing functions being configured to process voice queries having the third length, selecting a third processing function, of the plurality of processing functions, configured to process voice queries having a fourth length, wherein the third length is less than the fourth length and greater than the first length and the second length;

padding the data associated with the third voice query to have a revised length that is equal to the fourth length; and

sending the data associated with the third voice query for speech-to-text processing by the third processing function to determine text associated with the third voice query.

12 . The method of claim 8 , further comprising:

sending the data associated with the first voice query to the first processing function based on determining that the first processing function is available to execute the first voice query; and

sending the data associated with the second voice query to the second processing function based on determining that the second processing function is available to execute the second voice query.

13 . The method of claim 8 , wherein at least one of the first processing function or the second processing function are executing on at least one other server.

14 . The method of claim 8 , wherein each of the plurality of processing functions comprises a compute unified device architecture (CUDA) graph.

15 . A method comprising:

determining a rate of arrival associated with executed queries, wherein the executed queries are associated with a first plurality of lengths;

determining a distribution associated with the first plurality of lengths;

based at least on the rate of arrival and the distribution associated with the first plurality of lengths, determining a second plurality of lengths; and

based on the second plurality of lengths, initializing a plurality of processing functions.

16 . The method of claim 15 , wherein the rate of arrival indicates a rate at which the executed queries were received at one or more of an automatic speech recognition (ASR) system, a natural language understanding (NLU) system, or a machine translation system.

17 . The method of claim 15 , wherein initializing the plurality of processing functions comprises:

for each of the plurality of processing functions, recording a sequence of kernels associated with processing an unexecuted query associated with a particular length of the second plurality of lengths.

18 . The method of claim 15 , wherein initializing the plurality of processing functions comprises causing the plurality of processing functions to run on a plurality of servers.

19 . The method of claim 15 , further comprising:

receiving, by a server, a first unexecuted query associated with a first length of the second plurality of lengths;

based on sending data associated with the first unexecuted query to a first processing function of the plurality of processing functions, causing output associated with the first unexecuted query, wherein the first processing function is associated with the first length;

receiving, by the server, a second unexecuted query associated with a second length of the second plurality of lengths; and

based on sending data associated with the second unexecuted query to a second processing function of the plurality of processing functions, causing output associated with the second unexecuted query, wherein the second processing function is associated with the second length.

20 . The method of claim 15 , wherein each of the plurality of processing functions comprises a compute unified device architecture (CUDA) graph.

21 . The method of claim 15 , further comprising:

receiving data associated with a first unexecuted query, wherein the first unexecuted query has a first length of the second plurality of lengths;

based on the first length, selecting a first processing function of the plurality of processing functions, configured to process unexecuted queries having the first length; and

sending the data associated with the first unexecuted query for processing by the first server to cause output associated with the first unexecuted query.