Virtual assistant domain selection analysis
A virtual assistant platform provides a user interface for app developers to configure the enablement of domains for virtual assistants. Sets of test queries can be uploaded and statistical analyses displayed for the numbers of test queries served by each selected domain and costs for usage of each domain. Costs can vary according to complex pricing models. The user interface provides display views of tables, cost stack charts, and histograms to inform decisions that trade-off costs with benefits to the virtual assistant user experience. The platform interface shows, for individual queries, responses possible from different domains. Platform providers promote certain chosen domains.
1 . A computer-implemented method of selecting components for inclusion in configuring a virtual assistant, the method comprising, performing by a hardware processor, actions including:
receiving a multiplicity of test queries and a selection of a plurality of selected domains, wherein the selected domains have domain grammars that define semantic arguments expected in queries;
parsing each test query, using a neural network that has been trained based on example queries, the selected domains and the domain grammars that define the semantic arguments expected in the example queries, and deriving semantic arguments responsive to the domain grammars;
testing, for each interpreted test query, the selected domains that are able to respond to the semantic arguments derived from each interpreted test query;
evaluating responses from the tested domains, for highest rates of affirmatively responding to the semantic arguments from the interpreted test queries; and
including in a build of the virtual assistant those selected domains having the highest rates of affirmative responses.
2 . The computer-implemented method of claim 1 , further comprising a step of including in the build of the virtual assistant to include a set of one or more domains having a rate of affirmative response below the highest rate, where the set of one or more domains having a rate of affirmative response below the highest rate offer incentives for inclusion into the virtual assistant.
3 . The computer-implemented method of claim 1 , further comprising a step of producing a histogram showing for each domain the number of interpreted test queries that were successfully responded to.
4 . A computer-implemented method of selecting components for inclusion in a virtual assistant, the method comprising software running on a hardware processor:
receiving a multiplicity of test queries and a selection of a plurality of selected domains, wherein the selected domains have domain grammars that define semantic arguments expected in queries;
interpreting each test query, using a neural network that has been trained based on the domain grammars, and deriving semantic arguments responsive to the domain grammars;
testing, for each interpreted test query, the selected domains that are able to respond to the semantic arguments derived from each interpreted test query;
evaluating responses from the tested domains, for highest rates of affirmatively responding to the semantic arguments from the interpreted test queries; and
including for a build of the virtual assistant those selected domains having the highest rates of affirmative responses.
5 . The computer-implemented method of claim 4 , wherein said step of interpreting each test query into the semantic arguments is performed for the plurality of domains using one or more algorithms associated with each of the domains.
6 . The computer-implemented method of claim 4 , further comprising a step of including in the build of the virtual assistant to include a set of one or more domains having a rate of affirmative response below the highest rate, where the set of one or more domains having a rate of affirmative response below the highest rate offer incentives for inclusion into the virtual assistant.
7 . The computer-implemented method of claim 4 , further comprising a step of producing a histogram showing for each domain the number of interpreted test queries that were successfully responded to.
8 . A non-transitory computer readable medium holding instructions that, when executed on hardware, cause the hardware to carry out actions comprising:
receiving a multiplicity of test queries and a selection of a plurality of selected domains, wherein the selected domains have domain grammars that define semantic arguments expected in queries;
interpreting each test query, using a neural network that has been trained based on the domain grammars, and deriving semantic arguments responsive to the domain grammars;
testing, for each interpreted test query, the selected domains that are able to respond to the semantic arguments derived from each interpreted test query;
evaluating responses from the tested domains, for highest rates of affirmatively responding to the semantic arguments from the interpreted test queries; and
including for a build of the virtual assistant those selected domains having the highest rates of affirmative responses.
9 . The non-transitory computer readable medium of claim 8 , the actions further comprising interpreting each test query using the neural network.
10 . The non-transitory computer readable medium of claim 8 , the actions further comprising interpretation of each test query into the semantic arguments being requested by the plurality of domains.
11 . A non-transitory computer readable medium holding instructions that, when executed on hardware, cause the hardware to carry out actions comprising:
receiving a multiplicity of test queries and a selection of a plurality of selected domains, wherein the selected domains have domain grammars that define semantic arguments expected in queries;
parsing each test query and deriving one or more semantic arguments expected by the domain grammars;
testing, for each interpreted test query, the selected domains that are able to respond to the semantic arguments derived from each interpreted test query;
evaluating responses from the tested domains, for highest rates of affirmatively responding to the semantic arguments from the interpreted test queries; and
including for a build of the virtual assistant those selected domains having the highest rates of affirmative responses.
12 . The non-transitory computer readable medium of claim 11 , wherein said action of parsing each test query into the semantic arguments is performed for the plurality of domains using one or more algorithms associated with each of the domains.
13 . The non-transitory computer readable medium of claim 11 , further comprising an action of including in the build of the virtual assistant to include a set of one or more domains having a rate of affirmative response below the highest rate, where the set of one or more domains having a rate of affirmative response below the highest rate offer incentives for inclusion into the virtual assistant.
14 . The non-transitory computer readable medium of claim 11 , further comprising an action of producing a histogram showing for each domain the number of interpreted test queries that were successfully responded to.