Natural language module store
View Patent ↗The present invention extends to methods, systems, and computer program products for a natural language module store. In general, the invention can be used to manage natural language modules offered through a natural language module store. Natural language module (NLM) developers can post NLMs at a NLM store to make the NLMs available for use by others. Developers can select NLMs for inclusion in natural language interpreters (NLIs) containing (and possibly integrating the functionality of) one or more NLMs. Prior to selecting a NLM, a developer can search or browse NLMs to identify an appropriate NLM. Optionally, a developer can test a NLM in the NLM store prior to inclusion in an NLI. For example, multiple NLMs purporting to provide the same specified natural language functionality can be tested relative to one another prior to selection of one of the NLMs for inclusion in an NLI.
1. A method comprising:
presenting a plurality of natural language modules and corresponding defined pricing models at a store, each of the plurality of natural language modules associated with a domain selected by a developer from among a plurality of domains;
receiving a natural language query for an application;
using a natural language processor to interpret the natural language query, the natural language processor including the plurality of natural language modules;
for each of the plurality of natural language modules:
the natural language module producing a meaning representation encoding semantics of the natural language query; and
calculating a charge for processing the natural language query, the charge based on the domain and determined in accordance with the pricing model defined for the natural language module; and
providing a meaning representation and applying the calculated charges.
2. The method of claim 1 , wherein the natural language query is a spoken command.
3. The method of claim 2 , wherein calculating the charge for processing the natural language query comprises calculating part of the charge based on the duration of verbal input used to issue the spoken command.
4. The method of claim 1 , wherein the natural language query comprises text input.
5. The method of claim 1 , wherein calculating a charge for processing the natural language query comprises calculating a module usage charge for using the natural language module to interpret the natural language query.
6. The method of claim 1 , wherein calculating a charge for processing the natural language query comprises:
determining an additional charge for use of a third party service, the third party service used to identify data or take action responsive to the natural language query; and
adding the additional charge to the charge for processing the natural language query.
7. The method of claim 6 , wherein determining an additional charge for use of a third party service comprises determining an additional charge for use of a third party service provided through a third party programming interface.
8. The method of claim 1 , wherein calculating a charge for processing the natural language query comprises calculating a per-query charge.
9. The method of claim 1 , wherein the natural language query comprises speech.
10. A method for assessing natural language recognition functionality, the method comprising:
selecting a plurality of natural language modules from a natural language store as candidates for assembly into a natural language interpreter, each of the plurality of natural language modules having a defined pricing model and configured to provide a domain of natural language functionality offered by the natural language interpreter;
for a subset of the plurality of natural language modules corresponding to the domain, testing each of the subsets of natural language modules, including:
exercising a test case including evaluating the ability of the natural language module to interpret a natural language query input to the natural language module into a meaning representation in the domain; and
verifying the accuracy of the interpretation; and
providing a natural language module, from among the subset of natural language modules, for assembly into the natural language interpreter based on the verified accuracy of the interpretation.
11. The method of claim 10 , further comprising comparing the accuracy of meaning representations for the plurality of natural language modules to one another in a side-by-side comparison.
12. The method of claim 11 , further comprising selecting a natural language module, from among the plurality of natural language modules, for assembly into the natural language interpreter based on accuracy of meaning representations indicated in the side-by-side comparison.
13. The method of claim 11 , wherein comparing the results meaning representations in a side-by-side comparison comprises testing the plurality of natural language modules using one or more pre-defined test use cases.
14. The method of claim 11 , wherein comparing the meaning representations in a side-by-side comparison comprises testing the performance of each of the plurality of natural language modules against one or more test databases.
15. The method of claim 10 , wherein selecting a plurality of natural language modules comprises entering access credentials for at least one private natural language module.
16. The method of claim 10 , wherein the natural language query comprises text.
17. The method of claim 10 , wherein the natural language query comprises speech.
18. A method for selecting a natural language module from a natural language module store comprising:
inputting a natural language query to the natural language module store, the natural language module store offering use of natural language modules in a domain for compensation in accordance with corresponding defined pricing models;
responsive to the natural language query, receiving from the natural language module store a list of natural language modules capable of interpreting at least a part of the natural language query to produce a meaning representation in the domain; and
selecting a natural language module, from among the plurality of natural language modules, for inclusion in a natural language interpreter to interpret natural language queries in the domain.
19. The method of claim 18 , wherein selecting a natural language module from a natural language module store comprises:
browsing through the plurality of natural language modules presented by the natural language module store; and
selecting the natural language module from among the plurality of natural language modules.
20. The method of claim 18 , wherein receiving from the natural language module store a list of a plurality of natural language modules comprises receiving a private natural language module; and
wherein selecting a natural language module from among the plurality of natural language modules comprises entering access credentials for the private natural language module.
21. A method for processing a natural language query, the method comprising:
presenting a plurality of natural language modules for a domain along with corresponding pricing models defined for each of the plurality of natural language modules at a natural language module store;
using each of the plurality of natural language modules to interpret the natural language query into a meaning representation in the domain;
choosing a natural language module, from among the plurality of natural language modules, based on the interpreting in the domain; and
calculating a charge for the chosen natural language module to process the natural language query, the charge determined in accordance with the corresponding pricing model defined for the chosen natural language module.
22. The method of claim 21 , wherein calculating a charge for processing the natural language query comprises calculating a per-query charge.
23. The method of claim 21 , wherein the natural language query comprises text.
24. The method of claim 21 , wherein the natural language query comprises speech.