IP Library › Granted Patent US 12,229,585
Granted Patent B2
US 12,229,585 · App. 17/145,133 · Granted Feb 18, 2025

Back-end task fulfillment for dialog-driven applications

Inventors: Vikram Sathyanarayana Anbazhagan (Seattle, WA); Swaminathan Sivasubramanian (Sammamish, WA); Stefano Stefani (Issaquah, WA); Vladimir Zhukov (Seattle, WA)
Assignee: Amazon Technologies, Inc.
G06F9/4843G06F40/35
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,229,585
App. No.
17/145,133
Granted
Feb 18, 2025
Kind
B2
Abstract

A determination is made as to whether a value of a first parameter of a first application is to be obtained using a natural language interaction. Based on received input, a first service of a plurality of services is identified. The first service is to be used to perform a first task associated with the first parameter. Portions of the first application to determine the value of the first parameter and to invoke the first service are generated.

Claims (73)

1. A computer-implemented method, comprising:

prior to depolying an application to make it available for a user:

obtaining, via one or more programmatic interfaces of a dialog-driven-application management service of a cloud computing environment, a request to define a first intent of the application, wherein the first intent corresponds to a first task to be initiated on behalf of the user in response to one or more dialog inputs obtained from the user via on or more end user-related programmatic interfaces;

storing an indication of one or more parameters of the first intent; wherein the first task is to be performed based on the one or more parameters of the first internet;

provisioning, in the cloud computing environment, one or more computer resources to perform a least a first portion of the first task wherein the one or more compute resources includes one or more virtual machines allocated for the application, and

generating integration code to integrate the application with a fulfillment resource external to the cloud computing environment, wherein the fulfillment resource is configured to perform a least a second portion of the first task; and

after the application is deployed:

receiving, from the application, one or more respective values of the one or more parameters of the first intent determined based at least in part on processing of a dialog input from the user using one or more machine learning models, wherein the one or more machine learning models generates a match score of the dialog input with respect to the first intent and the match score exceeds a match threshold,

causing the one or more compute resources of the cloud computing environment to perform the first portion of the first task based on the respective values of the one or more parameters; and

causing the second portion of the first task to be performed at by the fulfillment resource external to the cloud computing environment.

2. The computer-implemented method as recited in claim 1 , further comprising:

obtaining, via the one or more programmatic interfaces, an indication of another service of the cloud computing environment which is to be used to perform the first portion of the first task, and

causing, during the execution of the application, the first portion of the first task to be performed at the other service.

3. The computer-implemented method as recited in claim 1 , further comprising:

obtaining, via the one or more programmatic interfaces, an indication of a sequence in which a plurality of resources are to be employed to perform one or more tasks of the application; and

causing, during the execution of the application, the one or more tasks to be performed at the plurality of resources in the indicated sequence.

4. The computer-implemented method as recited in claim 1 , further comprising:

obtaining, via the one or more programmatic interfaces, an indication of one or more initiation words for a dialog of the application, and

initiating execution of the application based at least in part on a detection, within input provided by the user, of an initiation word of the one or more initiation words.

5. The computer-implemented method as recited in claim 1 , wherein the integration code transmits one or more commands to a web service interface.

6. The computer-implemented method as recited in claim 1 , further comprising:

obtaining, via the one or more programmatic interfaces, an indication of a number of attempts to be made to determine a value of a first parameter of the one or more parameters before abandoning a dialog interaction of the application.

7. The computer-implemented method as recited in claim 1 , further comprising:

inserting, into a lexicon associated with the application, one or more words learned by a natural language understanding model from log records of user interactions with the application, and

utilizing the one or more learned words in one or more other user interactions.

8. A system, comprising:

one or more computing devices,

wherein the one or more computing devices include instructions that upon execution on or across the one or more computing devices cause the one or more computing devices to:

prior to deployment of an application to make it available for a user:

obtain, via one or more development-related programmatic interfaces of a dialog-driven application management service of a cloud computing environment, a request to define a first intent of the application, wherein the first intent corresponds to a first task to be initiated on behalf of the user in response to one or more dialog inputs obtained from the user via one or more end user-related programmatic interfaces,

store an indication of one or more parameters of the first intent, wherein the first task is to be performed based on the one or more parameters of the first intent;

provision, in the cloud computing environment, one or more compute resources to perform a least a first portion of the first task, wherein the one or more compute resources includes one or more virtual machines allocated for the application, and

generating integration code to integrate the application with a fulfillment resource external to the cloud computing environment, wherein the fulfillment resource is configured to perform a least a second portion of the first task; and

after the deployment of the application:

receive, from the application, one or more respective values of the one or more parameters of the first intent determined based at least in part on processing of a dialog input from the user using one or more machine learning model, wherein the one or more machine learning models generates a match score of the dialog input with respect to the first intent and the match score exceeds a match threshold,

cause the one or more compute resources of the cloud computing environment to perform the first portion of the first task based on the respective values of the one or more parameters; and

cause the second portion of the first task to be performed at by the fulfillment resource external to the cloud computing environment.

9. The system as recited in claim 8 , wherein the one or more computing devices include further instructions that upon execution on or across the one or more computing devices further cause the one or more computing devices to:

obtain, via the one or more programmatic interfaces, an indication of another service of the cloud computing environment which is to be used to perform the first portion of the first task, and

initiate, during the execution of the application, the first portion of the first task at the other service.

10. The system as recited in claim 8 , wherein the one or more computing devices include further instructions that upon execution on or across the one or more computing devices further cause the one or more computing devices to:

obtain, via the one or more programmatic interfaces, an indication of a sequence in which a plurality of resources are to be employed to perform one or more tasks of the application, and

initiate, during the execution of the application, the one or more tasks at the plurality of resources in the indicated sequence.

11. The system as recited in claim 8 , wherein the one or more computing devices include further instructions that upon execution on or across the one or more computing devices further cause the one or more computing devices to:

obtain, via the one or more programmatic interfaces, an indication of one or more initiation words for a dialog of the application; and

initiate execution of the application based at least in part on a detection, within input provided by the user, of an initiation word of the one or more initiation words.

12. The system as recited in claim 8 , wherein the integration code transmits one or more commands to a web service interface.

13. The system as recited in claim 8 , wherein the one or more computing devices include further instructions that upon execution on or across the one or more computing devices further cause the one or more computing devices to:

obtain, via the one or more programmatic interfaces, an indication of a number of attempts to be made to determine a value of a first parameter of the one or more parameters before abandoning a dialog interaction of the application.

14. The system as recited in claim 8 , wherein the one or more computing devices include further instructions that upon execution on or across the one or more computing devices further cause the one or more computing devices to:

inserting, into a lexicon associated with the application, one or more words learned by a natural language understanding model from log records of user interactions with the application, and utilizing the one or more learned words in one or more other user interactions.

15. One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors cause the one or more processors to:

prior to deployment of an application to make it available for a user:

obtain, via one or more development-related programmatic interfaces of a dialog-driven application management service of a cloud computing environment, a request to define a firstintent of the application, wherein the firstintent corresponds to a firsttask to be initiated on behalf of the user in response to one or more dialog inputs obtained from the user via one or more end user-related programmatic interfaces;

store an indication of one or more parameters of the firstintent, wherein the first task is to be performed based on the one or more parameters of the first intent,

provision, in the cloud computing environment, one or more compute resources to perform a least a first portion of the first task, wherein the one or more compute resources includes one or more virtual machines allocated for the application, and

generate integration code to integrate the application with a fulfillment resource external to the cloud computing environment, wherein the fulfillment resource is configured to perform a least a second portion of the first task; and

after the deployment of the application:

receive, from the application, one or more respective values of the one or more parameters of the firstintent determined based at least in part on processing of a dialog input from the user using one or more machine learning models, wherein the one or more machine learning models generates a match score of the dialog input with respect to the firstintent and the match score exceeds a match threshold;

cause the one or more compute resources of the cloud computing environment to perform the first portion of the first task based on the respective values of the one or more parameters; and

cause the second portion of the firsttask to be performed at by the fulfillment resource external to the cloud computing environment.

16. The one or more non-transitory computer-accessible storage media as recited in claim 15 , storing further program instructions that when executed on or across one or more processors further cause the one or more processors to:

obtain, via the one or more programmatic interfaces, an indication of another service of the cloud computing environment which is to be used to perform the first portion of the first task, and

initiate, during the execution of the application, the first portion of the first task at the other service.

17. The one or more non-transitory computer-accessible storage media as recited in claim 15 , storing further program instructions that when executed on or across one or more processors further cause the one or more processors to:

obtain, via the one or more programmatic interfaces, an indication of a sequence in which a plurality of resources are to be employed to perform one or more tasks of the application; and

initiate, during the execution of the application, the one or more tasks at the plurality of resources in the indicated sequence.

18. The one or more non-transitory computer-accessible storage media as recited in claim 15 , storing further program instructions that when executed on or across one or more processors further cause the one or more processors to:

obtain, via the one or more programmatic interfaces, an indication of one or more initiation words for a dialog of the application, and

initiate execution of the application based at least in part on a detection, within input provided by the user, of an initiation word of the one or more initiation words.

19. The one or more non-transitory computer-accessible storage media as recited in claim 15 , wherein the integration code transmits one or more commands to a web service interface.

20. The one or more non-transitory computer-accessible storage media as recited in claim 15 , storing further program instructions that when executed on or across one or more processors further cause the one or more processors to:

obtain, via the one or more programmatic interfaces, an indication of a number of attempts to be made to determine a value of a firstparameter of the one or more parameters before abandoning a dialog interaction of the application.

Continuity (2)
Continuation 15360818 · Nov 23, 2016
Related Publication 20210132986A1 · May 6, 2021
References Cited (52)
US 6246981B1 · Papineni et al. · 2001 [cited by applicant]
US 6415257B1 · Junqua et al. · 2002 [cited by applicant]
US 6510411B1 · Norton et al. · 2003 [cited by applicant]
US 7024348B1 · Scholz et al. · 2006 [cited by applicant]
US 7197460B1 · Gupta et al. · 2007 [cited by applicant]
US 9336494B1 · Purpura · 2016 [cited by examiner]
US 9742912B2 · Srivastava · 2017 [cited by examiner]
US 10891152B2 · Anbazhagan et al. · 2021 [cited by applicant]
US 20070143099A1 · Balchandran et al. · 2007 [cited by applicant]
US 20100298012A1 · Damarla · 2010 [cited by applicant]
US 20130102295A1 · Burke et al. · 2013 [cited by applicant]
US 20130268260A1 · Lundberg et al. · 2013 [cited by applicant]
US 20150263941A1 · Jung · 2015 [cited by applicant]
US 20160042748A1 · Jain et al. · 2016 [cited by applicant]
US 20160225370A1 · Kannan et al. · 2016 [cited by applicant]
US 20170116982A1 · Gelfenbeyn et al. · 2017 [cited by applicant]
US 20170125008A1 · Maisonnier et al. · 2017 [cited by applicant]
US 20170193387A1 · Lavallee · 2017 [cited by applicant]
US 20170286916A1 · Skiba et al. · 2017 [cited by applicant]
US 20170337478A1 · Sarkaya · 2017 [cited by applicant]
US 20170372200A1 · Chen · 2017 [cited by applicant]
US 20180075343A1 · van den Oord et al. · 2018 [cited by applicant]
US 20180090136A1 · Connell, II · 2018 [cited by examiner]
US 20180203852A1 · Goyal · 2018 [cited by applicant]
CN 102792320A · 2012 [cited by applicant]
CN 103095325A · 2013 [cited by applicant]
EP 2933070 · 2015 [cited by applicant]
WO 9723088 · 1997 [cited by applicant]
WO 2008054331 · 2008 [cited by applicant]
WO 2011088053A2 · 2011 [cited by applicant]
Yu, et al., “LSTM Encoder-ecorder for Dialogue Response Generation,” publisher: IEEE, published 2006 pp. 538-543. [cited by applicant]
Christian S. Perone, Machine Learning: Cosine Similarity for Vector Space Models (Part III),{ published: Terra Incognita, Dec. 9, 2013, pp. 1-4. [cited by applicant]
U.S. Appl. No. 15/197,022, filed Jun. 29, 2016 Simon Peter Reavely et al. [cited by applicant]
U.S. Appl. No. 15/248,211, filed Aug. 26, 2016, Shaun Nidhiri Joseph et al. [cited by applicant]
Robert Jamison, “Announcing a New Tool for Building Interactive Adventure Games on Alexa”, Amazon Mobile App Distribution Blog, Retrieved from URL: developer.amazon.com/public/community/post/TxEQV5K754YS77/Announcing-a-… [cited by applicant]
“Getting Started with the Alexa Skills Kit”, Amazon Apps & Games Developer Portal, Retrieved from URL: developer.amazon.com/pulbic/solutions/slexas/alexa-skills-kit/getting-started-guide on Oct. 30, 2016, pp. 1-7. [cited by applicant]
Seth Rosenberg, “How to Build Bots for Messenger”, Facebook for Developers, Retrieved from URL: developers.facebook.com/blog/post/2016/04/12/bots-for-messenger on Oct. 30, 2016, pp. 1-5. [cited by applicant]
Ali El-Kahky, et al., “Entending Domain Coverage of Language Understanding Systems via Intent Transfer Between Domains Using Knowledge Graphs and Search Query Click Logs”, 2014 IEEE International Conference on Acoustic,… [cited by applicant]
Elad Natanson, “Messaging Platforms, Bots and the Future of Mobile”, Retrieved from URL: www.forbes.com/sites/eladnatanson/2016/04/08/messaging-platforms-bot-and-the-future-of-mobile/#2d1ab79884af on Oct. 30, 2016. pp. … [cited by applicant]
“Messenger Platform”, Facebook for Developers, Retrieved from URL: developers.facebook.com/doc/messenger-platform on Oct. 30, 2016, pp. 1-3. [cited by applicant]
Collen Estrada, “Microsoft Bot Framework”, Mar. 30, 2016, Retrieved from URL: blog.botframework.com/2016/03/30/BotFramework/ on Oct. 30, 2016, pp. 1-7. [cited by applicant]
“Microsoft Cognitive Services—APIs”, Retrieved from URL: www.microsoft.com/cognitive-services/en-us/apis on Oct. 30, 2016, pp. 1-8. [cited by applicant]
Himanshu S. Bhatt, et al., “Cross-domain Text Classification with Multiple Domains and Disparate Label Sets”, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Aug. 7-12, 2016, pp.… [cited by applicant]
Amit Fulay, “Say hello to Google Allo: a smarter messaging app”, Retrieved from URL: blog.google/products/allo/google-allo-smater-messaging-app on Oct. 30, 2016, pp. 1-14. [cited by applicant]
“Training for the Alexa Skills Kit”, Amazon Apps & Games Developer Portal, Retrieved from URL: developer.amazon.com/public/solutions/alexa/alexa-skills-kits/content/alexa-skilss-developer-training on Oct. 30, 2016, pp. … [cited by applicant]
Wikipedia, “Vorbis”, Retrieved from URL: en.wikipedia.org/wiki/Vorbis on Sep. 26, 2016, pp. 1-10. [cited by applicant]
U.S. Appl. No. 15/360,814, filed Nov. 23, 2016, Vikram Sathyanarayana Anbazhagan, et al. [cited by applicant]
U.S. Appl. No. 15/841,122, filed Dec. 13, 2017, Rashmi Gangadhariah. [cited by applicant]
Claus Brabrand “PowerForms: Declarative client-side form field validation” Brics Report Series, Jan. 1, 2000, pp. 205-214. [cited by applicant]
Svetlana Stoyanchev et al “Rapid Prototyping of Form-driven Dialogue Systems Using an Open-Source Framework”, Proceddings of the Sigdial 2016 Conference, pp. 216-219. [cited by applicant]
Svetlana Stoyanchev et al., “Rapid Prototyping of Form-driven Dialogue Systems Using an Open-source Framework”, Proceedings of the SIGDIAL 2016 Conference, Sep. 13-15, 2016, pp. 216-219, Los Angeles, USA. [cited by applicant]
Office Action mailed Sep. 23, 2023 in Chinese Patent Application No. 201780072369.5, Amazon Technologies, Inc., pp. 1-14 (including translation). [cited by applicant]