IP Library › Granted Patent US 12,094,459
Granted Patent B2
US 12,094,459 · App. 17/568,960 · Granted Sep 17, 2024

Automated domain-specific constrained decoding from speech inputs to structured resources

Inventors: Ashish R Mittal (Bangalore, IN); Samarth Bharadwaj (Bangalore, IN); Shreya Khare (Bangalore, IN); Karthik Sankaranarayanan (Bangalore, IN)
Assignee: International Business Machines Corporation
G10L15/187G06F40/143G06F40/174G06N20/00G10L15/063G10L15/22G10L15/30G10L19/00H04L67/10G10L2015/0633G10L2015/0635G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,094,459
App. No.
17/568,960
Granted
Sep 17, 2024
Kind
B2
Abstract

Methods, systems, and computer program products for automated domain-specific constrained decoding from speech inputs to structured resources are provided herein. A computer-implemented method includes converting at least a portion of at least one user-provided speech utterance into text by processing the at least one user-provided speech utterance using an artificial intelligence-based automatic speech recognition model; automatically training an artificial intelligence-based decoding engine, wherein automatically training the artificial intelligence-based decoding engine comprising constraining the artificial intelligence-based decoding engine based at least in part on a domain-specific model and the artificial intelligence-based automatic speech recognition model; and generating at least one of one or more domain-specific text outputs related to one or more structured resources associated with the domain and one or more domain-specific action outputs related to the one or more structured resources associated with the domain by processing at least a portion of the text using the artificial intelligence-based decoding engine.

Claims (39)

1. A computer-implemented method comprising:

converting at least a portion of at least one user-provided speech utterance into text by processing the at least one user-provided speech utterance using an artificial intelligence-based automatic speech recognition model;

automatically training an artificial intelligence-based decoding engine, wherein automatically training the artificial intelligence-based decoding engine comprises constraining the artificial intelligence-based decoding engine based at least in part on a domain-specific model, one or more structured resources, and the artificial intelligence-based automatic speech recognition model, wherein the one or more structured resources comprise one or more web forms associated with the domain;

generating one or more domain-specific text outputs and automatically inserting at least a portion of the one or more domain-specific text outputs into at least one of the one or more web forms associated with the domain associated with the domain by processing at least a portion of the text using the artificial intelligence-based decoding engine; and

automatically performing one or more domain-specific actions within the one or more web forms associated with the domain by processing at least a portion of the text using the artificial intelligence-based decoding engine;

wherein the method is carried out by at least one computing device.

2. The computer-implemented method of claim 1 , further comprising:

automatically executing the one or more domain-specific action outputs.

3. The computer-implemented method of claim 2 , wherein automatically executing the one or more domain-specific action outputs comprises automatically selecting one or more radio buttons in connection with the one or more structured resources associated with the domain.

4. The computer-implemented method of claim 1 , wherein the artificial intelligence-based decoding engine comprises a set of one or more hierarchical long short-term memory decoders.

5. The computer-implemented method of claim 1 , wherein the artificial intelligence-based decoding engine comprises one or more combined attention mechanism and long short-term memory model (ATT-LSTM) components.

6. The computer-implemented method of claim 1 , wherein converting at least a portion of the at least one user-provided speech utterance into text comprises determining multiple phonetic variations of the user-provided speech utterance based at least in part on one or more phonemes by processing the at least one user-provided speech utterance using the artificial intelligence-based automatic speech recognition model.

7. The computer-implemented method of claim 1 , wherein the artificial intelligence-based automatic speech recognition model comprises a supervised encoder pertaining to speech representations.

8. The computer-implemented method of claim 1 , further comprising:

automatically training the domain-specific model using domain environment information.

9. The computer-implemented method of claim 1 , wherein the domain-specific model comprises at least one of a model pertaining to a specific language and a model pertaining to a specific topic.

10. The computer-implemented method of claim 1 , wherein the one or more web forms comprise one or more Hypertext Markup Language-based web forms.

11. The computer-implemented method of claim 1 , further comprising:

retraining the artificial intelligence-based decoding engine based at least in part on at least one of the one or more domain-specific text outputs and the one or more domain-specific action outputs.

12. The computer-implemented method of claim 1 , wherein software implementing the method is provided as a service in a cloud environment.

13. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

convert at least a portion of at least one user-provided speech utterance into text by processing the at least one user-provided speech utterance using an artificial intelligence-based automatic speech recognition model;

automatically train an artificial intelligence-based decoding engine, wherein automatically training the artificial intelligence-based decoding engine comprises constraining the artificial intelligence-based decoding engine based at least in part on a domain-specific model, one or more structured resources, and the artificial intelligence-based automatic speech recognition model, wherein the one or more structured resources comprise one or more web forms associated with the domain;

generate one or more domain-specific text outputs and automatically inserting at least a portion of the one or more domain-specific text outputs into at least one of the one or more web forms associated with the domain associated with the domain by processing at least a portion of the text using the artificial intelligence-based decoding engine; and

automatically perform one or more domain-specific actions within the one or more web forms associated with the domain by processing at least a portion of the text using the artificial intelligence-based decoding engine.

14. The computer program product of claim 13 , wherein the program instructions executable by a computing device to cause the computing device to:

automatically execute the one or more domain-specific action outputs.

15. The computer program product of claim 14 , wherein automatically executing the one or more domain-specific action outputs comprises automatically selecting one or more radio buttons in connection with the one or more structured resources associated with the domain.

16. A system comprising:

a memory configured to store program instructions; and

a processor operatively coupled to the memory to execute the program instructions to:

convert at least a portion of at least one user-provided speech utterance into text by processing the at least one user-provided speech utterance using an artificial intelligence-based automatic speech recognition model;

automatically train an artificial intelligence-based decoding engine, wherein automatically training the artificial intelligence-based decoding engine comprises constraining the artificial intelligence-based decoding engine based at least in part on a domain-specific model, one or more structured resources, and the artificial intelligence-based automatic speech recognition model, wherein the one or more structured resources comprise one or more web forms associated with the domain;

generate one or more domain-specific text outputs and automatically inserting at least a portion of the one or more domain-specific text outputs into at least one of the one or more web forms associated with the domain associated with the domain by processing at least a portion of the text using the artificial intelligence-based decoding engine; and

automatically perform one or more domain-specific actions within the one or more web forms associated with the domain by processing at least a portion of the text using the artificial intelligence-based decoding engine.

17. The system of claim 16 , wherein converting at least a portion of the at least one user-provided speech utterance into text comprises determining multiple phonetic variations of the user-provided speech utterance based at least in part on one or more phonemes by processing the at least one user-provided speech utterance using the artificial intelligence-based automatic speech recognition model.

18. The system of claim 16 , wherein the artificial intelligence-based decoding engine comprises a set of one or more hierarchical long short-term memory decoders.

19. The system of claim 16 , wherein the artificial intelligence-based decoding engine comprises one or more combined attention mechanism and long short-term memory model (ATT-LSTM) components.

20. The system of claim 16 , wherein the artificial intelligence-based automatic speech recognition model comprises a supervised encoder pertaining to speech representations.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2022
From: MITTAL, ASHISH R; BHARADWAJ, SAMARTH; KHARE, SHREYA; SANKARANARAYANAN, KARTHIK
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 058557/0197 →
Continuity (1)
Related Publication 20230215427A1 · Jul 6, 2023