Granted Patent B2

US 12,688,847 · App. 17/783,083 · Granted Jul 21, 2026

Inventors: Stefan Constantin (Karlsruhe, DE); Alexander Waibel (Sammamish, WA)

Assignee: Zoom Communications, Inc.

G10L15/01G10L15/063G10L15/22

Abstract

Claims (54)

1 . A system comprising:

a non-transitory computer-readable medium; and

one or more processor communicatively coupled to the non-transitory computer-readable medium, the one or more processors configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

receive, from a sensing means for sensing an operational-mode communication from a user,

a request indicating a user intention;

use a trained neural network to:

label every word token in the request;

determine one or more reparandums in the request based on the labels; and

determine one or more repair phrases in the request based on the labels;

use a second trained neural network to determine additional semantic information based on the request;

use the trained neural network to determine a corrected user intention based on the one or more reparandums and the one or more repair phrases and the additional semantic information;

update the second trained neural network based on first semantic information within the request;

use the trained neural network to generate one or more actions based on the corrected user intention; and

output one or more commands to a machine to cause the machine to perform the one or more actions.

2 . The system of claim 1 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

train the trained neural network through machine learning to output, for each training example in a training dataset, one or more updated actions; and wherein the training dataset comprises training communications, wherein the training communications each comprise a training user-intention and a training correction for the training user-intention.

3 . The system of claim 2 , wherein the trained neural network:

is trained to identify reparandums in the training user-intentions in the training dataset and to identify training repairs in corrections to the training user-intentions in the training dataset, and to generate, based on the identified reparandums and repairs in the training dataset, the corrected user intention; and

is configured to generate the one or more actions based on a reparandum identified in the request.

4 . The system of claim 3 , wherein the trained neural network is configured to generate the one or more actions based on a reparandum identified in the request and a repair.

5 . The system of claim 3 , wherein the one or more processors are configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to use a second neural network to determine relations between reparandums and the intention within the request.

6 . The system of claim 1 , wherein the trained neural network has a fixed size output vocabulary.

7 . The system of claim 3 , wherein the trained neural network is trained to assign a label to word tokens in communications in the training dataset and determine one or more corrected user-intention for each training example in the training dataset based on the assigned labels.

8 . The system of claim 1 , wherein the trained neural network does not have a fixed size output vocabulary.

9 . The system of claim 1 , wherein the machine comprises a processor-based device selected from the group consisting of a robot, a computer, a mobile device, an appliance, a home entertainment system, a personal assistant, an automotive system, a healthcare system, and a medical device.

10 . The system of claim 1 , wherein:

the request comprises audio from the user; and

the sensing means comprises a microphone and a Natural Language Processor (NLP).

11 . The system of claim 1 , wherein:

the request comprises an electronic message comprising text; and

the sensing means comprises a Natural Language Processor (NLP) for processing the text in the electronic message.

12 . The system of claim 1 , wherein the sensing means comprises a sensor selected from the group consisting of a motion sensor, a camera, a pressure sensor, a proximity sensor, a humidity sensor, an ambient light sensor, a GPS receiver, and a touch-sensitive display.

13 . The system of claim 1 , wherein the sensing means is part of the machine.

14 . The system of claim 1 , wherein the system is part of the machine.

15 . The system of claim 1 , wherein the request from the user comprises an imperative request from the user for the machine.

16 . The system of claim 1 , wherein the request sensed by the sensing means comprises a communication modality selected from the group consisting of text, speech, physical things like gestures, head movements, actions.

17 . The system of claim 1 , wherein the request sensed by the sensing means comprises a dialog stream, wherein the dialog stream comprises dialog from the user.

18 . A method comprising:

receiving, from a sensing means for sensing an operational-mode communication from a user,

a request from a user, the request indicating a user intention;

using a trained neural network to:

label every word token in the request;

determine one or more reparandums in the request based on the labels; and

determine one or more repair phrases in the request based on the labels;

using a second trained neural network to determine additional semantic information based on the request;

using the trained neural network to determine a corrected user intention based on one or more reparandums and the one or more repair phrases and the additional semantic information;

updating the second trained neural network based on first semantic information within the request;

generating, using the trained neural network, one or more actions based on the corrected user intention; and

outputting one or more commands to a robotic system to cause the robotic system to perform the one or more actions.

19 . The method of claim 18 , further comprising training the trained neural network through machine learning to output, for each training example in a training dataset, one or more corrected user-intention for the machine, wherein the machine is configured to act upon user-intentions from a user, wherein:

the training dataset comprises training communications, wherein the training communications each comprise a training user-intention for the machine and a training correction for the training user-intention.

20 . The method of claim 19 , wherein:

training the trained neural network comprises training the trained neural network to identify reparandums in the training user-intentions in the training dataset and to identify training repairs in corrections to the training user-intentions in the training dataset, and to generate, based on the identified reparandums and repairs in the training dataset, one or more corrected user-intentions for the machine; and

generating the corrected user intention comprises generating the corrected user intention based on a reparandum identified in the request and an operational-mode repair identified in the request.

Assignments (2)

Continuity (2)

References Cited (33)