IP Library Granted Patent US 11,790,611
Granted Patent B2
US 11,790,611 · App. 17/138,096 · Granted Oct 17, 2023

Visual editor for designing augmented-reality effects that utilize voice recognition

Inventors: Stef Marc Smet (London, GB); Hannes Luc Herman Verlinde (Ruislip, GB); Michael Slater (Nottingham, GB); Benjamin Patrick Blackburne (Baldock, GB); Ram Kumar Hariharan (Kirkland, WA); Chunjie Jia (Bothell, WA); Prakarn Nisarat (Seattle, WA)
Assignee: Meta Platforms, Inc.
G06T19/006G06F3/04815G10L15/22G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,790,611
App. No.
17/138,096
Granted
Oct 17, 2023
Kind
B2
Abstract

A computer-implemented method, comprising, by an artificial-reality (AR) design tool: receiving, through a user interface (UI) of the AR design tool, instructions to add a voice-command module to an AR effect, the voice-command module having an intent type and at least one slot, the slot associated with one or more entities; establishing, according to instructions received through the UI, a logical connection between the slot and a logic module configured to generate the AR effect depending on a runtime value associated with the slot; and generate, for the AR effect, an executable program configured to: determine that a detected utterance corresponds to the intent type and includes one or more words associated with the slot; select, based on the one or more words, one of the one or more entities as the runtime value for the slot; send the runtime value to the logic module according to the logical connection.

Claims (84)

1. A computer-implemented method, comprising, by an artificial-reality (AR) design tool:

receiving, through a user interface (UI) of the AR design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;

receiving, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect, the voice-command module having an intent type and at least one slot, the slot associated with one or more entities;

establishing, according to instructions received through the UI, a logical connection between the slot and a logic module configured to generate the AR effect depending on a runtime value associated with the slot; and

generating, for the AR effect, an executable program configured to:

detect an utterance at runtime;

determine that the detected utterance at runtime corresponds to the intent type and includes one or more words associated with the slot;

select, based on the one or more words, one of the one or more entities as the runtime value for the slot;

send the runtime value to the logic module according to the logical connection; and

after detecting the utterance at runtime, generate the AR effect based on the runtime value, received over the logical connection, that is selected based on the one or more words that are included by the detected utterance at runtime.

2. The computer-implemented method of claim 1 , further comprising:

customizing the voice-command module, wherein the customizing the voice-command module comprises:

receiving, through the UI, one or more additional entities for the slot; and

configuring the slot to be associated with the one or more additional entities.

3. The computer-implemented method of claim 2 , wherein selecting the runtime value for the slot further comprises:

selecting, based on the one or more words, one of the one or more additional entities as the runtime value for the slot.

4. The computer-implemented method of claim 1 , further comprising:

customizing the voice-command module, including:

receiving, through the UI, an additional slot for the voice-command module;

configuring the additional slot to be associated with one or more additional entities; and

establishing, according to instructions received through the UI, an additional logical connection between the additional slot and the logic module.

5. The computer-implemented method of claim 4 , wherein generating the executable program further comprises:

determining that the detected utterance at runtime corresponds to the intent type and includes one or more particular words associated with the additional slot;

selecting, based on the one or more particular words, one of the one or more additional entities as a particular runtime value for the additional slot; and

sending the particular runtime value to the logic module according to the logical connection.

6. The computer-implemented method of claim 1 , further comprising:

receiving, through the UI, a selection indicating synonyms of the one or more entities of the slot; and

configuring the slot to be associated with the one or more entities and the synonyms of the one or more entities.

7. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive, through a user interface (UI) of an artificial-reality (AR) design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;

receive, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect, the voice-command module having an intent type and at least one slot, the slot associated with one or more entities;

establish, according to instructions received through the UI, a logical connection between the slot and a logic module configured to generate the AR effect depending on a runtime value associated with the slot; and

generate, for the AR effect, an executable program configured to:

detect an utterance at runtime;

determine that the detected utterance at runtime corresponds to the intent type and includes one or more words associated with the slot;

select, based on the one or more words, one of the one or more entities as the runtime value for the slot;

send the runtime value to the logic module according to the logical connection; and

after detecting the utterance at runtime, generate the AR effect based on the runtime value, received over the logical connection, that is selected based on the one or more words that are included by the detected utterance at runtime.

8. The computer-readable non-transitory storage media of claim 7 , wherein the software is further operable whene executed to:

customize the voice-command module, wherein customizing the voice-command module comprises:

receiving, through the UI, one or more additional entities for the slot; and

configuring the slot to be associated with the one or more additional entities.

9. The computer-readable non-transitory storage media of claim 8 , wherein selecting the runtime value for the slot further comprises:

selecting, based on the one or more words, one of the one or more additional entities as the runtime value for the slot.

10. The computer-readable non-transitory storage media of claim 7 , wherein the software is further operable when executed to:

customize the voice-command module, wherein customizing the voice-command module comprises:

receiving, through the UI, an additional slot for the voice-command module;

configuring the additional slot to be associated with one or more additional entities; and

establishing, according to instructions received through the UI, an additional logical connection between the additional slot and the logic module.

11. The computer-readable non-transitory storage media of claim 10 , wherein generating the executable program further comprises:

determining that the detected utterance at runtime corresponds to the intent type and includes one or more particular words associated with the additional slot;

selecting, based on the one or more particular words, one of the one or more additional entities as a particular runtime value for the additional slot; and

sending the particular runtime value to the logic module according to the logical connection.

12. The computer-readable non-transitory storage media of claim 7 , wherein the software is further operable when executed to:

receive, through the UI, a selection indicating synonyms of the one or more entities of the slot; and

configure the slot to be associated with the one or more entities and the synonyms of the one or more entities.

13. A system comprising: one or more processors; and a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:

receive, through a user interface (UI) of an artificial-reality (AR) design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;

receive, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect, the voice-command module having an intent type and at least one slot, the slot associated with one or more entities;

establish, according to instructions received through the UI, a logical connection between the slot and a logic module configured to generate the AR effect depending on a runtime value associated with the slot; and

generate, for the AR effect, an executable program configured to:

detect an utterance at runtime;

determine that the detected utterance at runtime corresponds to the intent type and includes one or more words associated with the slot;

select, based on the one or more words, one of the one or more entities as the runtime value for the slot;

send the runtime value to the logic module according to the logical connection; and

after detecting the utterance at runtime, generate the AR effect based on the runtime value, received over the logical connection, that is selected based on the one or more words that are included by the detected utterance at runtime.

14. The system of claim 13 , wherein the processors are further operable when executing the instructions to:

customize the voice-command module, wherein customizing the voice-command module comprises:

receiving, through the UI, one or more additional entities for the slot; and

configuring the slot to be associated with the one or more additional entities.

15. The system of claim 14 , wherein selecting the runtime value for the slot further comprises:

selecting, based on the one or more words, one of the one or more additional entities as the runtime value for the slot.

16. The system of claim 13 , wherein the processors are further operable when executing the instructions to:

customize the voice-command module, wherein customizing the voice-command module comprises:

receiving, through the UI, an additional slot for the voice-command module;

configuring the additional slot to be associated with one or more additional entities; and

establishing, according to instructions received through the UI, an additional logical connection between the additional slot and the logic module.

17. The system of claim 16 , wherein generating the executable program further comprises:

determining that the detected utterance at runtime corresponds to the intent type and includes one or more particular words associated with the additional slot;

selecting, based on the one or more particular words, one of the one or more additional entities as a particular runtime value for the additional slot; and

sending the particular runtime value to the logic module according to the logical connection.

18. The system of claim 13 , wherein the processors are further operable when executing the instructions to:

receive, through the UI, a selection indicating synonyms of the one or more entities of the slot; and

configure the slot to be associated with the one or more entities and the synonyms of the one or more entities.

Assignments (2)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 3, 2021
From: SMET, STEF MARC; VERLINDE, HANNES LUC HERMAN; SLATER, MICHAEL; BLACKBURNE, BENJAMIN PATRICK; HARIHARAN, RAM KUMAR; JIA, CHUNJIE; NISARAT, PRAKARN
To: FACEBOOK, INC.
Reel/Frame 055133/0368 →
Cited By (1)
US 12,639,050