IP Library Granted Patent US 12,131,733
Granted Patent B2
US 12,131,733 · App. 17/521,623 · Granted Oct 29, 2024

Active listening for assistant systems

Inventors: Leif Haven Martinson (Oakland, CA); David Levison (Sebastopol, CA); Heath William Black (Alameda, CA); Ryan Frederick Stewart (Sammamish, WA); Tara Ramanan (Redmond, WA); Samuel Steele Noertker (Redmond, WA)
Assignee: Meta Platforms, Inc.
G10L15/22G10L15/08G10L15/30G10L2015/088G10L2015/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,131,733
App. No.
17/521,623
Filed
Nov 8, 2021
Granted
Oct 29, 2024
Kind
B2
Art Unit
2658
USPC
704/251
Abstract

In one embodiment, a method includes receiving a first user input comprising a wake word associated with an assistant xbot from a first client system, setting the assistant xbot into a listening mode, wherein a continuous non-visual feedback is provided via the first client system while the assistant xbot is in the listening mode, receiving a second user input comprising a user utterance from the first client system while the assistant xbot is in the listening mode, determining the second user input has ended based on a completion of the user utterance, and setting the assistant xbot into an inactive mode, wherein the non-visual feedback is discontinued via the first client system while the assistant xbot is in the inactive mode.

Claims (41)

1. A method comprising, by a computing system, the computing system comprising a client system and a companion device paired to the client system:

receiving, at the companion device from the client system, a first user input comprising a wake word associated with an assistant xbot;

setting the assistant xbot into a listening mode, wherein a continuous non-visual feedback is provided via the client system while the assistant xbot is in the listening mode;

receiving, at the companion device from the client system, a second user input comprising a user utterance while the assistant xbot is in the listening mode;

determining the second user input has ended based on a completion of the user utterance;

setting the assistant xbot into an inactive mode, wherein the non-visual feedback is discontinued via the client system while the assistant xbot is in the inactive mode; and

providing, via the companion device, a continuous visual feedback while the assistant xbot is in the listening mode.

2. The method of claim 1 , wherein the companion device is in a locked state when the wake word is received.

3. The method of claim 1 , wherein the assistant xbot is associated with a companion application executing on the companion device.

4. The method of claim 3 , wherein the companion application is executing as a background application on the companion device, and wherein the companion application is allowed by the companion device to access a computing capacity lower than a threshold capacity.

5. The method of claim 3 , further comprising:

executing the companion application as a foreground application in response to the continuous non-visual feedback; and

increasing the computing capacity the companion application is allowed to access a computing capacity greater than a threshold capacity.

6. The method of claim 1 , wherein the client system comprises one of a smart phone, smart glasses, augmented-reality (AR) glasses, a virtual-reality (VR) headset, or a smart watch.

7. The method of claim 1 , further comprising:

providing, via the client system in response to the first user input, an initial non-visual feedback before the continuous non-visual feedback, wherein the initial non-visual feedback indicates the initiation of the listening mode of the assistant xbot.

8. The method of claim 1 , wherein the continuous non-visual feedback is based on one or more of sound, vibration, or haptics.

9. The method of claim 1 , further comprising:

providing, via the client system, a continuous visual feedback while the assistant xbot is in the listening mode.

10. The method of claim 9 , wherein the continuous visual feedback is based on one or more of an icon associated with the assistant xbot, a visual indication of the listening mode, or light.

11. The method of claim 1 , wherein the continuous visual feedback is based on one or more of an icon associated with the assistant xbot, a visual indication of the listening mode, or light.

12. One or more computer-readable non-transitory non-volatile storage media embodying software that is operable when executed to:

receive, from a client system, a first user input comprising a wake word associated with an assistant xbot;

set the assistant xbot into a listening mode, wherein a continuous non-visual feedback is provided via the client system while the assistant xbot is in the listening mode;

receive, from the client system, a second user input comprising a user utterance while the assistant xbot is in the listening mode;

determine the second user input has ended based on a completion of the user utterance;

set the assistant xbot into an inactive mode, wherein the non-visual feedback is discontinued via the client system while the assistant xbot is in the inactive mode; and

provide, via a companion device paired to the client system, a continuous visual feedback while the assistant xbot is in the listening mode,

wherein the assistant xbot is associated with a companion application configured by the software.

13. The computer-readable non-transitory non-volatile storage media of claim 12 , wherein the client system comprises one or more of a smart phone, smart glasses, augmented-reality (AR) glasses, a virtual-reality (VR) headset, or a smart watch.

14. The computer-readable non-transitory non-volatile storage media of claim 12 , wherein the software is further operable when executed to: provide, via the client system in response to the first user input, an initial non-visual feedback before the continuous non-visual feedback, wherein the initial non-visual feedback indicates the initiation of the listening mode of the assistant xbot.

15. A system comprising: one or more processors; and a non-transitory non-volatile memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:

receive, from a client system, a first user input comprising a wake word associated with an assistant xbot;

set the assistant xbot into a listening mode, wherein a continuous non-visual feedback is provided via the client system while the assistant xbot is in the listening mode;

receive, from the client system, a second user input comprising a user utterance while the assistant xbot is in the listening mode;

determine the second user input has ended based on a completion of the user utterance;

set the assistant xbot into an inactive mode, wherein the non-visual feedback is discontinued via the client system while the assistant xbot is in the inactive mode; and

provide, via a companion device paired to the client system, a continuous visual feedback while the assistant xbot is in the listening mode,

wherein the assistant xbot is associated with a companion application configured by the instructions and executed by the processors.

16. The system of claim 15 , wherein the client system comprises one or more of a smart phone, smart glasses, augmented-reality (AR) glasses, a virtual-reality (VR) headset, or a smart watch.

17. The system of claim 15 , wherein the processors are further operable when executing the instructions to: provide, via the client system in response to the first user input, an initial non-visual feedback before the continuous non-visual feedback, wherein the initial non-visual feedback indicates the initiation of the listening mode of the assistant xbot.

Assignments (3)
CHANGE OF NAME Recorded Jan 5, 2022
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058645/0454 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 3, 2022
From: MARTINSON, LEIF HAVEN; LEVISON, DAVID; BLACK, HEATH WILLIAM; STEWART, RYAN FREDERICK; RAMANAN, TARA; NOERTKER, SAMUEL STEELE
To: FACEBOOK, INC.
Reel/Frame 058531/0267 →
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
Continuity (2)
Provisional Application 63177812 · Apr 21, 2021
Related Publication 20220366904A1 · Nov 17, 2022