IP Library › Granted Patent US 12664983
Granted Patent B2
US 12664983 · App. 18/105,158 · Granted Jun 23, 2026

Digital assistant for providing real-time social intelligence

Inventors: Eddy Zexin Liang (Seattle, WA); William Caruso (Saratoga, CA); Madhu Chinthakunta (Saratoga, CA)
Assignee: Apple Inc.
G10L15/22G06F3/013G06F3/017G10L15/1815G10L25/78G10L2015/226
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12664983
App. No.
18/105,158
Granted
Jun 23, 2026
Kind
B2
Abstract

This relates generally to intelligent automated assistants and, more specifically, to provide real-time social intelligence. An example method includes while providing one or more outputs to a user of the electronic device, determining whether the user is in an engaged state based on: detecting a human in a near-field scene of the user, determining that the user is gazing at the human, and detecting a speech input from at least one of the user or the human while the user is gazing at the human. In response to determining that the user is in the engaged state, foregoing providing the one or more outputs to the user while the user is in the engaged state.

Claims (88)

1 . A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

while providing one or more outputs to a user of the electronic device:

determine whether the user is in an engaged state based on:

detecting a human in a near-field scene of the user;

determining that the user is gazing at the human;

detecting a speech input from at least one of the user or the human while the user is gazing at the human; and

determining context of the speech input, wherein determining the context of the speech input includes determining user intent based on the speech input;

in response to determining that the user is in the engaged state:

forego providing the one or more outputs to the user while the user is in the engaged state.

2 . The non-transitory computer-readable storage medium of claim 1 , the one or more programs further comprising instructions, which when executed by the one or more processors, cause the electronic device to:

in response to determining that the user is not in the engaged state:

continue to provide the one or more outputs to the user.

3 . The non-transitory computer-readable storage medium of claim 1 , wherein each of the one or more outputs is an alert of an event associated with the electronic device.

4 . The non-transitory computer-readable storage medium of claim 1 , wherein whether the user is gazing at the human is determined based on a gaze direction of the user's eyes.

5 . The non-transitory computer-readable storage medium of claim 1 , wherein detecting the human in the near-field scene includes:

determining whether the human is in a visual proximity to the user.

6 . The non-transitory computer-readable storage medium of claim 1 , wherein determining whether a user is in an engaged state based on the user's interaction with one or more applications on the electronic device.

7 . The non-transitory computer-readable storage medium of claim 1 , wherein determining whether a user is in an engaged state based on a gesture of the user, wherein the gesture is a nod or a head shake of the user.

8 . The non-transitory computer-readable storage medium of claim 1 , wherein determining whether a user is in an engaged state based on one or more events on the user's electronic calendar.

9 . The non-transitory computer-readable storage medium of claim 1 , wherein determining whether a user is in an engaged state based on an engagement score periodically calculated on the electronic device.

10 . The non-transitory computer-readable storage medium claim 1 , the one or more programs further comprising instructions, which when executed by the one or more processors, cause the electronic device to:

in response to determining that the user is in the engaged state:

determine whether one or more notifications includes time-sensitive or important notification;

in response to determining that the one or more notifications includes time-sensitive or important notification:

provide the time-sensitive or important notification to the user, wherein the speech input is a natural language input from the user in form of a spoken command presented at the electronic device.

11 . The non-transitory computer-readable storage medium of claim 1 , wherein providing the one or more outputs to the user includes displaying the one or more outputs on a portion of a user interface of the electronic device.

12 . The non-transitory computer-readable storage medium of claim 1 , wherein providing the one or more outputs to the user includes displaying the one or more outputs on a portion of a user interface of the electronic device while announcing information about the one or more outputs to the user.

13 . The non-transitory computer-readable storage medium of claim 1 , wherein providing the one or more outputs to the user includes announcing information about one or more outputs to the user.

14 . An electronic device comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

while providing one or more outputs to a user of the electronic device:

determining whether the user is in an engaged state based on:

detecting a human in a near-field scene of the user;

determining that the user is gazing at the human;

detecting a speech input from at least one of the user or the human while the user is gazing at the human; and

determining context of the speech input, wherein determining the context of the speech input includes determining user intent based on the speech input;

in response to determining that the user is in the engaged state:

foregoing providing the one or more outputs to the user while the user is in the engaged state.

15 . The electronic device of claim 14 , the one or more programs further including instructions for:

in response to determining that the user is not in the engaged state:

continuing to provide the one or more outputs to the user.

16 . The electronic device of claim 14 , wherein each of the one or more outputs is an alert of an event associated with the electronic device.

17 . The electronic device of claim 14 , wherein whether the user is gazing at the human is determined based on a gaze direction of the user's eyes.

18 . The electronic device of claim 14 , wherein detecting the human in the near-field scene includes:

determining whether the human is in a visual proximity to the user.

19 . The electronic device of claim 14 , wherein determining whether a user is in an engaged state based on the user's interaction with one or more applications on the electronic device.

20 . The electronic device of claim 14 , wherein determining whether a user is in an engaged state based on a gesture of the user, wherein the gesture is a nod or a head shake of the user.

21 . The electronic device of claim 14 , wherein determining whether a user is in an engaged state based on one or more events on the user's electronic calendar.

22 . The electronic device of claim 14 , wherein determining whether a user is in an engaged state based on an engagement score periodically calculated on the electronic device.

23 . The electronic device of claim 14 , the one or more programs further including instructions for:

in response to determining that the user is in the engaged state:

determining whether one or more notifications includes time-sensitive or important notification;

in response to determining that the one or more notifications includes time-sensitive or important notification:

providing the time-sensitive or important notification to the user, wherein the speech input is a natural language input from the user in form of a spoken command presented at the electronic device.

24 . The electronic device of claim 14 , wherein providing the one or more outputs to the user includes displaying the one or more outputs on a portion of a user interface of the electronic device.

25 . The electronic device of claim 14 , wherein providing the one or more outputs to the user includes displaying the one or more outputs on a portion of a user interface of the electronic device while announcing information about the one or more outputs to the user.

26 . The electronic device of claim 14 , wherein providing the one or more outputs to the user includes announcing information about one or more outputs to the user.

27 . A method, comprising:

at an electronic device with one or more processors and memory:

while providing one or more outputs to a user of the electronic device:

determining whether the user is in an engaged state based on:

detecting a human in a near-field scene of the user;

determining that the user is gazing at the human;

detecting a speech input from at least one of the user or the human while the user is gazing at the human; and

determining context of the speech input, wherein determining the context of the speech input includes determining user intent based on the speech input;

in response to determining that the user is in the engaged state:

foregoing providing the one or more outputs to the user while the user is in the engaged state.

28 . The method of claim 27 , further comprising:

in response to determining that the user is not in the engaged state:

continuing to provide the one or more outputs to the user.

29 . The method of claim 27 , wherein each of the one or more outputs is an alert of an event associated with the electronic device.

30 . The method of claim 27 , wherein whether the user is gazing at the human is determined based on a gaze direction of the user's eyes.

31 . The method of claim 27 , wherein detecting the human in the near-field scene includes:

determining whether the human is in a visual proximity to the user.

32 . The method of claim 27 , wherein determining whether a user is in an engaged state based on the user's interaction with one or more applications on the electronic device.

33 . The method of claim 27 , wherein determining whether a user is in an engaged state based on a gesture of the user, wherein the gesture is a nod or a head shake of the user.

34 . The method of claim 27 , wherein determining whether a user is in an engaged state based on one or more events on the user's electronic calendar.

35 . The method of claim 27 , wherein determining whether a user is in an engaged state based on an engagement score periodically calculated on the electronic device.

36 . The method of claim 27 , further comprising:

in response to determining that the user is in the engaged state:

determining whether one or more notifications includes time-sensitive or important notification;

in response to determining that the one or more notifications includes time-sensitive or important notification:

providing the time-sensitive or important notification to the user, wherein the speech input is a natural language input from the user in form of a spoken command presented at the electronic device.

37 . The method of claim 27 , wherein providing the one or more outputs to the user includes displaying the one or more outputs on a portion of a user interface of the electronic device.

38 . The method of claim 27 , wherein providing the one or more outputs to the user includes displaying the one or more outputs on a portion of a user interface of the electronic device while announcing information about the one or more outputs to the user.

39 . The method of claim 27 , wherein providing the one or more outputs to the user includes announcing information about one or more outputs to the user.