Granted Patent B2

US 12,328,495 · App. 18/103,216 · Granted Jun 10, 2025

Inventors: Michael Yang Liu (Santa Clara, CA); Jeffrey Ching Wang (San Francisco, CA); Mohamed Mohy-Eldeen Abdelgany (San Jose, CA); John Jordan Nold (Longmont, CO); George Alban Heitz, III (Mountain View, CA); Siddarth Raghunathan (Millbrae, CA); Shayan Sayadi (San Francisco, CA); Scott Mullins (San Francisco, CA)

Assignee: Google LLC

H04N23/611G06F1/1605G06F21/31G06F21/83G06V20/40H04N7/142H04N23/64H04N23/69G06V20/44

Abstract

Claims (88)

1. A method comprising:

while a display assistant device is operating in a monitoring mode:

performing a first operation associated with a first device function, the first operation comprising continuously capturing, for a period of time, first video using a camera of the display assistant device to facilitate authorization of one or more users;

while performing the first operation, receiving a user request to switch from the monitoring mode to a second mode; and

in response to receiving the user request:

capturing one or more images of a field of view using the camera, where, consequent to the capturing of the one or more images, the one or more images are compared with one or more stored images of authorized users; and

responsive to a determination that there is a match between the one or more images and at least one image of the one or more stored images of the authorized users, switching from the monitoring mode to the second mode, wherein the display assistant device in the second mode is configured to allow user access while excluding the first operation of the monitoring mode.

2. The method as recited in claim 1 , further comprising:

pausing the first operation; and

responsive to determining that the user request is completed, resuming the first operation.

3. The method as recited in claim 1 , wherein the user request is a first user request, the method further comprising:

receiving a second user request; and

responsive to determining that the second user request is associated with the first device function, denying the second user request.

4. The method as recited in claim 1 , wherein:

the user access is to a plurality of functions;

the display assistant device in the second mode is configured to allow user access to the plurality of functions while excluding the first operation a monitoring function of the monitoring mode; and

the plurality of functions comprises a video communication function, a face recognition function, and/or a content curation function.

5. The method as recited in claim 1 , wherein the user request is a first user request, and the method further comprises:

receiving a second user request associated with a second device function; and

in response to receiving the second user request:

pausing the first operation; and

performing a second operation associated with the second device function, the second operation comprising:

continuously capturing second video using the camera; and

transmitting the second video to one or more second users participating in a video communication with a first user of the display assistant device.

6. The method as recited in claim 5 , wherein the camera has a field of view, and the method further comprises:

prior to the performing the first operation, changing the field of view from a first setting to a second setting.

7. The method as recited in claim 6 , further comprising:

after pausing the first operation and prior to performing the second operation, changing the field of view from the second setting to the first setting.

8. The method as recited in claim 1 , wherein the display assistant device comprises a physical button, and the user request comprises a user selection of the physical button.

9. The method as recited in claim 1 , further comprising:

responsive to determining that there is not a match between the one or more images and the images of the authorized users, continuing operating in the monitoring mode.

10. The method as recited in claim 1 , wherein the stored images of the authorized users are stored with hierarchy information of the authorized users, and the switching from the monitoring mode to the second mode is in accordance with a determination that a user matches a threshold hierarchy.

11. The method as recited in claim 2 , wherein the display assistant device comprises a physical button, and the method further comprises:

receiving a user selection of the physical button; and

in response to the user selection, disabling the camera and one or more microphones of the display assistant device.

12. The method as recited in claim 1 , wherein the display assistant device comprises one or more sensors that are configured to detect one or more events, the one or more events associated with at least one of smoke, a motion, a sound, or a person; and

wherein the method further comprises:

responsive to a detection of a first alert event occurring in front of the camera:

automatically enabling the camera and one or more microphones of the display assistant device; and

recording the first alert event.

13. The method as recited in claim 1 , wherein the performing the first operation comprises:

capturing, simultaneously with the first video, audio information using one or more microphones of the display assistant device.

14. A display assistant device comprising:

a display;

a camera;

one or more processing devices; and

memory that stores instructions that, when executed by the one or more processing devices, cause the one or more processing devices to perform operations comprising:

while the display assistant device is operating in a monitoring mode:

performing a first operation associated with a first device function, the first operation comprising continuously capturing, for a period of time, first video using a camera of the display assistant device to facilitate authorization of one or more users;

while performing the first operation, receiving a user request to switch from the monitoring mode to a second mode; and

in response to receiving the user request:

capturing one or more images of a field of view using the camera, where, consequent to the capturing of the one or more images, the one or more images are compared with one or more stored images of authorized users; and

responsive to a determination that there is a match between the one or more images and at least one image of the one or more stored images of the authorized users, switching from the monitoring mode to the second mode, wherein the display assistant device in the second mode is configured to allow user access while excluding the first operation of the monitoring mode.

15. The display assistant device as recited in claim 14 , wherein the operations further comprise:

responsive to determining that there is not a match between the one or more images and the images of the authorized users, continuing operating in the monitoring mode.

16. The display assistant device as recited in claim 14 , further comprising:

one or more sensors configured to detect one or more events, the one or more events associated with at least one of:

smoke, a motion, a sound, or a person; and

one or more microphones;

wherein the operations further comprise:

responsive to a detection of a first alert event occurring in front of the camera:

automatically enabling the camera and the one or more microphones; and

recording the first alert event using the camera and the one or more microphones.

17. The display assistant device as recited in claim 14 , wherein the user request is a first user request, and the operations further comprise:

while performing the first operation, receiving a second user request associated with a second device function; and

in response to the receiving the second user request:

pausing the first operation; and

performing a second operation associated with the second device function, the second operation comprising:

continuously capturing second video using the camera; and

transmitting the second video to one or more second users participating in a video communication with a first user of the display assistant device.

18. A non-transitory, machine-readable storage medium storing instructions that, when executed by one or more processing devices of a display assistant device, cause the one or more processing devices to perform operations comprising:

while the display assistant device is operating in a monitoring mode:

performing a first operation associated with a first device function, the first operation comprising continuously capturing, for a period of time, first video using a camera of the display assistant device to facilitate authorization of one or more users;

while performing the first operation, receiving a user request to switch from the monitoring mode to a second mode; and

in response to receiving the user request:

capturing one or more images of a field of view using the camera, where, consequent to the capturing of the one or more images, the one or more images are compared with one or more stored images of authorized users; and

responsive to a determination that there is a match between the one or more images and at least one image of the one or more stored images of the authorized users, switching from the monitoring mode to the second mode, wherein the display assistant device in the second mode is configured to allow user access while excluding the first operation of the monitoring mode.

19. The non-transitory, machine-readable storage medium as recited in claim 18 , wherein the user request is a first user request, and wherein the operations further comprise:

receiving a second user request associated with a second device function; and

in response to the receiving the second user request:

pausing the first operation; and

performing a second operation associated with the second device function, the second operation comprising:

continuously capturing second video using the camera; and

transmitting the second video to one or more second users participating in a video communication with a first user of the display assistant device.

20. The non-transitory, machine-readable storage medium as recited in claim 18 , wherein the operations further comprising:

responsive to a detection of a first alert event occurring in front of the camera:

automatically enabling the camera and one or more microphones of the display assistant device; and

recording the first alert event using the camera and the one or more microphones.

Continuity (7)

References Cited (20)