IP Library Granted Patent US 11,343,412
Granted Patent B2
US 11,343,412 · App. 16/660,600 · Granted May 24, 2022

User detection and user attention detection using multi-zone depth sensing

Inventors: Divyashree-Shivakumar Sreepathihalli (Santa Clara, CA); Michael Daniel Rosenzweig (San Ramon, CA); Uttam K. Sengupta (Portland, OR); Soethiha Soe (Beaverton, OR); Prasanna Krishnaswamy (Bangalore, IN)
Assignee: Intel Corporation
H04N5/2226G06T7/136G06T7/521H04N5/232411
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,343,412
App. No.
16/660,600
Granted
May 24, 2022
Kind
B2
Abstract

An electronic device receives depth sensor data that includes depths sensed in multiple zones in the field of view of a depth sensor. The device determines whether a user is in front of the device based on the depth sensor data. If the user is determined to be present, then the device causes a display to enter an operational mode. Otherwise, the device causes the display to enter a standby mode. The device may also determine whether the user's attention is on the device by determining whether the depth sensor data indicates that the user is facing the device. If so, the device causes the display to enter the operational mode. Otherwise, the device causes the display to enter a power saving mode. The device may use a machine learning algorithm to determine whether the depth sensor data indicates that the user is present and/or facing the device.

Claims (52)

1. An electronic device, comprising:

one or more human interface devices;

an electronic display configured to enter a plurality of power usage modes;

one or more depth sensors configured to detect a plurality of depths of a plurality of zones of a field of view of the one or more depth sensors; and

a controller comprising at least one processor and a memory, wherein the memory comprises machine-readable instructions configured to cause the at least one processor to:

receive the plurality of depths from the one or more depth sensors;

generate an image based on the plurality of depths of the plurality of zones;

apply a machine-learning model to determine that a user is present in the field of view of the one or more depth sensors based on the image;

cause the electronic display to enter a power usage mode of the plurality of power usage modes based on the machine-learning model determining that the user is present in the field of view of the one or more depth sensors;

determine that the one or more human interface devices have been inactive for a threshold duration of time; and

in response to determining that the one or more human interface devices of the electronic device have not been inactive for the threshold duration of time, train the machine-learning model to determine whether the user is present in the field of view of the one or more depth sensors based on the plurality of depths.

2. The electronic device of claim 1 , wherein the one or more depth sensors comprise a time-of-flight sensor, an infrared sensor, an ultra-wideband sensor, a radar sensor, a WiFi sensor, a sonar sensor, or any combination thereof.

3. The electronic device of claim 1 , wherein the machine-readable instructions are configured to cause the at least one processor to generate the image by:

assigning grayscale values to the plurality of depths; and

aggregating the grayscale values.

4. The electronic device of claim 1 , wherein the one or more depth sensors configured to detect the plurality of depths by receiving a plurality of return signals in response to sending a plurality of outgoing signals, and wherein the machine-readable instructions are configured to cause the at least one processor to generate the image by:

assigning grayscale values to a plurality of signal strengths of the plurality of return signals; and

aggregating the grayscale values.

5. The electronic device of claim 1 , wherein the machine-readable instructions are configured to cause the at least one processor to generate the image by performing a thresholding technique.

6. The electronic device of claim 1 , wherein the machine-readable instructions are configured to cause the at least one processor to process the image by performing an image scaling technique, an interpolation technique, a sharpening technique, a thresholding technique, or any combination thereof.

7. The electronic device of claim 1 , wherein the machine-readable instructions are configured to cause the at least one processor to cause the electronic display to enter a second power usage mode of the plurality of power usage modes in response to determining that the user is not present in the field of view of the one or more depth sensors, wherein the second power usage mode comprises a standby mode of the electronic display.

8. The electronic device of claim 7 , wherein the standby mode causes the electronic display to turn off.

9. An electronic device, comprising:

one or more human interface devices;

an electronic display configured to enter a plurality of power usage modes;

one or more depth sensors configured to detect a plurality of depths of a plurality of zones of a field of view of the one or more depth sensors; and

a controller comprising at least one processor and a memory, wherein the memory comprises machine-readable instructions configured to cause the at least one processor to:

receive the plurality of depths from the one or more depth sensors;

generate an image based on the plurality of depths of the plurality of zones;

apply a machine-learning model to determine that a user is focused on the electronic device based on the image;

cause the electronic display to enter a power usage mode of the plurality of power usage modes based on the machine-learning model determining that the user is focused on the electronic device;

determine that the one or more human interface devices have been inactive for a threshold duration of time; and

in response to determining that the one or more human interface devices of the electronic device have not been inactive for the threshold duration of time, train the machine-learning model to determine whether the user is focused on the electronic device based on the plurality of depths.

10. The electronic device of claim 9 , wherein the plurality of zones of the field of view of the one or more depth sensors consists of a four by four grid.

11. The electronic device of claim 9 , wherein the plurality of zones of the field of view of the one or more depth sensors consists of an eight by eight grid.

12. The electronic device of claim 9 , wherein the plurality of zones of the field of view of the one or more depth sensors consists of a sixteen by sixteen grid.

13. The electronic device of claim 9 , wherein the machine-readable instructions are configured to cause the at least one processor to cause the electronic display to enter a second power usage mode of the plurality of power usage modes in response to determining that the user is not focused on the electronic device, wherein the second power usage mode comprises a power saving mode of the electronic display.

14. The electronic device of claim 13 , wherein the power saving mode causes the electronic display to dim.

15. One or more tangible, non-transitory, machine-readable media, comprising machine-readable instructions that cause at least one processor to:

receive a plurality of depth measurements from one or more depth sensors of an electronic device, wherein the one or more depth sensors comprise a radio wave-based depth sensor, a sound wave-based depth sensor, or both;

generate an image based on the plurality of depth measurements;

apply a first machine-learning model to the image, wherein the first machine-learning model is configured to determine that a user is present in a field of view of the one or more depth sensors;

apply a second machine-learning model to the image in response to the first machine-learning model determining that the user is present in the field of view of the one or more depth sensors, wherein the second machine-learning model is configured to determine that the user is focused on the electronic device based on the image;

cause an electronic display of the electronic device to enter a power usage mode in response to the second machine-learning model determining that the user is focused on the electronic device based on the image;

determine that one or more human interface devices of the electronic device have been inactive for a threshold duration of time; and

in response to determining that the one or more human interface devices of the electronic device have not been inactive for the threshold duration of time, train the second machine-learning model using the plurality of depth measurements.

16. The one or more tangible, non-transitory, machine-readable media of claim 15 , wherein the power usage mode comprises an operational mode.

17. The one or more tangible, non-transitory, machine-readable media of claim 15 , wherein the machine-readable instructions cause the at least one processor to cause the electronic display to enter a standby mode in response to the second machine-learning model determining that the user is not focused on the electronic device based on the image, wherein the standby mode causes the electronic display to turn off.

18. The one or more tangible, non-transitory, machine-readable media of claim 15 , wherein the machine-readable instructions cause the at least one processor to cause the electronic display to enter a power saving mode in response to the second machine-learning model determining that the user is not focused on the electronic device based on the image, wherein the power saving mode causes the electronic display to dim.

19. The one or more tangible, non-transitory, machine-readable media of claim 15 , wherein the machine-readable instructions cause the at least one processor to receive the plurality of depth measurements in response to determining that the one or more human interface devices of the electronic device have been inactive for the threshold duration of time.

20. The electronic device of claim 1 , wherein the one or more human interface devices comprises a keyboard, a mouse, a trackpad, a trackball, or any combination thereof.

21. The electronic device of claim 9 , wherein the machine-readable instructions are configured to train the machine-learning model to determine that the user is not focused on the electronic device based on determining that the user is facing away from the electronic device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2026
From: INTEL CORPORATION
To: INTEL PRODUCTS IP LLC
Reel/Frame 076025/0681 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 24, 2019
From: SREEPATHIHALLI, DIVYASHREE-SHIVAKUMAR; ROSENZWEIG, MICHAEL DANIEL; SENGUPTA, UTTAM K.; SOE, SOETHIHA; KRISHNASWAMY, PRASANNA
To: INTEL CORPORATION
Reel/Frame 050819/0893 →
Continuity (1)
Related Publication 20200053257A1 · Feb 13, 2020