IP Library › Granted Patent US 12,019,796
Granted Patent B2
US 12,019,796 · App. 17/816,228 · Granted Jun 25, 2024

User attention determination for extended reality

Inventors: Danlu Zhang (San Diego, CA); Paul Holdaway (Carlsbad, CA); Jiaying Pan (San Diego, CA); Ajinkya Shyam Ambre (San Diego, CA); Vinayak Muralidharan (San Diego, CA); Martin Renschler (San Diego, CA); Michael Mager (San Diego, CA); Brian Momeyer (Escondido, CA); Rashmi Kulkarni (Redwood City, CA); Damir Didjusto (San Diego, CA)
Assignee: QUALCOMM Incorporated
G06F3/013G06F3/015G06T19/006G06V10/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,019,796
App. No.
17/816,228
Filed
Jul 29, 2022
Granted
Jun 25, 2024
Kind
B2
Art Unit
2612
USPC
345/633
Abstract

Attention evaluation by an extended reality system, the system determining one or more regions of interest (ROI) for an image displayed to a user. The system may also receive eye tracking information indicating an area of the image that the user is looking at. The system may further generate focus statistics based on the area of the image at which the user is looking at and the one or more ROI; and output the generated focus statistics.

Claims (56)

1. A method for attention evaluation, comprising:

determining a region of interest (ROI) for an image displayed to a user;

receiving, via an image sensor, eye tracking information indicating an area of the image at which the user is gazing;

generating, via a processor, focus statistics based on the area of the image at which the user is gazing and the ROI, wherein the focus statistics indicate a first amount of time when the area of the image at which the user is gazing overlaps with the ROI, and a second amount of time when the area of the image at which the user is gazing does not overlap with the ROI;

determining an attention level based on the first amount of time and the second amount of time; and

outputting the attention level.

2. The method of claim 1 , wherein the focus statistics are based on a comparison of the area of the image and the ROI, the method further comprising:

determining, based on the focus statistics, potential fatigue of the user; and

outputting an indication of the potential fatigue.

3. The method of claim 2 , wherein outputting the indication of the potential fatigue includes displaying a fatigue warning to the user.

4. The method of claim 1 , wherein the focus statistics are based on a comparison of the area of the image and the ROI, wherein the focus statistics indicate that eyes of the user are gazing at different areas of the image, the method further comprising:

determining, based on the focus statistics, that the user has a potential visual condition; and

outputting an indication of the potential visual condition.

5. The method of claim 1 , wherein the attention level is determined to be a low attention level when the second amount of time exceeds the first amount of time by a threshold amount.

6. The method of claim 1 , further comprising adjusting content in an additional image based on the generated focus statistics.

7. The method of claim 1 , wherein the ROI is determined based on a received indication of a location of the ROI in the image.

8. The method of claim 1 , wherein the ROI is determined based on an analysis of the image.

9. The method of claim 1 , wherein the ROI is determined based on an object detected in the image.

10. An apparatus for attention evaluation, comprising:

at least one memory comprising instructions; and

at least one processor coupled to the at least one memory and configured to:

determine a region of interest (ROI) for an image displayed to a user;

receive eye tracking information indicating an area of the image at which the user is gazing;

generate focus statistics based on the area of the image at which the user is gazing and the ROI, wherein the focus statistics indicate a first amount of time when the area of the image at which the user is gazing overlaps with the ROI, and a second amount of time when the area of the image at which the user is gazing does not overlap with the ROI;

determine an attention level based on the first amount of time and the second amount of time; and

output the attention level.

11. The apparatus of claim 10 , wherein the focus statistics are based on a comparison of the area of the image and the ROI, and wherein the at least one processor is further configured to:

determine, based on the focus statistics, potential fatigue of the user; and

output an indication of the potential fatigue.

12. The apparatus of claim 11 , wherein the at least one processor is further configured to output the indication of the potential fatigue by displaying a fatigue warning to the user.

13. The apparatus of claim 10 , wherein the focus statistics are based on a comparison of the area of the image and the ROI, wherein the focus statistics indicate that eyes of the user are gazing at different areas of the image, and wherein the at least one processor is further configured to:

determine, based on the focus statistics, that the user has a potential visual condition; and

output an indication of the potential visual condition.

14. The apparatus of claim 10 , wherein the at least one processor is configured to determine the attention level to be a low attention level when the second amount of time exceeds the first amount of time by a threshold amount.

15. The apparatus of claim 10 , wherein the at least one processor is further configured to adjust content in an additional image based on the generated focus statistics.

16. The apparatus of claim 10 , wherein the at least one processor is configured to determine the ROI based on a received indication of a location of the ROI in the image.

17. The apparatus of claim 10 , wherein the at least one processor is configured to determine the ROI based on an analysis of the image.

18. The apparatus of claim 10 , wherein the at least one processor is configured to determine the ROI based on an object detected in the image.

19. A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to:

determine a region of interest (ROI) for an image displayed to a user;

receive eye tracking information indicating an area of the image at which the user is gazing;

generate focus statistics based on the area of the image at which the user is gazing and the ROI, wherein the focus statistics indicate a first amount of time when the area of the image at which the user is gazing overlaps with the ROI, and a second amount of time when the area of the image at which the user is gazing does not overlap with the ROI;

determine an attention level based on the first amount of time and the second amount of time; and

output the attention level.

20. The non-transitory computer-readable medium of claim 19 , wherein the focus statistics are based on a comparison of the area of the image and the ROI, and wherein the instructions further cause the at least one processor to:

determine, based on the focus statistics, potential fatigue of the user; and

output an indication of the potential fatigue.

21. The non-transitory computer-readable medium of claim 20 , wherein the instructions further cause the at least one processor to output the indication of the potential fatigue by displaying a fatigue warning to the user.

22. The non-transitory computer-readable medium of claim 19 , wherein the focus statistics are based on a comparison of the area of the image and the ROI, wherein the focus statistics indicate that eyes of the user are gazing at different areas of the image, and wherein the instructions further cause the at least one processor to:

determine, based on the focus statistics, that the user has a potential visual condition; and

output an indication of the potential visual condition.

23. The non-transitory computer-readable medium of claim 19 , wherein the instructions further cause the at least one processor to determine the attention level to be a low attention level when the second amount of time exceeds the first amount of time by a threshold amount.

24. The non-transitory computer-readable medium of claim 19 , wherein the instructions further cause the at least one processor to adjust content in an additional image based on the generated focus statistics.

25. The non-transitory computer-readable medium of claim 19 , wherein the instructions further cause the at least one processor to determine the ROI based on a received indication of a location of the ROI in the image.

26. The non-transitory computer-readable medium of claim 19 , wherein the instructions further cause the at least one processor to determine the ROI based on an analysis of the image.

27. The non-transitory computer-readable medium of claim 19 , wherein the instructions further cause the at least one processor to determine the ROI based on an object detected in the image.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE NAME OF THE 4TH INVENTOR PREVIOUSLY RECORDED AT REEL: 67142 FRAME: 758. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 20, 2024
From: ZHANG, DANLU; HOLDAWAY, PAUL; PAN, JIAYING; AMBRE, AJINKYA SHYAM; MURALIDHARAN, VINAYAK; RENSCHLER, MARTIN; MAGER, MICHAEL; MOMEYER, BRIAN; KULKARNI, RASHMI; DIDJUSTO, DAMIR
To: QUALCOMM INCORPORATED
Reel/Frame 068433/0922 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THE FORTH INVENTOR'S NAME PREVIOUSLY RECORDED AT REEL: 61344 FRAME: 341. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Apr 17, 2024
From: ZHANG, DANLU; HOLDAWAY, PAUL; PAN, JIAYING; AMBRE, AJINKYA; MURALIDHARAN, VINAYAK; RENSCHLER, MARTIN; MAGER, MICHAEL; MOMEYER, BRIAN; KULKARNI, RASHMI; DIDJUSTO, DAMIR
To: QUALCOMM INCORPORATED
Reel/Frame 067142/0758 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 7, 2022
From: ZHANG, DANLU; HOLDAWAY, PAUL; PAN, JIAYING; AMBRE, AJINKYA SHYAM; MURALIDHARAN, VINAYAK; RENSCHLER, MARTIN; MAGER, MICHAEL; MOMEYER, BRIAN; KULKARNI, RASHMI; DIDJUSTO, DAMIR
To: QUALCOMM INCORPORATED
Reel/Frame 061344/0341 →
Continuity (1)
Related Publication 20240036640A1 · Feb 1, 2024
Cited By (2)
US 12,511,768 US 12,651,480