IP Library Granted Patent US 12,322,136
Granted Patent B2
US 12,322,136 · App. 17/483,585 · Granted Jun 3, 2025

Methods, systems, apparatus, and articles of manufacture for camera image stream selection for electronic user devices

Inventor: Tigi Thomas (Bangalore, IN)
Assignee: Intel Corporation
G06T7/75G06V40/166G06T2207/10016G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,322,136
App. No.
17/483,585
Filed
Sep 23, 2021
Granted
Jun 3, 2025
Kind
B2
Art Unit
2668
USPC
382/103
Abstract

Methods, apparatus, systems, and articles of manufacture for camera image stream selection for electronic user devices are disclosed. An example apparatus disclosed herein includes head pose estimation circuitry to determine, based on first image data and second image data, (a) a first orientation of a user relative to a first camera and (b) a second orientation of the user relative to a second camera, the first image data associated with a first video stream output by the first camera and the second video data associated with a second video stream output by the second camera, and video stream selection circuitry to select one of the first video stream or the second video stream based on the first orientation and the second orientation.

Claims (71)

1. A non-transitory computer readable medium comprising machine-readable instructions to cause at least one processor circuit to at least:

determine, based on a first video frame and a second video frame, (a) a first orientation of at least a portion of a face of a user relative to a first camera at a first time and (b) a second orientation of the at least the portion of the face of the user relative to a second camera at the first time, the first video frame output by the first camera and the second video frame output by the second camera;

detect, based on the first orientation and the second orientation, that the user is oriented toward the first camera for a first duration and is not oriented toward the second camera for the first duration;

in response to determining that the user is oriented toward the first camera for the first duration, increment a first attention score associated with the first camera to generate an adjusted first attention score;

in response to determining that the user is not oriented toward the second camera for the first duration, reduce a second attention score associated with the second camera to generate an adjusted second attention score;

select one of a first video stream generated by the first camera or a second video stream generated by the second camera based on the adjusted first attention score and the adjusted second attention score, the first video stream including the first video frame and the second video stream including the second video frame; and

cause the selected one of the first video stream or the second video stream to be output for presentation.

2. The non-transitory computer readable medium of claim 1 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to:

determine the first orientation based on a first yaw angle of the at least the portion of the face of the user relative to the first camera; and

determine the second orientation based on a second yaw angle of the at least the portion of the face of the user relative to the second camera.

3. The non-transitory computer readable medium of claim 1 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to:

determine the first attention score for the first camera based on a third yaw angle of the at least the portion of the face of the user relative to the first camera at a second time, the second time before the first time; and

determine the second attention score for the second camera based on a fourth yaw angle of the at least the portion of the face of the user relative to the second camera at the second time.

4. The non-transitory computer readable medium of claim 3 , wherein the instructions, when executed, cause the at least one processor circuitry circuit to:

determine the first attention score based on a comparison of the first third yaw angle to a threshold yaw angle; and

determine the second attention score based on a comparison of the fourth yaw angle to the threshold yaw angle.

5. The non-transitory computer readable medium of claim 2 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to further determine the adjusted first attention score based on a duration for which the first yaw angle satisfies a threshold yaw angle.

6. The non-transitory computer readable medium of claim 2 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to:

perform a comparison of the adjusted first attention score to an attention detection threshold;

perform a comparison of the adjusted second attention score to the attention detection threshold; and

in response to determining that each of the adjusted first attention score and the adjusted second attention score satisfies the attention detection threshold, select the one of the first video stream or the second video stream corresponding to a lesser one of the first yaw angle or the second yaw angle.

7. The non-transitory computer readable medium of claim 6 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to select the first video stream in response to determining that the adjusted first attention score and the adjusted second attention score do not satisfy the attention detection threshold.

8. The non-transitory computer readable medium of claim 1 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to detect, using a neural network, the at least the portion of the face of the user in the first video frame and the second video frame.

9. The non-transitory computer readable medium of claim 8 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to determine (a) the first orientation based on a facial feature in the at least the portion of the face in the first video frame and (b) the second orientation based on the facial feature in the at least the portion of the face in the second video frame.

10. The non-transitory computer readable medium of claim 1 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to:

increment the first attention score based on the first duration and an attack factor; and

reduce the second attention score based on the first duration and a decay factor, the decay factor less than the attack factor.

11. An apparatus comprising:

memory;

machine-readable instructions stored in the apparatus; and

at least one processor circuit to execute the machine-readable instructions to at least:

determine, based on first image data and second image data, (a) a first yaw angle of at least a portion of a face of a user relative to a first camera at a first time and (b) a second yaw angle of the at least the portion of the face of the user relative to a second camera at the first time, the first image data associated with a first video stream output by the first camera and the second image data associated with a second video stream output by the second camera;

detect, based on the first yaw angle and the second yaw angle, that the user is oriented toward the first camera for a first duration and is not oriented toward the second camera for the first duration;

in response to determining that the user is oriented toward the first camera for the first duration, increment a first attention score associated with the first camera to generate an adjusted first attention score;

in response to determining that the user is not oriented toward the second camera for the first duration, reduce a second attention score associated with the second camera to generate an adjusted second attention score; and

select one of the first video stream or the second video stream based on the adjusted first attention score and the adjusted second attention score as a video stream output.

12. The apparatus of claim 11 , wherein one or more of the at least one processor circuit is to execute the machine-readable instructions to:

determine the first attention score based on (a) a comparison of a third yaw angle of the at least the portion of the face of the user relative to the first camera at a second time to a threshold yaw angle and (b) a second duration for which the first third yaw angle satisfies the threshold yaw angle; and

determine the second attention score based on (c) a comparison of a fourth yaw angle of the at least the portion of the face of the user relative to the second camera at the second time to the threshold yaw angle and (d) a third duration for which the fourth yaw angle satisfies the threshold yaw angle.

13. The apparatus of claim 11 , wherein one or more of the at least one processor circuit is to execute the machine-readable instructions to:

perform a comparison of the adjusted first attention score to an attention detection threshold;

perform a comparison of the adjusted second attention score to the attention detection threshold; and

in response to determining that each of the adjusted first attention score and the adjusted second attention score satisfies the attention detection threshold, select the one of the first video stream or the second video stream corresponding to a lesser one of the first yaw angle or the second yaw angle.

14. The apparatus of claim 13 , wherein one or more of the at least one processor circuit is to execute the machine-readable instructions to select the first camera in response to determining that the adjusted first attention score and the adjusted second attention score do not satisfy the attention detection threshold.

15. The apparatus of claim 11 , wherein one or more of the at least one processor circuit is to execute the machine-readable instructions to detect, using a neural network, the at least the portion of the face of the user in the first image data and the second image data.

16. The apparatus of claim 15 , wherein one or more of the at least one processor circuit is to execute the machine-readable instructions to determine (a) the first yaw angle based on a facial feature in the at least the portion of the face in the first image data and (b) the second yaw angle based on the facial feature in the at least the portion of the face in the second image data.

17. The apparatus of claim 11 , wherein one or more of the at least one processor circuit is to execute the machine-readable instructions to:

increment the first attention score based on the first duration and an attack factor; and

reduce the second attention score based on the first duration and a decay factor, the decay factor less than the attack factor.

18. A method comprising:

determining, based on first image data and second image data, (a) a first orientation of at least a portion of a face of a user relative to a first camera at a first time and (b) a second orientation of the at least the portion of the face of the user relative to a second camera at the first time, the first image data associated with a first video stream output by the first camera and the second image data associated with a second video stream output by the second camera;

determining, based on the first orientation and the second orientation and by at least one processor circuit programmed by at least one instruction, that the user is oriented toward the first camera for a first duration and is not oriented toward the second camera for the first duration;

in response to determining that the user is oriented toward the first camera for the first duration, incrementing a first attention score associated with the first camera to generate an adjusted first attention score;

in response to determining that the user is not oriented toward the second camera for the first duration, reducing a second attention score associated with the second camera to generate an adjusted second attention score; and

selecting one of the first video stream or the second video stream for output based on the adjusted first attention score and the adjusted second attention score.

19. The method of claim 18 , further including:

determining the first orientation based on a first yaw angle of the at least the portion of the face of the user relative to the first camera; and

determining the second orientation based on a second yaw angle of the at least the portion of the face of the user relative to the second camera.

20. The method of claim 18 , further including:

determining the first attention score based on a third yaw angle of the at least the portion of the face of the user relative to the first camera at a second time, the second time before the first time; and

determining the second attention score based on a fourth yaw angle of the at least the portion of the face of the user relative to the second camera at the second time.

21. The method of claim 20 , further including:

determining the first attention score based on (a) a comparison of the third yaw angle to a threshold yaw angle and (b) a second duration for which the third yaw angle satisfies the threshold yaw angle; and

determining the second attention score based on (c) a comparison of the fourth yaw angle to the threshold yaw angle and (d) a third duration for which the fourth yaw angle satisfies the threshold yaw angle.

22. The method of claim 19 , further including:

performing a comparison of the adjusted first attention score to an attention detection threshold;

performing a comparison of the adjusted second attention score to the attention detection threshold; and

in response to determining that each of the adjusted first attention score and the adjusted second attention score satisfies the attention detection threshold, selecting the one of the first video stream or the second video stream corresponding to a lesser one of the first yaw angle or the second yaw angle.

23. The method of claim 22 , further including selecting the first video stream in response to determining that the adjusted first attention score and the adjusted second attention score do not satisfy the attention detection threshold.

24. The method of claim 18 , further including detecting, using a neural network, the at least the portion of the face of the user in the first image data and the second image data.

25. The method of claim 24 , further including determining (a) the first orientation based on a facial feature in the at least the portion of the face in the first image data and (b) the second orientation based on the facial feature in the at least the portion of the face in the second image data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 14, 2021
From: THOMAS, TIGI
To: INTEL CORPORATION
Reel/Frame 057800/0481 →
Continuity (1)
Related Publication 20220012913A1 · Jan 13, 2022
References Cited (8)
US 9491374B1 · Avrahami · 2016 [cited by examiner]
US 11024002B2 · Isikdogan et al. · 2021 [cited by applicant]
US 20160286120A1 · Kuo et al. · 2016 [cited by applicant]
US 20200401793A1 · Leung et al. · 2020 [cited by applicant]
US 20210112238A1 · Bylicka et al. · 2021 [cited by applicant]
US 20220400216A1 · Wang · 2022 [cited by examiner]
US 20230298204A1 · Wang et al. · 2023 [cited by applicant]
Avrahami, Daniel, Eveline van Everdingen, and Jennifer Marlow. “Supporting Multitasking in Video Conferencing using Gaze Tracking and On-Screen Activity Detection.” Proceedings of the 21st International Conference on In… [cited by examiner]