Granted Patent B2

US 11,463,270 · App. 17/161,582 · Granted Oct 4, 2022

Inventors: Todd E. Swierk (Austin, TX); Vivek Viswanathan Iyer (Austin, TX)

Assignee: Dell Products, LP

H04L12/1822G06N3/04G06V40/166H04N5/23296

Abstract

Claims (77)

1. An information handling system executing an intelligent face framing management system comprising:

a processor configured to execute code instructions of a multimedia multi-user collaboration application to join a videoconference session with a remotely located computing device;

a display screen, a speaker, and a microphone;

a video camera configured to capture a videoframe of a user's image;

the processor to input the videoframe data, including the user's image, and audiovisual (AV) processing instructions to be applied to videoframes into a trained neural network of the intelligent face framing management system;

the processor to execute code instructions of the trained neural network of the intelligent face framing management system to determine a face center and a face image;

the intelligent face framing management system to output an optimized normalizing face framing adjustments to adjust the size of the face image relative to the captured videoframe to within a size percentage range threshold of the face image in the captured videoframe;

the processor to execute an AV processing instruction module configured to execute a digital zoom process on the face image in the captured videoframe to meet the size percentage range threshold in the videoframe; and

a network interface device configured to transmit a processed, encoded media sample, including the captured videoframe to a remotely located computing device participating in the videoconference session.

2. The information handling system of claim 1 further comprising:

the processor to detect user movement with a proximity sensor, during additional captured videoframes;

the processor to execute the intelligent face framing management system to output an adjusted optimized normalizing face framing adjustment in response to the detected user movement.

3. The information handling system of claim 1 further comprising:

the processor to detect user movement toward or away from the video camera in additional captured videoframes;

the processor to the additional captured videoframes, including the user's image, into the trained neural network of the intelligent face framing management system; and

the processor to execute the AV processing instruction module configured to execute the digital zoom process on the face image in the additional captured videoframes to meet the size percentage range threshold in the videoframe according to a second optimized normalizing face framing adjustment.

4. The information handling system of claim 1 further comprising:

the processor to execute code instructions of the intelligent face framing management system to output an optimized centering face framing adjustment to center the face image location in the captured videoframe; and

the processor to execute an AV processing instruction module configured to crop the videoframes to center the face image location in the captured videoframe according to the optimized normalizing face framing adjustment.

5. The information handling system of claim 4 further comprising:

the processor to detect user movement in additional captured videoframes;

the processor to execute the intelligent face framing management system to determine movement of the user to a first side of the captured videoframe that is off-center beyond a threshold amount of shift from center; and

the processor to execute the AV processing instruction module configured to crop the captured videoframes to re-center the face image location in the additional captured videoframes according to a second optimized centering face framing adjustment.

6. The information handling system of claim 1 further comprising:

the processor to execute code instructions of an intelligent face framing management system user interface to receive a user image normalizing setting from a user to set a desired size of the user's image to determine the size percentage range threshold of the face image in the captured videoframe.

7. The information handling system of claim 1 further comprising:

a proximity sensor to detect persons or objects in front of the video camera;

the processor to execute code instructions of an unusual movement detection (UMD) software module to receive proximity sensor distance data and to determine movement of a user location in front of the video camera relative to a face image location within a series of captured videoframes.

8. The information handling system of claim 6 , wherein the proximity sensor is a time-of-flight (TOF) sensor.

9. A method for intelligently face framing a user's image within captured videoframes in a collaboration videoconference session for an information handling system, comprising:

executing code instructions, via a processor, of a multimedia multi-user collaboration application, via a processor, to join a videoconference session of a remotely located computing device;

capturing a videoframe of a user's image via a video camera;

inputting videoframe data, including the user's image, and audiovisual (AV) processing instructions to be applied to videoframes into a trained neural network of the intelligent face framing management system to determine a face center;

executing code instructions of the intelligent face framing management system to output an optimized centering face framing adjustment to center a face image location in the captured videoframe;

executing an AV processing instruction module configured to crop the videoframes to center the face image location in the captured videoframe according to the optimized centering face framing adjustment; and

transmitting, via a network interface device, a processed, encoded media sample, including the captured videoframe to a remotely located computing device participating in the videoconference session.

10. The method of claim 9 further comprising:

detecting user movement, via a proximity sensor, during additional captured videoframes;

the processor to detect user movement with a proximity sensor, during additional captured videoframes;

executing the intelligent face framing management system to output an adjusted optimized centering face framing adjustment in response to the detected user movement.

11. The method of claim 9 further comprising:

detecting user movement, via a proximity sensor, in additional captured videoframes;

executing the intelligent face framing management system to determine movement of the user to a first side of the captured videoframe that is off-center beyond a threshold amount of shift from center; and

executing the AV processing instruction module configured to crop the captured videoframes to re-center the face center location in the captured videoframe according to a second optimized centering face framing adjustment.

12. The method of claim 9 further comprising:

executing code instructions of the trained neural network of the intelligent face framing management system to determine a face image in the user's image of the captured videoframe and to output optimized face framing adjustments to adjust the size of the face image relative to the captured videoframe to within a size percentage range threshold of the face image in the captured videoframe; and

executing an AV processing instruction module configured to execute a digital zoom process on the face image in the captured videoframe to meet the size percentage range threshold pursuant to an optimized normalizing face framing adjustment.

13. The method of claim 12 further comprising:

detecting user movement, via a proximity sensor, in additional captured videoframes;

inputting the captured videoframe data, including the user's image, into the trained neural network of the intelligent face framing management system to determine an updated face image in the captured videoframe in response to user movement changes; and

executing an AV processing instruction module configured to execute a digital zoom process on the updated face image in the captured videoframe to meet the size percentage range threshold.

14. The method of claim 9 further comprising:

executing code instructions of an intelligent face framing management system user interface to receive feature settings from a user to select face centering or face size normalizing to be activated during the videoconference session.

15. The method of claim 9 further comprising:

detecting, via a proximity sensor, persons or objects in front of the video camera;

inputting the proximity sensor distance data and a plurality of captured videoframes into the trained neural network of the intelligent face framing management system to determine the user location in front of the video camera and motion of the user location relative to the face image location within the captured videoframe.

16. An information handling system executing an intelligent face framing management system comprising:

a processor configured to execute code instructions of a multimedia multi-user collaboration application to join a videoconference session with a remotely located computing device;

a display screen, a speaker, and a microphone;

a plurality of video cameras configured to capture videoframes of a user;

the processor to execute code instructions of plural AV processing instruction modules to determine gaze and head orientation vectors of a user's image within the captured videoframes from each of the plurality of video cameras;

the processor to input the videoframe data from the plural cameras, including a series of plural videoframes from each of the plurality of video cameras, the gaze and head orientation vectors of a user's image within the captured videoframes, and audiovisual (AV) processing instructions to be applied to videoframes into the intelligent face framing management system;

the processor to execute code instructions of the intelligent face framing management system to select a gaze-centered video camera from the plurality of video cameras that has the minimum deviation from a central gaze of the user in the user's image in the captured videoframe;

the selected gaze-centered video camera to capture the videoframes of the user for the video conference; and

a network interface device configured to transmit a processed, encoded media sample, including the captured videoframe from the gaze-centered video camera to a remotely located computing device participating in the videoconference session.

17. The information handling system of claim 16 further comprising:

the processor to input the user's image in the captured videoframe from the gaze-centered video camera into the trained neural network of the intelligent face framing management system to determine a face center and a face image; and

the processor to execute code instructions of the intelligent face framing management system to output optimized face framing adjustments to center the face image location in the captured videoframe via an AV processing instruction module configured to crop the videoframes to center the face image location in the captured videoframe.

18. The information handling system of claim 16 further comprising:

the processor to input the user's image in the captured videoframe from the gaze-centered video camera into the trained neural network of the intelligent face framing management system to determine a face center and a face image; and

the processor to execute code instructions of the intelligent face framing management system to output optimized face framing adjustments to adjust the size of the face image relative to the captured videoframe to a size percentage range threshold of the face image in the captured videoframe via an AV processing instruction module configured to execute a digital zoom function on the face image in the captured videoframe to meet the size percentage range threshold.

19. The information handling system of claim 16 further comprising:

the processor to detect user movement in additional captured videoframes; and

the processor to execute code instructions of the trained neural network of the intelligent face framing management system to change selection of the gaze-centered video camera from the plural video cameras when the minimum deviation from a central gaze of the user in the user's image in the captured videoframe among the plural video cameras changes based on updated captured videoframes from the plurality of cameras, and updated gaze and head orientation vectors of the user's image within the captured videoframes.

20. The information handling system of claim 19 further comprising:

a proximity sensor to detect persons or objects in front of the plural video cameras; and

the processor to execute code instructions of an unusual movement detection (UMD) software module to receive proximity sensor distance data and to determine movement of a user location in front of the video camera relative to a user's image location within a series of captured videoframes to determine a change in selection of the gaze-centered video camera from the plurality of video cameras.

Assignments (9)

Continuity (1)

Cited By (2)

US 12,475,268 US 12,568,189