IP Library Granted Patent US 10,887,548
Granted Patent B2
US 10,887,548 · App. 16/104,572 · Granted Jan 5, 2021

Scaling image of speaker's face based on distance of face and size of display

Inventors: Brandon Mayer (New York, NY); Luke Sorenson (New York, NY); Matthew Frazier (Long Island City, NY); Wei Guan (Jersey City, NY); James McGill (New York, NY); Zachary Yorke (Brooklyn, NY); Adrien Baranes (Brooklyn, NY); John Finley (Brooklyn, NY)
Assignee: GOOGLE LLC
H04N7/15G06K9/00248G06K9/00335G10L17/00H04N7/147G06T3/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,887,548
App. No.
16/104,572
Granted
Jan 5, 2021
Kind
B2
Abstract

A method can include determining a distance of a person's face from a camera that captured an image of the person's face, determining a size of a display in communication with the camera, and scaling the image based on the determined distance of the person's face and the determined size of the display.

Claims (68)

1. A method of facilitating a video conference, the method comprising:

determining a distance of a person's face from a camera that captured an image of the person's face, the determining the distance of the person's face from the camera including:

measuring at least one of a width between a left eye and a right eye in the image of the person's face or a height from a mouth midpoint to an inter-eye midpoint in the image of the person's face;

determining at least one of a width ratio by dividing a predetermined width by the measured width or a height ratio by dividing a predetermined height by the measured height; and

multiplying at least one of the width ratio or the height ratio by a predetermined distance;

determining a size of a display in communication with the camera, the display being in a different room than the camera; and

scaling the image based on the determined distance of the person's face and the determined size of the display.

2. The method of claim 1 , wherein the determining the distance of the person's face includes:

recognizing the left eye and the right eye in the image of the person's face;

measuring the width between the left eye and the right eye in the image of the person's face;

dividing the predetermined width by the measured width to determine the width ratio; and

multiplying the width ratio by the predetermined distance.

3. The method of claim 1 , wherein the determining the distance of the person's face includes:

recognizing the left eye and the right eye in the image of the person's face;

determining the inter-eye midpoint of the left eye and the right eye;

recognizing a mouth in the image of the person's face;

determining the mouth midpoint of the mouth;

measuring the height from the mouth midpoint to the inter-eye midpoint in the image of the person's face;

dividing the predetermined height by the measured height to determine the height ratio; and

multiplying the height ratio by the predetermined distance.

4. The method of claim 1 , wherein the determining the distance of the person's face includes:

measuring the width between the left eye and the right eye in the image of the person's face;

measuring the height between the inter-eye midpoint and the mouth midpoint in the image of the person's face;

dividing a greater of the width and the height by the predetermined width if the width is greater and by the predetermined height if the height is greater to generate a ratio; and

multiplying the ratio by the predetermined distance.

5. The method of claim 1 , wherein the determining the size of the display includes determining the size of the display based on a display size indicator included in a Hypertext Transfer Protocol (HTTP) message received from a computing device associated with the display.

6. The method of claim 1 , wherein the scaling the image includes scaling the image to a size on the display that matches a size that the person's face would appear to a viewer at a location of the display from the determined distance of the person's face from the camera.

7. The method of claim 1 , further comprising recognizing the person's face as an active speaker based on a time difference between audio inputs indicating that a location of the active speaker matches a determined location of the person's face.

8. The method of claim 7 , wherein the location of the person's face is determined based on the determined distance of the person's face and a direction of the person's face from the camera.

9. The method of claim 1 , further comprising recognizing the person's face as an active speaker based on recognizing lip movement of the person's face.

10. The method of claim 1 , further comprising recognizing the person's face as an active speaker based on recognizing eye gazes of other persons toward the person's face.

11. The method of claim 1 , further comprising sending the scaled image to the display.

12. The method of claim 1 , further comprising receiving the captured image from the camera.

13. The method of claim 1 , wherein the method is performed by a computing system that controls the display.

14. The method of claim 1 , wherein the method is performed by a computing system that controls the camera.

15. A non-transitory computer-readable storage medium comprising instructions stored thereon for facilitating a video conference that, when executed by at least one processor, are configured to cause a computing system to at least:

determine a distance of a person's face from a camera that captured an image of the person's face, the determining the distance of the person's face from the camera including:

measuring at least one of a width between a left eye and a right eye in the image of the person's face or a height from a mouth midpoint to an inter-eye midpoint in the image of the person's face;

determining at least one of a width ratio by dividing a predetermined width by the measured width or a height ratio by dividing a predetermined height by the measured height; and

multiplying at least one of the width ratio or the height ratio by a predetermined distance;

determine a size of a display in communication with the camera; and

scale the image based on the determined distance of the person's face and the determined size of the display, the scaled image having a size inversely proportional to the determined distance of the person's face from the camera.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the determining the distance of the person's face includes:

recognizing the left eye and the right eye in the image of the person's face;

measuring the width between the left eye and the right eye in the image of the person's face;

dividing the predetermined width by the measured width to determine the width ratio; and

multiplying the width ratio by the predetermined distance.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the determining the distance of the person's face includes:

recognizing the left eye and the right eye in the image of the person's face;

determining the inter-eye midpoint of the left eye and the right eye;

recognizing a mouth in the image of the person's face;

determining the mouth midpoint of the mouth;

measuring the height from the mouth midpoint to the inter-eye midpoint in the image of the person's face;

dividing the predetermined height by the measured height to determine a height ratio; and

multiplying the height ratio by the predetermined distance.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the non-transitory computer-readable storage medium is stored on a computing system that controls the display.

19. The non-transitory computer-readable storage medium of claim 15 , wherein the non-transitory computer-readable storage medium is stored on a computing system that controls the camera.

20. A system for facilitating a video conference, the system comprising:

a camera configured to capture images;

a display configured to present images;

at least one processor configured to execute instructions; and

a non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by the at least one processor, are configured to cause the system to at least:

determine a distance of a person's face from the camera that captured an image of the person's face, the determining the distance of the person's face from the camera including:

measuring at least one of a width between a left eye and a right eye in the image of the person's face or a height from a mouth midpoint to an inter-eye midpoint in the image of the person's face;

determining at least one of a width ratio by dividing a predetermined width by the measured width or a height ratio by dividing a predetermined height by the measured height; and

multiplying at least one of the width ratio or the height ratio by a predetermined distance;

determine a size of the display, the display being in communication with the camera; and

scale the image based on the determined distance of the person's face and the determined size of the display, the scaled image having a size, when measured as a proportion of the display, inversely proportional to the determined size of the display.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2019
From: MAYER, BRANDON; SORENSON, LUKE; FRAZIER, MATTHEW; GUAN, WEI; YORKE, ZACHARY; BARANES, ADRIEN; FINLEY, JOHN
To: GOOGLE LLC
Reel/Frame 048473/0490 →
Cited By (2)
US 12,186,019 US 12,642,429