Granted Patent B2

US 12,597,290 · App. 18/246,625 · Granted Apr 7, 2026

Inventors: Sascha Haeberling (San Francisco, CA); Jason Lawrence (Seattle, WA)

Assignee: GOOGLE LLC

G06V40/174G06T15/00G06V10/30G06V40/171G06V2201/07

Abstract

Claims (92)

1 . A method comprising:

capturing at least one image using a plurality of cameras;

identifying a plurality of facial features associated with the at least one image;

stabilizing a location of at least one facial feature of the plurality of facial features;

converting the location of the at least one facial feature to a three-dimensional (3D) location;

reducing a noise associated with the 3D location of the at least one facial feature;

predicting a future location of the at least one facial feature; and

rendering a 3D image, on a flat panel display, using the future location of the at least one facial feature.

2 . The method of claim 1 , wherein

the at least one image includes two or more images, and

the two or more images are captured at least one of sequentially by the same camera and at the same time by two or more cameras.

3 . The method of claim 1 , wherein the identifying a plurality of facial features includes:

identifying a face associated with the at least one image,

generating a plurality of landmarks corresponding to the plurality of facial features on the face, and

associating a location with each of the plurality of landmarks.

4 . The method of claim 1 , wherein the identifying a plurality of facial features includes:

identifying a first face associated with the at least one image,

identifying a second face associated with the at least one image,

generating a first plurality of landmarks corresponding to the plurality of facial features on the first face,

generating a second plurality of landmarks corresponding to the plurality of facial features on the second face,

associating a first location with each of the first plurality of landmarks, and

associating a second location with each of the second plurality of landmarks.

5 . The method of claim 1 , wherein the stabilizing of the location of the at least one facial feature includes:

selecting a facial feature from at least two of the at least one image,

selecting at least two landmarks associated with the facial feature, and

averaging the location of the at least two landmarks.

6 . The method of claim 1 , wherein the stabilizing of the location of the at least one facial feature includes:

generating a plurality of landmarks corresponding to the plurality of facial features,

selecting a subset of the plurality of landmarks,

determining a motion of a face based on a velocity associated with the location of each landmark of the subset of the plurality of landmarks, and

the stabilizing of the location of the at least one facial feature is based on the motion of the face.

7 . The method of claim 1 , wherein the converting of the location of the at least one facial feature to a 3D location includes:

triangulating the location of the at least one facial feature based on location data generated using images captured by three or more of the plurality of cameras.

8 . The method of claim 1 , wherein the noise is reduced by applying a double exponential filter to the 3D location of the at least one facial feature.

9 . The method of claim 1 , wherein the predicting of the future location of the at least one facial feature includes:

applying a double exponential filter to a sequence of the 3D location of the at least one facial feature, and

selecting a value at a time greater than zero (0) that is based on the double exponential filtered sequence of the 3D location before the time zero (0).

10 . The method of claim 1 , further comprising generating audio using the future location of the at least one facial feature.

11 . A three-dimensional (3D) content system comprising:

a memory including code segments representing a plurality of computer instructions; and

a processor configured to execute the code segments, the computer instructions including:

capturing at least one image using a plurality of cameras;

identifying a plurality of facial features associated with the at least one image;

stabilizing a location of at least one facial feature of the plurality of facial features;

converting the location of the at least one facial feature to a three-dimensional (3D) location;

reducing a noise associated with the 3D location of the at least one facial feature;

predicting a future location of the at least one facial feature; and

rendering a 3D image, on a flat panel display, using the future location of the at least one facial feature.

12 . The 3D content system of claim 11 , wherein

the at least one image includes two or more images, and

the two or more images are captured at least one of sequentially by the same camera and at the same time by two or more cameras.

13 . The 3D content system of claim 11 , wherein the identifying a plurality of facial features includes:

identifying a face associated with the at least one image,

generating a plurality of landmarks corresponding to the plurality of facial features on the face, and

associating a location with each of the plurality of landmarks.

14 . The 3D content system of claim 11 , wherein the identifying a plurality of facial features includes:

identifying a first face associated with the at least one image,

identifying a second face associated with the at least one image,

generating a first plurality of landmarks corresponding to the plurality of facial features on the first face,

generating a second plurality of landmarks corresponding to the plurality of facial features on the second face,

associating a first location with each of the first plurality of landmarks, and

associating a second location with each of the second plurality of landmarks.

15 . The 3D content system of claim 11 , wherein the stabilizing of the location of the at least one facial feature includes:

selecting a facial feature from at least two of the at least one image,

selecting at least two landmarks associated with the facial feature, and

averaging the location of the at least two landmarks.

16 . The 3D content system of claim 11 , wherein the stabilizing of the location of the at least one facial feature includes:

generating a plurality of landmarks corresponding to the plurality of facial features,

selecting a subset of the plurality of landmarks,

determining a motion of a face based on a velocity associated with the location of each landmark of the subset of the plurality of landmarks, and

the stabilizing of the location of the at least one facial feature is based on the motion of the face.

17 . The 3D content system of claim 11 , wherein the converting of the location of the at least one facial feature to a 3D location includes:

triangulating the location of the at least one facial feature based on location data generated using images captured by three or more of the plurality of cameras.

18 . The 3D content system of claim 11 , wherein the noise is reduced by applying a double exponential filter to the 3D location of the at least one facial feature.

19 . The 3D content system of claim 11 , wherein the predicting of the future location of the at least one facial feature includes:

applying a double exponential filter to a sequence of the 3D location of the at least one facial feature, and

selecting a value at a time greater than zero (0) that is based on the double exponential filtered sequence of the 3D location before the time zero (0).

20 . The 3D content system of claim 11 , wherein the computer instructions further include generating audio using the future location of the at least one facial feature.

21 . A non-transitory computer-readable storage medium having stored thereon computer executable program code which, when executed on a computer system, causes the computer system to perform a method comprising:

capturing at least one image using a plurality of cameras;

identifying a plurality of facial features associated with the at least one image;

stabilizing a location of at least one facial feature of the plurality of facial features;

converting the location of the at least one facial feature to a three-dimensional (3D) location;

applying a double exponential filter to the 3D location of the at least one facial feature to reduce a noise associated with the 3D location of the at least one facial feature;

predicting a future location of the at least one facial feature;

determining a display position for rendering a 3D image, on a flat panel display, using the future location of the at least one facial feature; and

determining an audio position for generating audio using the future location of the at least one facial feature.

22 . The non-transitory computer-readable storage medium of claim 21 , wherein the stabilizing of the location of the at least one facial feature includes:

generating a plurality of landmarks corresponding to the plurality of facial features,

selecting a subset of the plurality of landmarks,

determining a motion of a face based on a velocity associated with the location of each landmark of the subset of the plurality of landmarks, and

the stabilizing of the location of the at least one facial feature is based on the motion of the face.

Assignments (1)

Continuity (1)

References Cited (26)