IP Library › Granted Patent US 11,228,709
Granted Patent B2
US 11,228,709 · App. 16/605,895 · Granted Jan 18, 2022

Constructing images of users' faces by stitching non-overlapping images

Inventors: Madhu Sudan Athreya (Palo Alto, CA); William J. Allen (Corvallis, OR); Cheng I Huang (Palo Alto, CA)
Assignee: Hewlett-Packard Development Company, L.P.
H04N5/23238G06K9/00268H04N5/247
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,228,709
App. No.
16/605,895
Granted
Jan 18, 2022
Kind
B2
Abstract

An apparatus includes a support, a first camera, and second camera, and a processor. The support is configured to be mounted to a head of a user. The first camera is mounted on the support and positioned to capture a first image of a face of the user. The second camera is mounted on the support and positioned to capture a second image of the face of the user that is non-overlapping with the first image. The processor is mounted to the support and programmed to stitch the first image and the second image together to construct an image of the face of the user.

Claims (46)

1. An apparatus, comprising:

a support configured to project a plurality of fiducials onto a head of a user;

a first camera mounted on the support and positioned to capture a first image of a face of the user, wherein the first image depicts a first portion of the face and a first subset of the plurality of fiducials;

a second camera mounted on the support and positioned to capture a second image of the face of the user that is non-overlapping with the first image, wherein the second image depicts a second portion of the face and a second subset of the plurality of fiducials; and

a processor mounted to the support and programmed to identify spatial coordinates of a feature on the face that falls outside a first field of view of the first camera and a second field of view of the second camera, based on positions of the first subset of the plurality of fiducials and positions of the second subset of the plurality of fiducials.

2. The apparatus of claim 1 , wherein the first camera and the second camera are mounted on an inward-facing side of the support.

3. The apparatus of claim 1 , further comprising:

a light source mounted to the support.

4. A method, comprising:

projecting a plurality of fiducials onto a face of a user who is using a display, using a light source of the display;

capturing a first image using a first camera of the display, wherein the first image depicts a first portion of the face and a first subset of the plurality of fiducials;

capturing a second image using a second camera of the display, wherein the second image depicts a second portion of the face and a second subset of the plurality of fiducials, wherein the first image and the second image are non-overlapping; and

identifying spatial coordinates of a feature on the face that falls outside a first field of view of the first camera and a second field of view of the second camera, based on positions of the first subset of the plurality of fiducials and positions of the second subset of the plurality of fiducials.

5. The method of claim 4 , further comprising:

stitching the first image and the second image together to construct a single image of the face by establishing a correspondence between at least one fiducial of the first subset and at least one fiducial of the second subset.

6. The method of claim 4 , wherein a geometric separation between the first camera and the second camera is known prior to the projecting.

7. The method of claim 4 , further comprising:

interpolating between pixels of the first image and pixels of the second image; and

generating a third image to fill a gap between the first image and the second image, based on the interpolating.

8. The method of claim 4 , further comprising:

prior to the projecting, calibrating the first camera and the second camera to correct for static distortions.

9. The method of claim 8 , wherein the calibrating comprises:

deriving a first set of static transforms for the first camera and a second set of static transforms for the second camera.

10. The method of claim 9 , wherein the deriving comprises:

projecting a known pattern onto a surface, using the light source of the display;

capturing a first image of the known pattern on the surface, using the first camera;

capturing a second image of the known pattern on the surface, using the second camera;

correcting geometric distortions in the first image based on knowledge of the known pattern, wherein corrections to the geometric distortions in the first image become the first set of static transforms; and

correcting geometric distortions in the second image based on knowledge of the known pattern, wherein corrections to the geometric distortions in the second image become the second set of static transforms.

11. The method of claim 8 , further comprising:

subsequent to the calibrating to correct for static distortions and prior to the projecting, calibrating the first camera and the second camera to correct for dynamic distortions.

12. The method of claim 11 , wherein the calibrating comprises:

deriving a first set of dynamic transforms for the first camera and a second set of dynamic transforms for the second camera.

13. The method of claim 12 , wherein the deriving comprises:

projecting a known pattern onto a surface, using the light source of the display;

capturing a first image of the known pattern on the surface, using the first camera;

capturing a second image of the known pattern on the surface, using the second camera;

correcting geometric distortions in the first image based on knowledge of the known pattern, wherein corrections to the geometric distortions in the first image become the first set of dynamic transforms; and

correcting geometric distortions in the second image based on knowledge of the known pattern, wherein corrections to the geometric distortions in the second image become the second set of dynamic transforms.

14. A non-transitory machine-readable storage medium encoded with instructions executable by a processor, the machine-readable storage medium comprising:

instructions to project a plurality of fiducials onto a face of a user who is using a display, using a light source of the display;

instructions to capture a first image using a first camera of the display, wherein the first image depicts a first portion of the face and a first subset of the plurality of fiducials;

instructions to capture a second image using a second camera of the display, wherein the second image depicts a second portion of the face and a second subset of the plurality of fiducials, wherein the first image and the second image are non-overlapping; and

instructions to identify spatial coordinates of a feature on the face that falls outside a first field of view of the first camera and a second field of view of the second camera, based on positions of the first subset of the plurality of fiducials and positions of the second subset of the plurality of fiducials.

15. The non-transitory machine-readable storage medium of claim 14 , wherein the instructions further comprise:

instructions to stitch the first image and the second image together to construct a single image of the face by establishing a correspondence between at least one fiducial of the first subset and at least one fiducial of the second subset.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 17, 2019
From: ATHREYA, MADHU SUDAN; ALLEN, WILLIAM J.; HUANG, CHENG I
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 050747/0363 →
Continuity (1)
Related Publication 20210289134A1 · Sep 16, 2021
Cited By (1)
US 12,380,682