IP Library Granted Patent US 10,110,792
Granted Patent B2
US 10,110,792 · App. 14/695,506 · Granted Oct 23, 2018

Background replacement system and methods involving multiple captured images

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,110,792
App. No.
14/695,506
Granted
Oct 23, 2018
Kind
B2
Abstract

A method of illuminating a subject during photography is disclosed. The method includes illuminating a background lighting system during a first image capture, illuminating a subject lighting system during a second image capture, and illuminating the background lighting system during a third image capture. The time from the first image capture to the third image capture is less than about 100 milliseconds. Also disclosed is a method for generating a composite mask of a subject, including receiving the three images, determining any movements of the subject between the first and third images, estimating an intermediate position of the subject during the second image, and generating a composite mask at the estimated intermediate position.

Claims (62)

1. A method of illuminating a subject during a photography session, comprising:

receiving an image capture signal; and

after receiving the image capture signal, initiating an illumination sequence for a single image capture sequence, including:

initiating a first illumination condition while a first image is captured;

initiating a second illumination condition while a second image is captured; and

initiating the first illumination condition while a third image is captured,

wherein at least one of the first and the second illumination conditions is an illumination of the subject using a subject lighting system, and wherein the other is an illumination of a background using a background lighting system;

wherein capturing of the images occurs with equal time intervals therebetween; and

wherein capturing of the images occurs in less than 100 milliseconds.

2. The method of claim 1 , wherein the illumination sequence for a single image capture sequence is for a single pose of the subject.

3. The method of claim 1 , wherein the capturing of the images occurs in less than about 20 milliseconds.

4. The method of claim 1 , wherein the first illumination condition is the illumination of the background using the background lighting system, and wherein the second illumination condition is the illumination of the subject using the subject lighting system.

5. The method of claim 3 , wherein the background lighting system is not illuminated when the second image is captured.

6. The method of claim 5 , wherein the subject lighting system is not illuminated when the first image is captured and when the third image is captured.

7. A method of generating a composite mask, the method comprising:

receiving a first image of a subject representative of a first position of the subject at a first time before an event;

receiving a second image of the subject representative of a second position of the subject at a second time after the event;

identifying differences between the first image and the second image representative of movement of the subject between the first position and the second position;

estimating an intermediate position of the subject at a time of the event based at least in part on the differences between the first image and the second image, the intermediate position being between the first position and the second position; and

generating the composite mask identifying the intermediate position of the subject.

8. The method of claim 7 , wherein the event is the capturing of a subject-illuminated image of the subject.

9. The method of claim 8 , further comprising using the composite mask to remove a background portion of the subject-illuminated image.

10. The method of claim 7 , wherein the first image of the subject is a first mask generated from a first background-illuminated image of the subject; and wherein the second image of the subject is a second mask generated from a second background-illuminated image of the subject.

11. The method of claim 10 , wherein the second background-illuminated image is captured within less than about 100 milliseconds of the first background-illuminated image capture.

12. The method of claim 10 , wherein identifying differences between the first image and the second image comprises:

dividing the first mask into a first set of tiles;

dividing the second mask into a second set of tiles; and

comparing one of the first set of tiles to one or more of the second set of tiles.

13. The method of claim 12 , wherein comparing one of the first set of tiles to one or more of the second set of tiles includes using a spatial weighting array.

14. The method of claim 13 , wherein estimating an intermediate position of the subject at the time of the event includes averaging one or more adjacent tiles.

15. A system for capturing and processing multiple images, comprising:

a controller;

a lighting system synchronized by the controller, the lighting system including:

a background lighting system in communication with the controller;

a subject lighting system in communication with the controller; and

an image capture initiation module,

wherein the controller includes a non-transitory storage medium and one or more processors, the non-transitory storage medium storing instructions that, when executed by the processor, cause the controller to:

receive an initiation signal from the image capture initiation module;

illuminate the background lighting system during a first image capture of the subject at a first time;

illuminate the subject lighting system during a second image capture of the subject at a second time; and

illuminate the background lighting system during a third image capture at a third time,

wherein each of the first image capture, second image capture, and

third image capture are for one pose;

a mask generation system including a second non-transitory storage medium and one or more processors, the mask generation system configured to:

receive the first image capture of the subject at the first time;

receive the second image capture of the subject at the second time; and

receive the third image capture of the subject at the third time.

16. The system of claim 15 , wherein the second non-transitory storage medium includes instructions that, when executed by the one or more processors, cause the mask generation system to:

determine any movements of the subject between the first and second images;

estimate an intermediate position of the subject at a time of the event between the first time and the second time; and

generate the composite mask at the intermediate position of the subject.

17. The system of claim 16 , wherein the difference between the third time and the first time is less than about 100 milliseconds;

wherein a difference between the first time and the second time is equal to a difference between the second time and the third time;

wherein the background lighting system is not illuminated during the second image capture; and

wherein the subject lighting system is not illuminated during the first image capture and the third image capture.

18. The system of claim 16 , wherein the difference between the third time and the first time is less than about 20 milliseconds;

wherein a difference between the first time and the second time is equal to a difference between the second time and the third time;

wherein determining any movements of the subject between the first and second images includes:

generating a first mask using the first image of the subject;

generating a second mask using the second image of the subject; and

comparing the first mask to the second mask.

19. The system of claim 18 , further comprising a mirrorless digital camera configured to capture each of the first image, the second image, and the third image, wherein the mirrorless digital camera is in communication with the controller and the mask generation system.

Assignments (7)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR AND ASSIGNEE NAMES PREVIOUSLY RECORDED AT REEL: 051688 FRAME: 0117. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER. Recorded Feb 24, 2020
From: LIFETOUCH, LLC
To: SHUTTERFLY, LLC
Reel/Frame 052003/0378 →
MERGER Recorded Jan 31, 2020
From: LIFETOUCH, LLC.
To: SHUTTERFLY, LLC.
Reel/Frame 051688/0117 →
ENTITY CONVERSION Recorded Jan 27, 2020
From: LIFETOUCH INC.
To: LIFETOUCH, LLC
Reel/Frame 051706/0875 →
FIRST LIEN SECURITY AGREEMENT Recorded Sep 27, 2019
From: LIFETOUCH INC.
To: WILMINGTON TRUST, NATIONAL ASSOCIATION
Reel/Frame 050548/0462 →
RELEASE OF SECURITY INTEREST Recorded Sep 26, 2019
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: SHUTTERFLY, INC.; LIFETOUCH INC.; LIFETOUCH NATIONAL SCHOOL STUDIOS INC.
Reel/Frame 050527/0868 →
SECURITY INTEREST Recorded May 23, 2018
From: SHUTTERFLY, INC.; LIFETOUCH INC.; LIFETOUCH NATIONAL SCHOOL STUDIOS INC.
To: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
Reel/Frame 046216/0396 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2015
From: BENSON, KEITH A.
To: LIFETOUCH INC.
Reel/Frame 035512/0287 →