IP Library Granted Patent US 10,735,636
Granted Patent B2
US 10,735,636 · App. 15/450,909 · Granted Aug 4, 2020

Background replacement system and methods

Inventor: Keith A. Benson (Eden Prairie, MN)
Assignee: Shutterfly, LLC
H04N5/2256G06K9/2027G06K9/6202G06T7/11G06T7/194H04N5/2354G06T2207/10152
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,735,636
App. No.
15/450,909
Granted
Aug 4, 2020
Kind
B2
Abstract

A method of illuminating a subject during photography is disclosed. The method includes illuminating a background lighting system during a first image capture, illuminating a subject lighting system during a second image capture, and illuminating the background lighting system during a third image capture. The time from the first image capture to the third image capture is less than about 100 milliseconds. Also disclosed is a method for generating a composite mask of a subject, including receiving the three images, determining any movements of the subject between the first and third images, estimating an intermediate position of the subject during the second image, and generating a composite mask at the estimated intermediate position.

Claims (70)

1. A method of photographing a subject during a photography session, the method comprising:

receiving an image capture signal;

after receiving the image capture signal, initiating a lighting sequence for a single image capture sequence, the lighting sequence including:

initiating a first portrait lighting condition while a first image is captured;

initiating a second portrait lighting condition while a second image is captured; and

initiating a third portrait lighting condition while a third image is captured;

wherein the first, second, and third portrait lighting conditions are different from each other;

wherein at least one of the first, second, and third portrait lighting conditions is an illumination of the subject using a main light unit of a subject lighting system causing a shadowing effect on the subject;

wherein at least one other of the first, second, and third portrait lighting conditions is an illumination of a background using a background lighting system; and

wherein at least one further of the first, second, and third portrait lighting conditions is an illumination of the subject using a fill light unit of the subject lighting system causing a different shadowing effect on the subject than the main light unit; and

generating a new image from the first, second, and third images.

2. The method according to claim 1 , further comprising initiating a fourth portrait lighting condition while a fourth image is captured, wherein at least one of the first, second, third, and fourth portrait lighting conditions is ambient lighting.

3. The method according to claim 2 , the ambient lighting being illumination without using the subject lighting system and without using the background lighting system.

4. The method according to claim 1 , wherein the first portrait lighting condition is the illumination of the background using the background lighting system; and

wherein the second portrait lighting condition is the illumination of the subject using the main light unit of the subject lighting system.

5. The method according to claim 4 , wherein the background lighting system is not illuminated when the second image is captured.

6. The method according to claim 5 , wherein the main light unit of the subject lighting system is not illuminated when the first image is captured and wherein the main light unit of the subject lighting system is not illuminated when the third image is captured.

7. The method according to claim 1 , further comprising initiating a fourth portrait lighting condition while a fourth image is captured.

8. The method according to claim 7 , wherein the fourth portrait lighting condition includes illuminating the main light unit of the subject lighting system.

9. The method according to claim 7 , wherein the fourth portrait lighting condition includes illuminating the fill light unit of the subject lighting system.

10. The method according to claim 7 , wherein the fourth portrait lighting condition includes illuminating both the main light unit and the fill light unit of the subject lighting system.

11. The method according to claim 1 , wherein generating the new image from the first, second, and third images comprises:

generating transformed image data using the first, second, and third images captured, wherein the transformed image data includes at least one brightness correction.

12. The method according to claim 1 , wherein the different shadowing effect on the subject caused by the fill light unit is no shadowing.

13. A controller for photographing a subject during a photography session, the controller comprising:

a processing device; and

memory storing instructions that, when executed by the processing device, cause the controller to:

receive an image capture signal;

after receiving the image capture signal, initiate a lighting sequence for a single image capture sequence, the lighting sequence comprising:

initiating a first portrait lighting condition while a first image is captured;

initiating a second portrait lighting condition while a second image is captured; and

initiating a third portrait lighting condition while a third image is captured;

wherein the first, second, and third portrait lighting conditions are different from each other;

wherein at least one of the first, second, and third portrait lighting conditions is an illumination of the subject using a subject lighting system;

wherein at least one other of the first, second, and third portrait lighting conditions is an illumination of a background using a background lighting system; and

wherein at least one further of the first, second, and third portrait lighting conditions is ambient lighting to measure the ambient lighting for subtraction from one or more of the first and second images; and

generate a new image from the first, second, and third images.

14. The controller according to claim 13 , wherein the ambient lighting includes illumination without using the subject lighting system and without using the background lighting system.

15. The controller according to claim 14 , wherein the first portrait lighting condition is the ambient lighting;

wherein the second portrait lighting condition is the illumination of the subject using the subject lighting system; and

wherein the background lighting system is not illuminated when the second image is captured.

16. The controller according to claim 13 , the memory further storing instructions that, when executed by the processing device, cause the controller to:

initiate a fourth portrait lighting condition while a fourth image is captured.

17. The controller according to claim 16 , wherein the fourth portrait lighting condition includes illuminating a main light unit.

18. The controller according to claim 16 , wherein the fourth portrait lighting condition includes illuminating a fill light unit.

19. The controller according to claim 16 , wherein the fourth portrait lighting condition includes illuminating both a main light unit and a fill light unit.

20. The controller according to claim 13 , wherein at least one of the first, second, and third portrait lighting conditions includes illumination by a fill light.

21. A photography system for capturing multiple images of a subject, the system comprising:

a controller, including:

a processing device; and

a non-transitory storage medium;

a camera operable to capture first, second, and third image captures, wherein each of the first, second, and third image captures occur with same image focus and same image resolution;

a lighting system synchronized by the controller, the lighting system including:

a background lighting system in communication with the controller; and

a subject lighting system in communication with the controller;

wherein the non-transitory storage medium stores instructions that, when executed by the processing device, cause the controller to:

receive an image capture initiation signal;

initiate a first portrait lighting condition during the first image capture;

initiate a second portrait lighting condition during the second image capture; and

initiate a third portrait lighting condition during the third image capture;

wherein the first, second, and third portrait lighting conditions are different from each other;

wherein each of the first image capture, the second image capture, and the third image capture are for one pose;

wherein at least one of the first, second, and third portrait lighting conditions is an illumination of the subject using a main light unit of the subject lighting system causing a shadowing effect on the subject;

wherein at least one other of the first, second, and third portrait lighting conditions is an illumination of a background using the background lighting system; and

wherein at least one further of the first, second, and third portrait lighting conditions is an illumination of the subject using a fill light unit of the subject lighting system causing a different shadowing effect on the subject than the main light unit or ambient lighting to measure the ambient lighting for subtraction from one or more of the first and second images; and

generate a new image from the first, second, and third images.

22. The system according to claim 21 , the memory further storing instructions that, when executed by the processing device, cause the controller to:

initiate a fourth portrait lighting condition while a fourth image is captured;

wherein the ambient lighting includes illumination without using the subject lighting system and without using the background lighting system.

23. The system according to claim 22 , wherein the fourth portrait lighting condition includes illuminating both the main light unit and the fill light unit of the subject lighting system.

Assignments (7)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR AND ASSIGNEE NAMES PREVIOUSLY RECORDED AT REEL: 051688 FRAME: 0117. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER. Recorded Feb 24, 2020
From: LIFETOUCH, LLC
To: SHUTTERFLY, LLC
Reel/Frame 052003/0378 →
MERGER Recorded Jan 31, 2020
From: LIFETOUCH, LLC.
To: SHUTTERFLY, LLC.
Reel/Frame 051688/0117 →
ENTITY CONVERSION Recorded Jan 27, 2020
From: LIFETOUCH INC.
To: LIFETOUCH, LLC
Reel/Frame 051706/0875 →
FIRST LIEN SECURITY AGREEMENT Recorded Sep 27, 2019
From: LIFETOUCH INC.
To: WILMINGTON TRUST, NATIONAL ASSOCIATION
Reel/Frame 050548/0462 →
RELEASE OF SECURITY INTEREST Recorded Sep 26, 2019
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: SHUTTERFLY, INC.; LIFETOUCH INC.; LIFETOUCH NATIONAL SCHOOL STUDIOS INC.
Reel/Frame 050527/0868 →
SECURITY INTEREST Recorded May 23, 2018
From: SHUTTERFLY, INC.; LIFETOUCH INC.; LIFETOUCH NATIONAL SCHOOL STUDIOS INC.
To: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
Reel/Frame 046216/0396 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 3, 2017
From: BENSON, KEITH A.
To: LIFETOUCH INC.
Reel/Frame 043186/0991 →
Continuity (2)
Continuation 14695506 · Apr 24, 2015
Related Publication 20170289417A1 · Oct 5, 2017