IP Library Patent Application 15591347
Patent Application
App. No. 15/591,347

BACKGROUND REPLACEMENT UTILIZING INFRARED LIGHT AND VISIBLE LIGHT

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
15/591,347
Abstract

Systems and methods for background replacement of images captured using both infrared light and visible light are disclosed. Example methods of imaging a subject include capturing an image while illuminating a subject with a visible light assembly and illuminating a background with an infrared light assembly. Digital cameras capturing subject images during illumination include image sensor arrays having both visible light sensors and infrared light sensors.

Claims (56)

1 . A method of imaging a subject with a digital camera, comprising:

illuminating a subject with a visible light assembly;

illuminating a background with an infrared light assembly; and

while illuminating the subject and illuminating the background, capturing an image with the digital camera, the digital camera having an image sensor array, the image sensor array including sensors configured to receive visible light and sensors configured to receive infrared light.

2 . The method according to claim 1 , further comprising:

using the image, generating a color image file; and

using the image, generating an infrared image file.

3 . The method according to claim 1 , further comprising:

filtering out infrared light from the visible light assembly.

4 . The method according to claim 3 , further comprising:

filtering out visible light from the infrared light.

5 . The method according to claim 1 , further comprising:

testing an environment for infrared light, the image being captured in the environment; and

determining an adjustment factor based on the testing, wherein generating the infrared image file includes using the adjustment factor.

6 . The method according to claim 1 , further comprising:

generating a mask based on the infrared image file.

7 . The method according to claim 6 , further comprising:

using the mask to remove a background portion of the color image file to generate a modified color image file.

8 . The method according to claim 7 , further comprising:

receiving a substitute background; and

generating a composite image using the modified color image file and the substitute background.

9 . The method according to claim 8 , further comprising:

illuminating a foreground object with a second infrared lighting unit.

10 . The method according to claim 9 , wherein using the mask to remove the background portion of the color image file further comprises:

removing the foreground object from the color image file.

11 . The method according to claim 1 , wherein the subject is an inanimate object.

12 . A digital photography system, comprising:

a digital camera comprising a sensor array, the sensor array being configured to capture both visible light and infrared light;

a background lighting system including an infrared lighting unit;

a subject lighting system including a visible lighting unit; and

a controller in communication with the background lighting system and the subject lighting system, the controller comprising:

at least one processing device; and

at least one non-transitory data storage device, the data storage device storing instructions that, when executed by the at least one processing device, cause the controller to:

activate the background lighting system to emit infrared light;

activate the subject lighting system to emit visible light; and

when both the background lighting system and the subject lighting system are activated, activate the digital camera to capture an image with the sensor array.

13 . The digital photography system of claim 12 , further comprising: a mask generation system including a second non-transitory storage medium and one or more processors, the mask generation system configured to:

generate, using the image, a visible light image file and an infrared light image file.

14 . The digital photography system of claim 13 , wherein the mask generation system is further configured to:

using the infrared light image file, generate a mask file; and

using the mask file, remove a background portion of the visible light image file.

15 . The digital photography system of claim 13 , wherein the data storage device further stores instructions that, when executed by the at least one processing device, cause the controller to:

test an environment for infrared light, wherein the digital camera is located within the environment; and

determine an adjustment factor based on the test, wherein the infrared light image file is generated using the adjustment factor.

16 . The digital photography system of claim 12 , wherein the infrared lighting unit includes a visible light filter; and

wherein the visible lighting unit is configured to filter out infrared light.

17 . The digital photography system of claim 12 , wherein the sensor array includes a filter array including red filters, green filters, blue filters, and infrared filters.

18 . A method for replacing a background of a digital image, the digital image including a subject, and the method comprising:

receiving an infrared image file, the infrared image being captured with an image sensor array, the image sensor array including a plurality of infrared sensors and a plurality of visible light sensors; and

using the infrared image, generating a mask.

19 . The method according to claim 18 , further comprising:

receiving a visible light image file, the visible light image being captured with the image sensor array, the visible light image resulting from reading out the signals received by the plurality of visible light sensors; and

using the mask, removing a background from the visible light image to generate a subject only image.

20 . The method according to claim 19 , further comprising:

adding a modified background to the subject only image; and

processing the visible light image to correct for at least one of: pixel multiplexing, channel crosstalk, and chromatic aberrations.

Assignments (8)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR AND ASSIGNEE NAMES PREVIOUSLY RECORDED AT REEL: 051688 FRAME: 0117. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER. Recorded Feb 24, 2020
From: LIFETOUCH, LLC
To: SHUTTERFLY, LLC
Reel/Frame 052003/0378 →
MERGER Recorded Jan 31, 2020
From: LIFETOUCH, LLC.
To: SHUTTERFLY, LLC.
Reel/Frame 051688/0117 →
ENTITY CONVERSION Recorded Jan 27, 2020
From: LIFETOUCH INC.
To: LIFETOUCH, LLC
Reel/Frame 051706/0875 →
FIRST LIEN SECURITY AGREEMENT Recorded Sep 27, 2019
From: LIFETOUCH INC.
To: WILMINGTON TRUST, NATIONAL ASSOCIATION
Reel/Frame 050548/0462 →
RELEASE OF SECURITY INTEREST Recorded Sep 26, 2019
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: SHUTTERFLY, INC.; LIFETOUCH INC.; LIFETOUCH NATIONAL SCHOOL STUDIOS INC.
Reel/Frame 050527/0868 →
SECURITY INTEREST Recorded May 23, 2018
From: SHUTTERFLY, INC.; LIFETOUCH INC.; LIFETOUCH NATIONAL SCHOOL STUDIOS INC.
To: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
Reel/Frame 046216/0396 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2017
From: PETERSON, BRENT; SURMA, MICHAEL
To: LIFETOUCH INC.
Reel/Frame 044082/0630 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 19, 2017
From: PETERSON, BRENT; SURMA, MICHAEL
To: LIFETOUCH INC.
Reel/Frame 043039/0252 →