IP Library Granted Patent US 11,716,521
Granted Patent B2
US 11,716,521 · App. 17/036,876 · Granted Aug 1, 2023

Using IR sensor with beam splitter to obtain depth

Inventor: Joseph Wise (Los Angeles, CA)
Assignees: Sony Group Corporation; Sony Pictures Entertain Inc.
H04N23/11G01S17/894G02B27/1013H04N23/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,716,521
App. No.
17/036,876
Granted
Aug 1, 2023
Kind
B2
Abstract

Obtaining depth information using an IR sensor with a beam splitter including illuminating a subject with IR light using an IR light source; receiving reflected light including visible light and the IR light at a beam splitter; splitting the reflected light into two identical beams, a first beam and a second beam, using the beam splitter; receiving and processing the first beam at an IR sensor to pass the IR light and to block the visible light, to generate an IR image; receiving and processing the second beam at a visible light sensor to pass the visible light and to block the IR light, to generate a visible light image; and using a time of flight of the IR light transmitted by the IR light source and received by the IR sensor to calculate a distance of the subject from the beam splitter.

Claims (54)

1. A system comprising:

an infra-red (IR) light source to illuminate a subject with IR light;

a beam splitter to receive a beam of visible light and IR light reflected by the subject,

the beam splitter to split the beam into two identical beams, a first beam and a second beam, each of the first and second beams having the visible light and the IR light;

an IR sensor positioned next to the beam splitter, wherein the IR sensor is a time of flight sensor, the IR sensor to receive and process the first beam to pass the IR light and to block the visible light to generate an IR image,

wherein the IR light source is positioned next to the IR sensor such that a time of flight of the IR light illuminated by the IR light source and received by the IR sensor is used to calculate a distance of the subject from the beam splitter;

a visible light sensor coupled to the beam splitter,

the visible light sensor to receive and process the second beam to pass the visible light and to block the IR light to generate a visible light image;

a processor to receive and process the IR image from the IR sensor and the visible light image from the visible light sensor,

wherein the processor processes the visible light image to generate a 2-D image of the subject,

wherein the processor processes the IR image from the time of flight sensor and the calculated distance to generate depth information for the 2-D image of the subject, and

wherein the beam splitter generates two identical beams, each beam having the visible light and the IR light, to provide an identical spatial perspective for the IR image and the visible light image such that positions of the IR sensor and the visible light sensor do not need to be calibrated, wherein having the identical spatial perspective for the IR image and the visible light image provides synchronization between the two images.

2. The system of claim 1 , wherein the IR sensor is a time of flight sensor.

3. The system of claim 2 , wherein the IR sensor includes the IR light source to operate as the time of flight sensor.

4. The system of claim 1 , further comprising:

a visible light source including light from the natural environment to illuminate the subject.

5. A method comprising:

illuminating a subject with IR light using an IR light source;

positioning the IR light source next to an IR sensor;

receiving reflected light including visible light and the IR light at a beam splitter,

wherein the IR sensor is positioned next to the beam splitter;

splitting the reflected light into two identical beams, a first beam and a second beam, using the beam splitter, wherein each of the first and second beams has the visible light and the IR light;

receiving and processing the first beam at an IR sensor to pass the IR light and to block the visible light, to generate an IR image, wherein the IR sensor is a time of flight sensor;

receiving and processing the second beam at a visible light sensor to pass the visible light and to block the IR light, to generate a visible light image;

using a time of flight of the IR light transmitted by the IR light source and received by the IR sensor, which is connected to the IR sensor, to calculate a distance of the subject from the beam splitter;

processing the visible light image to generate a 2-D image of the subject;

processing the IR image from the time of flight sensor and the calculated distance to generate depth information for the 2-D image of the subject; and

wherein splitting the reflected light into two identical beams, each beam having the visible light and the IR light, provides an identical spatial perspective for the IR image and the visible light image such that positions of the IR sensor and the visible light sensor do not need to be calibrated, wherein having the identical spatial perspective for the IR image and the visible light image provides synchronization between the two images.

6. The method of claim 5 , further comprising:

illuminating the subject with visible light including light from the natural environment.

7. The method of claim 5 , wherein the reflected light is formed as a beam of visible light and IR light.

8. A non-transitory computer-readable storage medium storing a computer program to obtain depth information using an IR sensor with a beam splitter, the computer program comprising executable instructions that cause a computer to:

command an IR light source to illuminate a subject with IR light,

wherein the IR light source is positioned next to the IR sensor;

command a beam splitter to receive reflected light including visible light and the IR light;

command the beam splitter to split the reflected light into two identical beams, a first beam and a second beam, wherein the IR sensor is positioned next to the beam splitter;

command an IR sensor to process the first beam to pass the IR light and to block the visible light, to generate an IR image, wherein the IR sensor is a time of flight sensor;

command a visible light sensor to process the second beam to pass the visible light and to block the IR light, to generate a visible light image;

use a time of flight of the IR light transmitted by the IR light source and received by the IR sensor to calculate a distance of the subject from the beam splitter;

process the visible light image to generate a 2-D image of the subject; and

process the IR image from the time of flight sensor and the calculated distance to generate depth information for the 2-D image of the subject,

illuminating a subject with IR light using an IR light source;

connecting the IR light source to an IR sensor;

receiving reflected light including visible light and the IR light at a beam splitter;

splitting the reflected light into two identical beams, a first beam and a second beam, using the beam splitter, wherein each of the first and second beams has the visible light and the IR light;

receiving and processing the first beam at an IR sensor to pass the IR light and to block the visible light, to generate an IR image, wherein the IR sensor is a time of flight sensor;

receiving and processing the second beam at a visible light sensor to pass the visible light and to block the IR light, to generate a visible light image;

using a time of flight of the IR light transmitted by the IR light source and received by the IR sensor, which is connected to the IR sensor, to calculate a distance of the subject from the beam splitter;

processing the visible light image to generate a 2-D image of the subject;

processing the IR image from the time of flight sensor and the calculated distance to generate depth information for the 2-D image of the subject; and

wherein splitting the reflected light into two identical beams, each beam having the visible light and the IR light, provides an identical spatial perspective for the IR image and the visible light image such that positions of the IR sensor and the visible light sensor do not need to be calibrated, wherein having the identical spatial perspective for the IR image and the visible light image provides synchronization between the two images.

9. The non-transitory computer-readable storage medium of claim 8 , further comprising executable instructions that cause the computer to:

command a visible light source to illuminate the subject with visible light including light from the natural environment.

10. The non-transitory computer-readable storage medium of claim 8 , wherein the reflected light is formed as a beam of visible light and IR light.

Assignments (2)
CHANGE OF NAME Recorded Mar 21, 2023
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 063124/0719 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2020
From: WISE, JOSEPH
To: SONY CORPORATION; SONY PICTURES ENTERTAINMENT INC.
Reel/Frame 054070/0079 →
Continuity (2)
Provisional Application 62947765 · Dec 13, 2019
Related Publication 20210185247A1 · Jun 17, 2021