IP Library Patent Application 14378119
Patent Application
App. No. 14/378,119

DEPTH IMAGE GENERATION UTILIZING DEPTH INFORMATION RECONSTRUCTED FROM AN AMPLITUDE IMAGE

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
14/378,119
Abstract

An image processing system comprises an image processor having image processing circuitry and an associated memory. The image processor is configured to identify a region of interest in an amplitude image, to detect one or more relatively low gradient regions in the region of interest, to reconstruct depth information for said one or more relatively low gradient regions, to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest, and to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information. The image processor in some embodiments is adapted for coupling to an active lighting image sensor, such as an infrared sensor that does not provide depth information corresponding to the amplitude image, or an SL or ToF sensor that provides depth information corresponding to the amplitude image.

Claims (46)

1 . A method comprising steps of:

identifying a region of interest in an amplitude image;

detecting one or more relatively low gradient regions in the region of interest;

reconstructing depth information for said one or more relatively low gradient regions in the region of interest;

extending the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and

generating a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information;

wherein the steps are implemented in an image processor comprising a processor coupled to a memory.

2 . The method of claim 1 wherein said one or more relatively low gradient regions comprise regions having respective gradients at or below a specified gradient threshold.

3 . The method of claim 1 wherein said one or more relatively low gradient regions comprise regions having respective substantially zero gradients.

4 . The method of claim 1 wherein identifying a region of interest comprises generating a binary region of interest mask in which pixels within the region of interest all have a first binary value and pixels outside the region of interest all have a second binary value complementary to the first binary value.

5 . The method of claim 1 wherein reconstructing depth information for said one or more relatively low gradient regions comprises reconstructing depth information for said one or more relatively low gradient regions but not for other portions of the region of interest.

6 . The method of claim 1 wherein the amplitude image comprises a luminance image generated by an active lighting imager.

7 . The method of claim 6 wherein the active lighting imager comprises one of an active lighting infrared image sensor, an SL image sensor and a ToF image sensor.

8 . The method of claim 1 wherein reconstructing depth information for said one or more relatively low gradient regions in the region of interest comprises:

determining at least one coefficient that relates amplitude to depth for a particular image sensor configuration; and

utilizing the coefficient to estimate one or more depth values from one or more corresponding amplitude values for respective pixels of said one or more relatively low gradient regions.

9 . The method of claim 8 wherein the determined coefficient comprises an element of a coefficient function that relates amplitude to depth for a given pixel as a function of a distance between that pixel and an approximate center of an image frame in the particular image sensor configuration.

10 . The method of claim 1 wherein extending the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest comprises computing one or more depth values for pixels outside of said one or more relatively low gradient regions as a function of depth values determined for pixels within said one or more relatively low gradient regions.

11 . The method of claim 10 wherein a depth value for a given pixel outside of said one or more relatively low gradient regions is computed utilizing a mean of a plurality of depth values for respective pixels within said one or more relatively low gradient regions.

12 . The method of claim 1 wherein identifying a region of interest comprises identifying the region of interest utilizing the amplitude image and a corresponding coarse depth image.

13 . The method of claim 1 wherein generating a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information further comprises:

generating a reconstructed depth image comprising at least portions of the reconstructed depth information and the extended reconstructed depth information; and

combining a coarse depth image with the reconstructed depth image.

14 . An article of manufacture comprising a computer-readable storage medium having computer program code embodied therein, wherein the computer program code when executed in the image processor causes the image processor to perform the method of claim 1 .

15 . An apparatus comprising:

an image processor adapted for coupling to an image sensor;

the image processor comprising image processing circuitry and an associated memory;

wherein the image processor is configured:

to identify a region of interest in an amplitude image;

to detect one or more relatively low gradient regions in the region of interest;

to reconstruct depth information for said one or more relatively low gradient regions in the region of interest;

to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and

to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information.

16 . An integrated circuit comprising the apparatus of claim 15 .

17 . An imager comprising:

an image sensor; and

an image processor coupled to the image sensor;

wherein the image processor is configured:

to identify a region of interest in an amplitude image;

to detect one or more relatively low gradient regions in the region of interest;

to reconstruct depth information for said one or more relatively low gradient regions in the region of interest;

to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and

to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information.

18 . The imager of claim 17 wherein the image sensor comprises an active lighting infrared image sensor that does not provide depth information corresponding to the amplitude image.

19 . The imager of claim 17 wherein the image sensor comprises an SL or ToF image sensor that provides depth information corresponding to the amplitude image.

20 . An image processing system comprising the imager of claim 17 .

Assignments (4)
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Feb 3, 2017
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
Reel/Frame 041710/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2016
From: LSI CORPORATION
To: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
Reel/Frame 038062/0967 →
PATENT SECURITY AGREEMENT Recorded Feb 11, 2016
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 037808/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2014
From: MAZURENKO, IVAN L.; RADOVANOVIC, NIKOLA; PARKHOMENKO, DENIS V.; KHOLODENKO, ALEXANDER B.; PARFENOV, DENIS V.
To: LSI CORPORATION
Reel/Frame 033512/0885 →