IP Library › Granted Patent US 9,761,000
Granted Patent B2
US 9,761,000 · App. 14/858,471 · Granted Sep 12, 2017

Systems and methods for non-obstacle area detection

Inventors: Gokce Dane (San Diego, CA); Vasudev Bhaskaran (San Diego, CA)
Assignee: QUALCOMM Incorporated
G06T7/0051G06K9/00791G06K9/00798G06T7/0081G06K9/6292G06T2207/10028G06T2207/30256
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,761,000
App. No.
14/858,471
Granted
Sep 12, 2017
Kind
B2
Abstract

A method performed by an electronic device is described. The method includes performing vertical processing of a depth map to determine a vertical non-obstacle estimation. The method also includes performing horizontal processing of the depth map to determine a horizontal non-obstacle estimation. The method further includes combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation. The method additionally includes generating a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

Claims (117)

1. A method performed by an electronic device, comprising:

performing vertical processing of a depth map to determine a vertical non-obstacle estimation, wherein performing vertical processing comprises:

dividing the depth map into segments, wherein at least one segment is a portion of a column of pixels in the depth map; and

estimating linear model parameters for the at least one segment to determine the vertical non-obstacle estimation;

performing horizontal processing of the depth map to determine a horizontal non-obstacle estimation;

combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation; and

generating a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

2. The method of claim 1 , wherein performing vertical processing further comprises:

generating a vertical reliability map that includes a reliability value for the vertical non-obstacle estimation.

3. The method of claim 2 , wherein determining the vertical reliability map comprises:

determining a segment fitting error for a given segment based on a difference between the estimated linear model parameters and pre-determined linear model parameters;

determining a reliability value for the given segment by comparing the segment fitting error to a vertical estimation threshold; and

applying the reliability value for the given segment to at least one pixel in the given segment.

4. The method of claim 3 , wherein the pre-determined linear model parameters are selected from among a plurality of road condition models, wherein the plurality of road condition models have a corresponding set of linear model parameters.

5. The method of claim 1 , wherein performing horizontal processing comprises:

obtaining a depth histogram for at least one row of pixels of the depth map;

determining a terrain line from the depth histogram;

determining a horizontal non-obstacle estimation based on a distance of a depth value of at least one pixel from the terrain line; and

generating a horizontal reliability map that includes a reliability value for the horizontal non-obstacle estimation.

6. The method of claim 5 , wherein generating the horizontal reliability map comprises:

determining whether the depth value of a given pixel is within a range of a mode of the depth histogram, wherein the given pixel has a high reliability value when the depth value of the given pixel is within the range of the mode of the depth histogram.

7. The method of claim 1 , wherein combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

performing both the vertical processing and the horizontal processing in parallel; and

merging the vertical non-obstacle estimation and the horizontal non-obstacle estimation based on a vertical reliability map and a horizontal reliability map.

8. The method of claim 7 , wherein a given pixel is identified as a non-obstacle area in the non-obstacle map where both the vertical reliability map and the horizontal reliability map are characterized by a high reliability value for the given pixel.

9. The method of claim 7 , wherein a given pixel is identified as an obstacle area in the non-obstacle map where at least one of the vertical reliability map or the horizontal reliability map are characterized by a low reliability value for the given pixel.

10. The method of claim 7 , wherein a given pixel is identified as a non-obstacle area or obstacle area in the non-obstacle map based on a coordinate of the given pixel where the vertical reliability map and the horizontal reliability map are characterized by different reliability values for the given pixel.

11. The method of claim 1 , wherein combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

performing vertical processing of the depth map;

obtaining a vertical reliability map based on a model fitting confidence;

identifying reliable regions of the vertical reliability map as non-obstacle areas; and

performing horizontal processing on unreliable regions of the vertical reliability map to determine whether the unreliable regions are non-obstacle areas.

12. The method of claim 1 , wherein combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

performing horizontal processing of the depth map;

obtaining a horizontal reliability map based on a depth histogram distance;

identifying reliable regions of the horizontal reliability map as non-obstacle areas; and

performing vertical processing on unreliable regions of the horizontal reliability map to determine whether the unreliable regions are non-obstacle areas.

13. The method of claim 1 , wherein the non-obstacle map is used in identifying a region of interest used by at least one of an object detection algorithm or a lane detection algorithm.

14. An electronic device, comprising:

a processor;

memory in electronic communication with the processor; and

instructions stored in the memory, the instructions being executable by the processor to:

perform vertical processing of a depth map to determine a vertical non-obstacle estimation, wherein the instructions being executable by the processor to perform vertical processing comprise instructions being executable by the processor to:

divide the depth map into segments, wherein at least one segment is a portion of a column of pixels in the depth map; and

estimate linear model parameters for the at least one segment to determine the vertical non-obstacle estimation;

perform horizontal processing of the depth map to determine a horizontal non-obstacle estimation;

combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation; and

generate a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

15. The electronic device of claim 14 , wherein the instructions being executable by the processor to perform vertical processing further comprise instructions being executable by the processor to:

generate a vertical reliability map that includes a reliability value for the vertical non-obstacle estimation.

16. The electronic device of claim 14 , wherein the instructions being executable by the processor to perform horizontal processing comprise instructions being executable by the processor to:

obtain a depth histogram for at least one row of pixels of the depth map;

determine a terrain line from the depth histogram;

determine a horizontal non-obstacle estimation based on a distance of a depth value of at least one pixel from the terrain line; and

generate a horizontal reliability map that includes a reliability value for the horizontal non-obstacle estimation.

17. The electronic device of claim 14 , wherein the instructions being executable by the processor to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise instructions being executable by the processor to:

perform both the vertical processing and the horizontal processing in parallel; and

merge the vertical non-obstacle estimation and the horizontal non-obstacle estimation based on a vertical reliability map and a horizontal reliability map.

18. The electronic device of claim 14 , wherein the instructions being executable by the processor to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise instructions being executable by the processor to:

perform vertical processing of the depth map;

obtain a vertical reliability map based on a model fitting confidence;

identify reliable regions of the vertical reliability map as non-obstacle areas; and

perform horizontal processing on unreliable regions of the vertical reliability map to determine whether the unreliable regions are non-obstacle areas.

19. The electronic device of claim 14 , wherein the instructions being executable by the processor to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise instructions being executable by the processor to:

perform horizontal processing of the depth map;

obtain a horizontal reliability map based on a depth histogram distance;

identify reliable regions of the horizontal reliability map as non-obstacle areas; and

perform vertical processing on unreliable regions of the horizontal reliability map to determine whether the unreliable regions are non-obstacle areas.

20. An apparatus, comprising:

means for performing vertical processing of a depth map to determine a vertical non-obstacle estimation, wherein the means for performing vertical processing comprise:

means for dividing the depth map into segments, wherein at least one segment includes a number of pixels in a column; and

means for estimating linear model parameters for at least one segment to determine the vertical non-obstacle estimation;

means for performing horizontal processing of the depth map to determine a horizontal non-obstacle estimation;

means for combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation; and

means for generating a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

21. The apparatus of claim 20 , wherein the means for performing vertical processing further comprise:

means for generating a vertical reliability map that includes a reliability value for the vertical non-obstacle estimation.

22. The apparatus of claim 20 , wherein the means for performing horizontal processing comprise:

means for obtaining a depth histogram for at least one row of pixels of the depth map;

means for determining a terrain line from the depth histogram;

means for determining a horizontal non-obstacle estimation based on a distance of a depth value of at least one pixel from the terrain line; and

means for generating a horizontal reliability map that includes a reliability value for the horizontal non-obstacle estimation.

23. The apparatus of claim 20 , wherein the means for combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise:

means for performing both the vertical processing and the horizontal processing in parallel; and

means for merging the vertical non-obstacle estimation and the horizontal non-obstacle estimation based on a vertical reliability map and a horizontal reliability map.

24. The apparatus of claim 20 , wherein the means for combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise:

means for performing vertical processing of the depth map;

means for obtaining a vertical reliability map based on a model fitting confidence;

means for identifying reliable regions of the vertical reliability map as non-obstacle areas; and

means for performing horizontal processing on unreliable regions of the vertical reliability map to determine whether the unreliable regions are non-obstacle areas.

25. The apparatus of claim 20 , wherein the means for combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise:

means for performing horizontal processing of the depth map;

means for obtaining a horizontal reliability map based on a depth histogram distance;

means for identifying reliable regions of the horizontal reliability map as non-obstacle areas; and

means for performing vertical processing on unreliable regions of the horizontal reliability map to determine whether the unreliable regions are non-obstacle areas.

26. A non-transitory tangible computer-readable medium comprising computer executable code, comprising:

code for causing an electronic device to perform vertical processing of a depth map to determine a vertical non-obstacle estimation, wherein the code for causing the electronic device to perform vertical processing comprises:

code for causing the electronic device to divide the depth map into segments, wherein at least one segment includes a number of pixels in a column; and

code for causing the electronic device to estimate linear model parameters for at least one segment to determine the vertical non-obstacle estimation;

code for causing the electronic device to perform horizontal processing of the depth map to determine a horizontal non-obstacle estimation;

code for causing the electronic device to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation; and

code for causing the electronic device to generate a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

27. The non-transitory tangible computer-readable medium of claim 26 , wherein the code for causing the electronic device to perform vertical processing further comprises:

code for causing the electronic device to generate a vertical reliability map that includes a reliability value for the vertical non-obstacle estimation.

28. The non-transitory tangible computer-readable medium of claim 26 , wherein the code for causing the electronic device to perform horizontal processing further comprises:

code for causing the electronic device to obtain a depth histogram for at least one row of pixels of the depth map;

code for causing the electronic device to determine a terrain line from the depth histogram;

code for causing the electronic device to determine a horizontal non-obstacle estimation based on a distance of a depth value of at least one pixel from the terrain line; and

code for causing the electronic device to generate a horizontal reliability map that includes a reliability value for the horizontal non-obstacle estimation.

29. The non-transitory tangible computer-readable medium of claim 26 , wherein the code for causing the electronic device to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

code for causing the electronic device to perform both the vertical processing and the horizontal processing in parallel; and

code for causing the electronic device to merge the vertical non-obstacle estimation and the horizontal non-obstacle estimation based on a vertical reliability map and a horizontal reliability map.

30. The non-transitory tangible computer-readable medium of claim 26 , wherein the code for causing the electronic device to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

code for causing the electronic device to perform vertical processing of the depth map;

code for causing the electronic device to obtain a vertical reliability map based on a model fitting confidence;

code for causing the electronic device to identify reliable regions of the vertical reliability map as non-obstacle areas; and

code for causing the electronic device to perform horizontal processing on unreliable regions of the vertical reliability map to determine whether the unreliable regions are non-obstacle areas.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 19, 2015
From: DANE, GOKCE; BHASKARAN, VASUDEV
To: QUALCOMM INCORPORATED
Reel/Frame 037087/0327 →
Continuity (1)
Related Publication 20170084038A1 · Mar 23, 2017