IP Library Granted Patent US 10,591,601
Granted Patent B2
US 10,591,601 · App. 16/031,815 · Granted Mar 17, 2020

Camera-gated lidar system

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,591,601
App. No.
16/031,815
Filed
Jul 10, 2018
Granted
Mar 17, 2020
Kind
B2
Art Unit
3645
USPC
356/4.01
Abstract

A machine vision system comprises a camera configured to generate one or more images of a field of regard of the camera, a lidar system, and a processor. The lidar system includes a laser configured to emit light, where the emitted light is directed toward a region within the field of regard of the camera and a receiver configured to detect light returned from the emitted light. The processor is configured to receive an indication of a location based on the returned light and determine whether a solid object is present at the location based on the one or more images.

Claims (103)

1. A machine vision system comprising:

a camera configured to generate one or more images of a field of regard (FOR) of the camera;

a lidar system including:

a laser configured to emit light, wherein the lidar system directs the emitted light toward a region within the FOR of the camera, and

a receiver configured to detect light returned from the emitted light; and

a processor configured to:

receive an indication of a location based on the returned light,

determine whether a solid object is present at the location based on the one or more images by detecting an absence of targets along a path of the emitted light within a maximum range of the lidar system, using the one or more images, and

discard the indication of the location based on the detection of the absence of targets.

2. The machine vision system of claim 1 , wherein:

the indication of the location is one of a plurality of indications of location received by the processor, wherein the indications of location are based on the returned light; and

the processor is further configured to:

detect an obscurant in a path of the emitted light, using the one or more images generated by the camera.

3. The machine vision system of claim 2 , wherein the obscurant includes one of rain, fog, haze, dust, steam, or car exhaust.

4. The machine vision system of claim 2 , wherein the processor is further configured to:

discard all but a last one of the indications of location in response to detecting the obscurant; and

associate the last one of the indications of location with the solid object.

5. The machine vision system of claim 1 , wherein the emitted light is an emitted pulse of light, and wherein the processor is further configured to:

detect a shape of a pulse corresponding to the indication of the location,

classify the pulse based on whether the shape corresponds to a result of scattering by (i) rain or (ii) one of fog, haze, or dust, and

discard the indication of the location as a result of the classification along with the one or more images generated by the camera.

6. The machine vision system of claim 5 , wherein the processor is further configured to:

determine that the returned light comprises an extended return, shaped as a pulse extended over a period of time greater than a threshold value,

detect a likely target return, shaped as a short pulse superimposed on the extended return, and

in response to detecting the likely target return, select the likely target return to generate a lidar pixel value.

7. The machine vision system of claim 1 , wherein the indication of the location is a pixel comprising distance information.

8. The machine vision system of claim 1 , wherein the emitted light is an emitted pulse of light, and wherein the indication of the location is one of a plurality of indications of location received by the processor, each of the indications of location based on a respective pulse of returned light; and

wherein the processor is further configured to:

determine a type of the solid object,

determine, based on the determined type, an expected shape of a pulse of returned light, and

select, from among the pulses of returned light, a pulse with a shape most proximate to the expected shape to generate a lidar pixel value.

9. The machine vision system of claim 1 , wherein the emitted light is an emitted pulse of light, and wherein the indication of the location is one of a plurality of indications of location received by the processor, each of the indications of location based on a respective pulse of returned light; and

wherein the processor is further configured to:

detect an approximate distance to the solid object using the one or more images generated by the camera, and

select, from among the pulses of returned light, a pulse based on the detected approximate distance to generate a lidar pixel value.

10. A method for operating a machine vision system, the method comprising:

receiving, from a camera, data descriptive of a scene within a field of regard (FOR) of the camera;

emitting, by a laser operating in a lidar system, light;

directing the emitted light toward a region within the FOR of the camera;

detecting light returned from the emitted light;

receiving an indication of a location based on the returned light;

determining, by a processor, whether a solid object is present at the location, based on the data descriptive of the scene, including detecting an object in a path of the emitted light, at an approximate distance greater than a maximum range of the lidar system, using the data descriptive of the scene, wherein the object scatters the light corresponding to the indication of the location; and

discarding the indication of the location in view of the object.

11. The method of claim 10 , further comprising:

receiving a plurality of indications of location, wherein each of the indications of location is based on the returned light; and

detecting an obscurant in a path of the emitted light, using the data descriptive of the scene.

12. The method of claim 10 , further comprising:

receiving a plurality of indications of location, wherein each of the indications of location is based on the returned light;

estimating, using the data descriptive of the scene, a set of approximate distances to a set of targets; and

selecting the indications of locations that correspond to a set of the approximate distances to generate one or more lidar pixel values.

13. The method of claim 10 , further comprising:

receiving a plurality of indications of location, each of the indications of location based on a respective pulse of returned light;

for each of the pulses of returned light:

determining a shape of the pulse; and

determining whether the determine shape corresponds to a result of scattering by (i) rain or (ii) one of fog, haze, or dust, to generate pulse classification, and

selecting one or more of the indications of location, using the generated pulse classifications along with the data descriptive of the scene received from the camera, to generate one or more lidar pixel values.

14. A machine vision system comprising:

a camera configured to generate one or more images of a field of regard (FOR) of the camera;

a lidar system including:

a laser configured to emit light, wherein the lidar system directs the emitted light toward a region within the FOR of the camera, and

a receiver configured to detect light returned from the emitted light; and

a processor configured to:

receive an indication of a location based on the returned light,

determine whether a solid object is present at the location based on the one or more images by detecting an absence of targets along a path of the emitted light within a certain range of the lidar system, using the one or more images;

detect an object in a path of the emitted light, at a distance greater than a maximum range of the lidar system, using the one or more images, and

discard the indication of the location in response to detecting the absence of targets within the certain range and detecting the object at a distance greater than the maximum range.

15. A machine vision system comprising:

a camera configured to generate one or more images of a field of regard (FOR) of the camera;

a lidar system including:

a laser configured to emit light, wherein the lidar system directs the emitted light toward a region within the FOR of the camera, and

a receiver configured to detect light returned from the emitted light; and

a processor configured to:

receive an indication of a location based on the returned light,

determine whether a solid object is present at the location based on the one or more images by detecting an absence of targets along a path of the emitted light within a certain range of the lidar system, using the one or more images,

detect an object in a path of the emitted light, at a distance greater than a maximum range of the lidar system, using the one or more images, and

generate a pixel that corresponds to the range of the object in response to detecting the absence of targets within the certain range and detecting the object at a distance greater than the maximum range.

16. A machine vision system comprising:

a camera configured to generate one or more images of a field of regard (FOR) of the camera;

a lidar system including:

a laser configured to emit a pulse of light, wherein the lidar system directs the emitted light toward a region within the FOR of the camera, and

a receiver configured to detect light returned from the emitted light; and

a processor configured to:

receive an indication of a location based on the returned light, wherein the indication of the location is one of a plurality of indications of location received by the processor, and each of the indications of location is based on a respective pulse of returned light,

select, from among the pulses of returned light, a pulse to generate a lidar pixel value,

assign a confidence tag to the generated lidar pixel value in view of a shape of the selected pulse, and

determine whether a solid object is present at the location based on the one or more images.

17. A method for operating a machine vision system, the method comprising:

receiving, from a camera, data descriptive of a scene within a field of regard (FOR) of the camera;

emitting, by a laser operating in a lidar system, light;

directing the emitted light toward a region within the FOR of the camera;

detecting light returned from the emitted light;

receiving an indication of a location based on the returned light;

determining, by a processor, whether a solid object is present at the location, based on the data descriptive of the scene, including detecting an object in a path of the emitted light, at an approximate distance greater than a maximum range of the lidar system, using the data descriptive of the scene, wherein the object scatters the light corresponding to the indication of the location; and

generating a pixel that corresponds to the range of the object in response to detecting the object at a distance greater than the maximum range.

18. A method for operating a machine vision system, the method comprising:

receiving, from a camera, data descriptive of a scene within a field of regard (FOR) of the camera;

emitting, by a laser operating in a lidar system, light;

directing the emitted light toward a region within the FOR of the camera;

detecting light returned from the emitted light;

receiving an indication of a location based on the returned light;

detecting, using the data descriptive of the scene, a presence of a specularly reflective surface in the path of the light;

determining, by a processor, whether a solid object is present at the location, based on the data descriptive of the scene; and

discarding the indication of the location in view of a multipath return due to the specularly reflective surface.