High dynamic range imaging using external light source
Disclosed herein are devices, methods, and systems for providing an externally augmented camera that may utilize external light sources of a separate sensor to emit light toward a scene so as to provide accurate imaging of the scene, even in dark or low-light situations or where the scene has a high dynamic range of brightness. The externally augmented camera system may include a sensor with a light source capable of emitting light toward a scene, a camera separate from the light source, where the camera includes a detector capable of detecting emitted light from the light source. The externally augmented camera system also causes the sensor to emit light toward the scene via the light source, causes the camera to capture image data of the scene that has been illuminated by emitted light from the light source, and generates an image of the scene based on the image data.
1 . A system comprising:
a sensor comprising a light source capable of emitting light toward a scene;
a camera separate from the sensor, wherein the camera comprises a detector capable of detecting emitted light from the light source; and
a processor configured to:
cause the sensor to emit light toward the scene via the light source;
cause the camera to capture image data of the scene that has been illuminated by emitted light from the light source, wherein the image data comprises a series of image data frames of the scene, each frame representing a portion of the scene; and
combine the series of image data frames to generate an image of the scene based on the image data.
2 . The system of claim 1 , wherein the detector is configured to capture the image data at different points in time, wherein for each point in time, the detector is configured to detect, as the image data, emitted light reflected by the scene at varying intensities as a function of a distance between an incident location of the emitted light on the scene and a reflected location within the scene of the light reflected by the scene.
3 . The system of claim 1 , wherein the sensor comprises at least one of a light detection and ranging sensor comprising a laser as the light source, an infrared (IR) sensor comprising a light-emitting diode as the light source, or a radar sensor comprising a radar transmitter as the light source.
4 . The system of claim 1 , wherein the camera comprises a high-speed camera, an event camera, a filtering camera comprising a switchable optical filter, a red-green-blue (RGB) camera, or an IR camera.
5 . The system of claim 1 , wherein the image data of the scene comprises partial image data of a subregion of the scene that is at least partially illuminated by the emitted light.
6 . The system of claim 1 , wherein the processor is further configured to, in order to combine the series of image data frames, stitch together the image data frames or perform a high dynamic range processing of at least two of the image data frames.
7 . The system of claim 6 , wherein each frame of the at least two image frames comprises a different brightness of the scene with respect to each other of the at least two image frames.
8 . The system of claim 6 , wherein the series of image data frames comprise a first frame and a second frame of a portion of the scene, wherein the first frame comprises a first illumination of the portion of the scene, wherein the second frame comprises a second illumination of the portion of the scene, wherein the first illumination is different from the second illumination.
9 . The system of claim 6 , the system further comprising a plurality of cameras and a corresponding plurality of detectors, wherein the camera is one of the plurality of cameras and the detector is one of the corresponding plurality of detectors, wherein the processor is further configured to cause the plurality of cameras and the corresponding plurality of detectors to, in an alternating succession to one other, capture the series of image data frames.
10 . The system of claim 9 , wherein the alternating succession comprises a consecutive capture of each frame in the series of image data frames by a different camera of the plurality of cameras.
11 . The system of claim 1 , wherein the processor is further configured to cause the detector to capture the image data at a sensing time that is based on an emission time of the emitted light from the sensor.
12 . The system of claim 1 , wherein the processor is further configured to, in order to cause the sensor to emit light from the light source toward the scene:
cause a series of light pulses to be emitted as the emitted light; or
cause a continuous light source to be emitted as the emitted light, wherein the continuous light source comprises light emissions that vary over time by different frequencies or by different amplitudes.
13 . The system of claim 1 , wherein the processor is further configured to adjust an output intensity of the emitted light based on a distance to the scene or based on a previously generated image of the scene.
14 . The system of claim 13 , wherein the processor is further configured to, in order to adjust the output intensity of the emitted light based on the previously generated image of the scene, adjust the output intensity of the emitted light based on an exposure level of the previously generated image.
15 . The system of claim 1 , wherein the sensor is configured to emit, as the emitted light, a narrow beam of laser light that is substantially narrower than a dimension of the scene.
16 . A non-transitory computer readable medium that comprises instructions which, if executed, cause one or more processors to:
cause a sensor comprising a light source to emit light toward a scene in an environment of a vehicle;
cause a camera to capture a series of image data frames at different times of the scene that has been illuminated by emitted light from the light source, wherein the camera is separate from the sensor, wherein the camera comprises a detector capable of detecting emitted light from the light source; and
combine the series of image data frames for one time of the different times to generate an image of the scene based on the image data.
17 . The non-transitory computer readable medium of claim 16 , wherein the instructions also cause the one or more processors to focus the detector on a subregion of the scene based on a direction of the emitted light from the sensor.