IP Library Granted Patent US 11,070,756
Granted Patent B2
US 11,070,756 · App. 16/250,823 · Granted Jul 20, 2021

Method, device and system for image acquisition control

Inventors: Ming Wang (Beijing, CN); Jie Sun (Beijing, CN)
Assignee: TUSIMPLE, INC.
H04N5/36965B60W40/02G06K9/00684G06K9/20G06K9/3233G06N3/08H04N5/2353H04N5/33B60W2420/403B60W2555/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,070,756
App. No.
16/250,823
Granted
Jul 20, 2021
Kind
B2
Abstract

The present disclosure provides a method and an apparatus for image capturing control and a system for image capturing. The method includes: receiving environment information transmitted from one or more sensors; determining a current environment type based on the environment information; determining whether the current environment type is a predetermined harsh environment type; and controlling one or more Time-of-Flight (TOF) cameras to capture an image when it is determined that the current environment type is the harsh environment type.

Claims (59)

1. A method for image capturing control, comprising:

receiving environment information transmitted from one or more sensors;

extracting, based on the environment information, one or more environmental features;

determining, using at least one preconfigured neural network model, a current environment type based on the one or more environmental features;

determining whether the current environment type is a predetermined harsh environment type; and

capturing an image by controlling one or more Time-of-Flight (TOF) cameras to capture the image when it is determined that the current environment type is the harsh environment type; and

capturing an image by controlling one or more ordinary cameras when it is determined that the current environment type is not the harsh environment type,

wherein the controlling of the one or more TOF cameras includes:

determining a target distance range corresponding to the harsh environment type;

estimating a first time length required for the one or more TOF cameras to receive infrared light reflected by an object at a lower limit distance of the target distance range;

estimating a second time length required for the one or more TOF cameras to receive infrared light reflected by the object at an upper limit distance of the target distance range; and

determining an exposure start time and an exposure end time for the one or more TOF cameras based on the first time length and the second time length.

2. The method of claim 1 , wherein the determining the current environment type based on the one or more environmental features comprises:

matching the one or more environmental features with preconfigured environmental feature libraries for respective environment types, and determining a matched environment type as the current environment type.

3. The method of claim 2 , wherein each of the preconfigured environmental feature libraries is used by a corresponding preconfigured neural network model.

4. The method of claim 1 , wherein controlling the one or more TOF cameras to capture an image comprises:

transmitting parameter control information carrying an infrared light emission time, the exposure start time and the exposure end time to the one or more TOF cameras; and

receiving an image containing an object within the target distance range as captured by the one or more TOF cameras based on the parameter control information.

5. The method of claim 4 , wherein the one or more TOF cameras obtains the image containing the object within the target distance range by filtering out an image outside the target distance range.

6. The method of claim 1 , wherein the harsh environment type comprises one or more of a blizzard environment, a heavy rain environment, a sandstorm environment, a severe haze environment and a low illumination environment.

7. The method of claim 1 , wherein the one or more sensors comprises an air quality detection sensor.

8. The method of claim 7 , wherein the one or more sensors further comprises at least one of a camera, a humidity sensor, a temperature sensor, an illumination sensor, a laser radar, a millimeter wave radar or an infrared sensor.

9. The method of claim 1 , wherein the target distance range includes a first target distance range corresponding to a first harsh environment type and a second target distance range corresponding to a second harsh environment type indicating a harsher environment than the first harsh environment type, and wherein the lower limit distance and the upper limit distance of the second target distance range is shorter than the lower limit distance and the upper limit distance of the first target distance range, respectively.

10. An apparatus for image capturing control, comprising:

a Field-Programmable Gate Array (FPGA) comprising:

an Input Output Block (JOB), and

one or more Configurable Logic Blocks (CLBs),

wherein the IOB and the one or more CLBs are connected via interconnects, and the one or more CLBs are connected via interconnects,

wherein the IOB comprises a receiving unit configured to receive environment information transmitted from one or more sensors, and

wherein the one or more CLBs comprises

a determining unit configured to extract, based on the environment information, one or more environmental features and determine, using at least one preconfigured neural network model, a current environment type based on the one or more environmental features; and

a control unit configured to determine whether the current environment type is a predetermined harsh environment type, and capture an image by controlling one or more Time-of-Flight (TOF) cameras to capture the image when it is determined that the current environment type is the harsh environment type, and capture an image by controlling one or more ordinary cameras when it is determined that the current environment type is not the harsh environment type,

wherein the controlling of the one or more TOF cameras includes:

determining a target distance range corresponding to the harsh environment type;

estimating a first time length required for the one or more TOF cameras to receive infrared light reflected by an object at a lower limit distance of the target distance range;

estimating a second time length required for the one or more TOF cameras to receive infrared light reflected by the object at an upper limit distance of the target distance range; and

determining an exposure start time and an exposure end time for the one or more TOF cameras based on the first time length and the second time length.

11. The apparatus of claim 10 , wherein the determining unit is further configured to:

match the one or more environmental features with preconfigured environmental feature libraries for respective environment types, and

determine a matched environment type as the current environment type.

12. The apparatus of claim 10 , wherein the control unit is further configured, as part of controlling the one or more TOF cameras to capture an image, to:

transmit parameter control information carrying an infrared light emission time, the exposure start time and the exposure end time to the one or more TOF cameras; and

receive an image containing an object within the target distance range as captured by the one or more TOF cameras based on the parameter control information.

13. The apparatus of claim 10 , wherein the harsh environment type comprises one or more of a blizzard environment, a heavy rain environment, a sandstorm environment, a severe haze environment and a low illumination environment.

14. An image capturing system, comprising one or more ordinary cameras, one or more Time-of-Flight (TOF) cameras and an apparatus for image capturing control, wherein the apparatus for image capturing control is configured to:

receive environment information transmitted from one or more sensors;

extract, based on the environment information, one or more environmental features;

determine, using at least one preconfigured neural network model, a current environment type based on the one or more environmental features;

determine whether the current environment type is a predetermined harsh environment type; and

capture an image by controlling the one or more TOF cameras to capture the image when it is determined that the current environment type is the harsh environment type, and capture an image by controlling the one or more ordinary cameras to capture the image when it is determined that the current environment type is not the harsh environment type,

the one or more ordinary cameras are configured to capture an image under control of the apparatus for image capturing control, and

the one or more TOF cameras are configured to capture an image under control of the apparatus for image capturing control by:

determining a target distance range corresponding to the harsh environment type;

estimating a first time length required for the one or more TOF cameras to receive infrared light reflected by an object at a lower limit distance of the target distance range;

estimating a second time length required for the one or more TOF cameras to receive infrared light reflected by the object at an upper limit distance of the target distance range; and

determining an exposure start time and an exposure end time for the one or more TOF cameras based on the first time length and the second time length.

15. The system of claim 14 , wherein the apparatus for image capturing control is configured to control the one or more TOF cameras to capture an image by:

transmitting parameter control information carrying an infrared light emission time, the exposure start time and the exposure end time to the one or more TOF cameras; and

receiving an image containing an object within the target distance range as captured by the one or more TOF cameras based on the parameter control information.

Assignments (5)
CHANGE OF NAME Recorded Dec 9, 2025
From: TUSIMPLE, INC.
To: CREATEAI, INC.
Reel/Frame 073158/0715 →
CHANGE OF NAME Recorded Dec 9, 2025
From: BEIJING TUSEN ZHITU TECHNOLOGY CO., LTD.
To: BEIJING OCGEN TECHNOLOGY CO., LTD.
Reel/Frame 073916/0416 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2022
From: TUSIMPLE, INC.
To: BEIJING TUSEN ZHITU TECHNOLOGY CO., LTD.
Reel/Frame 058779/0374 →
CHANGE OF NAME Recorded Oct 30, 2020
From: TUSIMPLE
To: TUSIMPLE, INC.
Reel/Frame 054261/0596 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2020
From: SUN, JIE; WANG, MING
To: TUSIMPLE
Reel/Frame 053033/0941 →
Priority Claims (1)
CN 201810066893.8 · Jan 24, 2018 · national
Continuity (1)
Related Publication 20190230303A1 · Jul 25, 2019