IP Library Granted Patent US 10,339,400
Granted Patent B1
US 10,339,400 · App. 16/223,114 · Granted Jul 2, 2019

Traffic light detection using multiple cameras

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,339,400
App. No.
16/223,114
Granted
Jul 2, 2019
Kind
B1
Abstract

Various arrangements for imaging a traffic light are presented. A distance to the traffic light from a vehicle may be determined. A camera mode may be selected based on the determined distance to the traffic light. One or more images from one or more cameras may be captured and received based on the selected camera mode. A state of the traffic light within the one or more received images may then be determined. The vehicle may then be driven autonomously based on the determined state of the traffic light.

Claims (58)

1. A method for imaging a traffic light, the method comprising:

determining, by an on-vehicle processing system, a distance to the traffic light, wherein the on-vehicle processing system is installed on a vehicle;

selecting, by the on-vehicle processing system, a camera mode based on the determined distance to the traffic light;

receiving, by the on-vehicle processing system, one or more images from one or more cameras based on the selected camera mode;

determining, by the on-vehicle processing system, a state of the traffic light within the one or more received images; and

driving, by the on-vehicle processing system, the vehicle based on the determined state of the traffic light.

2. The method for imaging the traffic light of claim 1 , the method further comprising:

causing, by the on-vehicle processing system, a first camera to be supplied with power based on the selected camera mode; and

causing, by the on-vehicle processing system, a second camera to be disconnected from power.

3. The method for imaging the traffic light of claim 2 , wherein the camera mode is selected from the group consisting of:

a first camera mode in which a short-range camera is used as the first camera to capture the one or more images; and

a second camera mode in which a long-range camera is used as the first camera to capture the one or more images.

4. The method for imaging the traffic light of claim 3 , wherein selecting the camera mode based on the determined distance to the traffic light is based on comparing the determined distance to a stored threshold distance.

5. The method for imaging the traffic light of claim 4 , wherein the long-range camera is a thirty degree field-of-view camera and the short-range camera is a sixty degree field-of-view camera.

6. The method for imaging the traffic light of claim 1 , wherein the camera mode is selected from the group consisting of:

a first camera mode in which a short-range camera is used to capture the one or more images;

a second camera mode in which a long-range camera is used to capture the one or more images; and

a third camera mode in which the long-range camera and the short-range camera are used to capture the one or more images.

7. The method for imaging the traffic light of claim 6 , wherein selecting the camera mode based on the determined distance to the traffic light is based on comparing the determined distance to a first stored threshold distance and a second stored threshold distance.

8. The method for imaging the traffic light of claim 7 , wherein:

the third camera mode is selected when the determined distance is between the first stored threshold distance and the second stored threshold distance; and

the determined distance is between the first stored threshold distance and the second stored threshold distance.

9. The method for imaging the traffic light of claim 8 , further comprising:

based on the third camera mode being selected, rescaling, by the on-vehicle processing system, a first image of the one or more images captured using the short-range camera to match a region included in a second image of the one or more images captured using the long-range camera.

10. The method for imaging the traffic light of claim 9 , wherein determining the state of the traffic light within the one or more images comprises:

analyzing the rescaled first image of the one or more images to determine the state of the traffic light; and

analyzing the second image of the one or more images to determine the state of the traffic light.

11. The method for imaging the traffic light of claim 10 , wherein determining the state of the traffic light within the one or more images further comprises:

performing, by the on-vehicle processing system, a fusion process based on the state of the traffic light based on analyzing the rescaled first image and the state of the traffic light based on analyzing the second image.

12. A system for imaging a traffic light, the system comprising:

a short-range camera having a first field-of-view, wherein the short-range camera is installed on a vehicle;

a long-range camera having a second field-of-view that is narrower than the first field-of-view, wherein the long-range camera is installed on the vehicle and the first field-of-view overlaps the second field-of-view; and

an on-vehicle processing system, comprising one or more processors, that receives images from the short-range camera and the long-range camera, wherein the on-vehicle processing system is configured to:

determine a distance to the traffic light, wherein the on-vehicle processing system is installed on the vehicle;

select a camera mode based on the determined distance to the traffic light;

receive one or more images from the long-range camera, the short-range camera, or both based on the selected camera mode;

determine a state of the traffic light within the one or more received images; and

cause the vehicle to be driven based on the determined state of the traffic light.

13. The system for imaging the traffic light of claim 12 , wherein the on-vehicle processing system is further configured to:

cause the short-range camera to be supplied with power based on the selected camera mode; and

cause the long-range camera to be disconnected from power.

14. The system for imaging the traffic light of claim 13 , wherein the camera mode is selected from the group consisting of:

a first camera mode in which the long-range camera is used to capture the one or more images; and

a second camera mode in which the short-range camera is used to capture the one or more images.

15. The system for imaging the traffic light of claim 14 , wherein selecting the camera mode based on the determined distance to the traffic light is based on comparing the determined distance to a stored threshold distance.

16. The system for imaging the traffic light of claim 15 , wherein the long-range camera is a thirty degree field-of-view camera and the short-range camera is a sixty degree field-of-view camera.

17. The system for imaging the traffic light of claim 12 , wherein the camera mode is selected from the group consisting of:

a first camera mode in which the long-range camera is used to capture the one or more images;

a second camera mode in which the short-range camera is used to capture the one or more images; and

a third camera mode in which the long-range camera and the short-range camera are used to capture the one or more images.

18. The system for imaging the traffic light of claim 17 , wherein selecting the camera mode based on the determined distance to the traffic light is based on the on-vehicle processing system being configured to compare the determined distance to a first stored threshold distance and a second stored threshold distance.

19. The system for imaging the traffic light of claim 18 , wherein:

the third camera mode is selected when the determined distance is between the first stored threshold distance and the second stored threshold distance; and

the determined distance is between the first stored threshold distance and the second stored threshold distance.

20. The system for imaging the traffic light of claim 19 , wherein the on-vehicle processing system is further configured to:

rescale a first image of the one or more images captured using the short-range camera to match a region included in a second image of the one or more images captured using the long-range camera based on the third camera mode being selected, wherein the on-vehicle processing system being configured to determine the state of the traffic light within the one or more images comprises the on-vehicle processing system being configured to:

analyze the rescaled first image of the one or more images to determine the state of the traffic light; and

analyze the second image of the one or more images to determine the state of the traffic light.

Assignments (3)
CHANGE OF NAME Recorded Dec 3, 2024
From: CHONGQING JINKANG SERES NEW ENERGY VEHICLE RESEARCH INSTITUTE CO., LTD.
To: SERES AUTOMOBILE CO., LTD
Reel/Frame 069476/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 8, 2021
From: CHONGQING JINKANG NEW ENERGY AUTOMOBILE CO., LTD.; SF MOTORS, INC.
To: CHONGQING JINKANG SERES NEW ENERGY VEHICLE RESEARCH INSTITUTE CO., LTD.
Reel/Frame 057738/0590 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2018
From: CHAI, HUAJUN; CUI, CHEN; HUANG, KE; WANG, FAN; TANG, YIFAN
To: CHONGQING JINKANG NEW ENERGY AUTOMOBILE CO., LTD.; SF MOTORS INC.
Reel/Frame 047800/0911 →
Cited By (7)
US 12,221,127 US 12,412,402 US 12,423,992 US 12,434,703 US 12,434,706 US 12,525,030 US 12,573,209