Granted Patent B2

US 9,990,548 · App. 15/065,681 · Granted Jun 5, 2018

Abstract

Claims (52)

1. A traffic signal analysis system for an autonomous vehicle, the traffic signal analysis system comprising:

one or more cameras;

one or more processors; and

one or more memory resources storing signal analysis instructions that, when executed by the one or more processors, cause the one or more processors to:

receive image data from the one or more cameras, the image data including an upcoming traffic signaling system located at an intersection;

determine a pass-through action for the autonomous vehicle through the intersection;

determine a pose of the autonomous vehicle, the pose indicating a current position and orientation of the autonomous vehicle within a given region;

based on the pose of the autonomous vehicle, perform a lookup in a catalog of recorded and labeled signal maps to access a matching signal map, the matching signal map being specific to the upcoming traffic signaling system located at the intersection and comprising labeled data enabling the autonomous vehicle to determine properties specific to the upcoming traffic signaling system;

based on the labeled data, (i) generate a signal template for the upcoming traffic signaling system on the image data, the signal template defining at least one region of interest that comprises a plurality of traffic signal faces of the upcoming traffic signaling system, and (ii) determine a primary subset and a secondary subset of the plurality of traffic signal faces that apply to the pass-through action;

dynamically analyze the primary subset and the secondary subset to determine a state of the upcoming traffic signaling system for the pass-through action; and

generate an output for the autonomous vehicle indicating the state of the upcoming traffic signaling system for the pass-through action.

2. The traffic signal analysis system of claim 1 , wherein the executed signal analysis instructions cause the one or more processors to dynamically analyze the primary and secondary subsets by performing a probabilistic matching operation for one or more portions of the at least one region of interest to determine a state of the primary and secondary subsets for each traffic light face in the primary and secondary subsets.

3. The traffic signal analysis system of claim 2 , wherein the primary and secondary subsets comprise more than one traffic signal face that applies to the pass-through action, and wherein the executed signal analysis instructions further cause the one or more processors to:

when one of the more than one traffic signal face is at least partially occluded, prioritize a remainder of the more than one traffic signal face in the primary and secondary subsets, during the probabilistic matching operation, to determine the state of the upcoming traffic signaling system for the pass-through action.

4. The traffic signal analysis system of claim 1 , wherein the executed signal analysis instructions further cause the one or more processors to:

determine a plurality of pass-through options for the plurality of traffic signal faces;

wherein the generated signal template comprises a signal subset for each of the plurality of pass-through options.

5. The traffic signal analysis system of claim 4 , wherein the plurality of pass-through options comprises two or more of a left turn option, a right turn option, a U-turn option, or a straight-through option.

6. The traffic signal analysis system of claim 1 , wherein the one or more cameras comprise a dedicated signal camera specific for identifying traffic signals.

7. The traffic signal analysis system of claim 6 , wherein the dedicated signal camera comprises a set of camera settings specifically configured to detect traffic signals, the set of camera settings being different from camera settings of a set of perception cameras utilized by a control system of the autonomous vehicle to detect potential obstacles or road hazards.

8. The traffic signal analysis system of claim 1 , wherein the generated output for the autonomous vehicle, indicating the state of the upcoming traffic signaling system for the pass-through action, comprises (i) a command instructing a control system of the autonomous vehicle to stop at the intersection, (ii) an indication to the control system that the autonomous vehicle has a right of way through the intersection, or (iii) an indication to the control system to yield to right of way vehicles before passing through the intersection.

9. The traffic signal analysis system of claim 1 , wherein the executed signal analysis instructions further cause the one or more processors to:

predict the state of the upcoming traffic signaling system for the pass-through action when the autonomous vehicle reaches the intersection;

wherein the generated output comprises the predicted state of the upcoming traffic signaling system for the pass-through action.

10. An autonomous vehicle comprising:

a sensor system to dynamically generate sensor data indicating a situational environment of the autonomous vehicle;

acceleration, braking, and steering systems; and

a control system comprising one or more processing resources executing instructions that cause the control system to:

process the sensor data to autonomously operate the acceleration, braking, and steering systems of the autonomous vehicle;

receive image data from the sensor system, the image data including an upcoming traffic signaling system located at an intersection;

determine a pass-through action for the autonomous vehicle through the intersection;

determine a pose of the autonomous vehicle, the pose indicating a current position and orientation of the autonomous vehicle within a given region;

based on the pose of the autonomous vehicle, perform a lookup in a catalog of recorded and labeled signal maps to access a matching signal map, the matching signal map being specific to the upcoming traffic signaling system located at the intersection and comprising labeled data enabling the autonomous vehicle to determine properties specific to the upcoming traffic signaling system;

based on the labeled data, (i) generate a signal template for the upcoming traffic signaling system on the image data, the signal template defining at least one region of interest that comprises a plurality of traffic signal faces of the upcoming traffic signaling system, and (ii) determine a primary subset and a secondary subset of the plurality of traffic signal faces that apply to the pass-through action;

dynamically analyze the primary subset and the secondary subset to determine a state of the upcoming traffic signaling system for the pass-through action; and

execute a corresponding action based on the state of the upcoming traffic signaling system for the pass-through action.

11. The autonomous vehicle of claim 10 , wherein the executed instructions cause the control system to analyze the primary and secondary subsets by performing a probabilistic matching operation for one or more portions of the at least one region of interest to determine a state of the primary and secondary subsets for each traffic light face in the primary and secondary subsets.

12. The autonomous vehicle of claim 11 , wherein the primary and secondary subsets comprise more than one traffic signal face that applies to the pass-through action, and wherein the executed instructions further cause the control system to:

when one of the more than one traffic signal face is at least partially occluded, prioritize a remainder of the more than one traffic signal face in the primary and secondary subsets, during the probabilistic matching operation, to determine the state for the pass-through action.

13. The autonomous vehicle of claim 10 , wherein the at least one region of interest comprises a plurality of traffic signal faces, and wherein the executed instructions further cause the control system to:

determine a plurality of pass-through options for the plurality of traffic signal faces;

wherein the generated signal template comprises a signal subset for each of the plurality of pass-through options.

14. The autonomous vehicle of claim 13 , wherein the plurality of pass-through options comprises two or more of a left turn option, a right turn option, a U-turn option, or a straight-through option.

15. The autonomous vehicle of claim 10 , wherein the one or more cameras comprise a dedicated signal camera specific for identifying traffic signals.

16. A non-transitory computer readable medium storing signal analysis instructions that when executed by one or more processors of an autonomous vehicle, cause the one or more processors to:

receive image data from one or more cameras of the autonomous vehicle, the image data including an upcoming traffic signaling system located at an intersection;

determine a pass-through action for the autonomous vehicle through the intersection;

determine a pose of the autonomous vehicle, the pose indicating a current position and orientation of the autonomous vehicle within a given region;

based on the pose of the autonomous vehicle, perform a lookup in a catalog of recorded and labeled signal maps to access a matching signal map, the matching signal map being specific to the upcoming traffic signaling system located at the intersection and comprising labeled data enabling the autonomous vehicle to determine properties specific to the upcoming traffic signaling system;

based on the labeled data, (i) generate a signal template for the upcoming traffic signaling system on the image data, the signal template defining at least one region of interest that comprises a plurality of traffic signal faces of the upcoming traffic signaling system, and (ii) determine a primary subset and a secondary subset of the plurality of traffic signal faces that apply to the pass-through action;

dynamically analyze the primary subset and the secondary subset to determine a state of the upcoming traffic signaling system for the pass-through action; and

generate an output for the autonomous vehicle indicating the state of the upcoming traffic signaling system for the pass-through action.

Assignments (6)