IP Library Granted Patent US 11,462,025
Granted Patent B2
US 11,462,025 · App. 17/035,103 · Granted Oct 4, 2022

Method of and system for determining traffic signal state

Inventors: Aleksey Sergeevich Artamonov (p Metallost, RU); Dmitry Alekseevich Kalyuzhny (Saint Petersburg, RU); Yuliya Alekseevna Yakovleva (Chelyabinsk, RU)
Assignee: YANDEX SELF DRIVING GROUP LLC
G06V20/584G06K9/6227G06K9/6257G06N20/20G06V20/46G06V20/54
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,462,025
App. No.
17/035,103
Granted
Oct 4, 2022
Kind
B2
Abstract

There is disclosed a method and system for determining a predicted state of a traffic signal. A video of the traffic signal is received. Still images of the traffic signal are generated based on the video. A first machine learning algorithm (MLA) outputs a vector for each bulb in each still image, the vector indicating a predicted status of the bulb. A second MLA determines a predicted state of the traffic signal based on the vectors.

Claims (64)

1. A method for determining a predicted state of a traffic signal, the method executable on a server, the method comprising:

receiving, by the server, a video of the traffic signal;

generating, based on the video, a plurality of still images of the traffic signal;

determining, for each still image of the plurality of still images, and by a first Machine Learning Algorithm (MLA), a vector for each bulb in the respective still image, thereby generating a plurality of vectors corresponding to the video,

each vector indicating a predicted likelihood of each of a plurality of statuses of the respective bulb,

the first MLA having been trained based on a plurality of labeled still images of traffic signals, and

each labeled still image of the plurality of labeled still images indicating a status of each bulb of the respective labeled still image;

determining, based on the plurality of vectors, and by a second MLA, the predicted state of the traffic signal,

the second MLA having been trained based on a plurality of labeled vectors, wherein each labeled vector is indicative of a predicted likelihood of each of the plurality of statuses for a given bulb of a given traffic signal,

wherein each labeled vector is associated with a label indicating an actual state of the given traffic signal,

wherein the second MLA outputs an indicator of confidence for each state, and

wherein the determining the predicted state of the traffic signal comprises comparing the indicators of confidence for each state; and

storing an indicator of the predicted state of the traffic signal.

2. The method of claim 1 , wherein the plurality of statuses comprise a pre-defined set of statuses.

3. The method of claim 1 , wherein a first status of the plurality of statuses corresponds to a first color and a second status of the plurality of statuses corresponds to a second color.

4. The method of claim 3 , wherein a third status of the plurality of statuses indicates whether the bulb is lit.

5. The method of claim 1 , wherein the predicted state of the traffic signal indicates a predicted color displayed by the traffic signal.

6. The method of claim 1 , wherein the predicted state of the traffic signal indicates a predicted symbol displayed by the traffic signal.

7. The method of claim 1 , wherein the predicted state of the traffic signal indicates a predicted pattern displayed by the traffic signal.

8. The method of claim 1 , wherein the plurality of labeled vectors were generated using the first MLA.

9. The method of claim 1 , further comprising:

storing the plurality of vectors in a matrix; and

inputting the matrix to the second MLA.

10. The method of claim 1 , further comprising determining, based on stored location data of traffic signals, coordinates of the traffic signal in the video.

11. The method of claim 1 , wherein the first MLA predicts a color displayed by each bulb in an input still image.

12. A method for determining a predicted state of a traffic signal, the method executable on a server, the method comprising:

receiving, by the server, a video of a plurality of traffic signals, wherein the plurality of traffic signals comprises the traffic signal;

generating, based on the video, a plurality of still images of the plurality of traffic signals;

determining, for each still image of the plurality of still images, and by a first Machine Learning Algorithm (MLA), a vector for each bulb in the respective still image, thereby generating a plurality of vectors corresponding to the video,

each vector indicating a predicted likelihood of each of a plurality of statuses of the respective bulb,

the first MLA having been trained based on a plurality of labeled still images of traffic signals, and

each labeled still image of the plurality of labeled still images indicating a status of each bulb of the respective labeled still image;

determining, based on the plurality of vectors, and by a second MLA, a predicted state of each of the plurality of traffic signals,

the second MLA having been trained based on a plurality of labeled vectors, wherein each labeled vector indicating a predicted likelihood of each of the plurality of statuses for a given bulb of a given traffic signal,

wherein each labeled vector is associated with a label indicating an actual state of the given traffic signal,

wherein the second MLA outputs an indicator of confidence for each state, and wherein the determining the predicted state for each of the plurality of traffic signals comprises comparing the indicators of confidence for each state of a respective traffic signal;

determining, based on the predicted state of each of the plurality of traffic signals, the predicted state of the traffic signal; and

storing an indicator of the predicted state of the traffic signal.

13. The method of claim 12 , wherein determining the predicted state of the traffic signal comprises comparing the predicted state of each of the plurality of traffic signals.

14. The method of claim 12 , wherein the plurality of labeled vectors were generated using the first MLA.

15. The method of claim 12 , wherein generating the plurality of still images comprising capturing, at a pre-defined interval, frames of the video.

16. The method of claim 12 , further comprising:

storing the plurality of vectors in a matrix; and

inputting the matrix to the second MLA.

17. The method of claim 12 , further comprising determining, based on stored location data of traffic signals, coordinates of each of the traffic signals in the video.

18. A system for determining a predicted state of a traffic signal, the system comprising:

a processor; and

a non-transitory computer-readable medium comprising instructions,

the processor, upon executing the instructions, being configured to:

receive a video of the traffic signal;

generate, based on the video, a plurality of still images of the traffic signal;

determine, for each still image of the plurality of still images, and by a first Machine Learning Algorithm (MLA), a vector for each bulb in the respective still image, thereby generating a plurality of vectors corresponding to the video,

wherein each vector indicates a predicted likelihood of each of a plurality of statuses of the respective bulb,

wherein the first MLA is trained based on a plurality of labeled still images of traffic signals, and

wherein each labeled still image of the plurality of labeled still images indicates a status of each bulb of the respective labeled still image;

store the plurality of vectors in a matrix;

input the matrix to a second MLA;

determine, based on the plurality of vectors, and by the second MLA, the predicted state of the traffic signal,

wherein the second MLA is trained based on a plurality of labeled vectors,

wherein each labeled vector indicates a predicted likelihood of each of the plurality of statuses for a bulb of a traffic signal, and

wherein each labeled vector is associated with a label indicating a state of the traffic signal; and

store an indicator of the predicted state of the traffic signal.

19. The system of claim 18 , wherein the second MLA outputs an indicator of confidence for each state.

20. The system of claim 19 , wherein the instructions that cause the processor to determine the predicted state of the traffic signal comprise instructions that cause the processor to select a state having a highest confidence.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2024
From: DIRECT CURSUS TECHNOLOGY L.L.C
To: Y.E. HUB ARMENIA LLC
Reel/Frame 068534/0619 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 2, 2023
From: YANDEX SELF DRIVING GROUP LLC
To: DIRECT CURSUS TECHNOLOGY L.L.C
Reel/Frame 065447/0048 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2020
From: ARTAMONOV, ALEKSEY SERGEEVICH; KALYUZHNY, DMITRY ALEKSEEVICH; YAKOVLEVA, YULIYA ALEKSEEVNA
To: YANDEX.TAXI LLC
Reel/Frame 053906/0707 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2020
From: YANDEX.TAXI LLC
To: YANDEX SELF DRIVING GROUP LLC
Reel/Frame 053906/0801 →