IP Library Granted Patent US 11,164,028
Granted Patent B2
US 11,164,028 · App. 16/444,941 · Granted Nov 2, 2021

License plate detection system

Inventors: Ilya Popov (Novgorod, RU); Krishna Khadloya (San Jose, CA); Sofiya Klyan (Novgorod, RU)
Assignee: Nortek Security & Control LLC
G06K9/3258G06K9/00718G06K9/6256G06K9/6262G06N3/04G06N3/08G06K2209/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,164,028
App. No.
16/444,941
Granted
Nov 2, 2021
Kind
B2
Abstract

A system for detecting license plates is described. The system receives raw data comprising images of license plates. A base version of a ground truth is prepared based on the raw data, using a generic license plate detection (LPD). The system prepares input data for training a deep learning network. The deep learning network is trained with the prepared input data. A newly trained generic (LPD) is formed using data generated by the existing generic (LPD).

Claims (39)

1. A computer-implemented method comprising: receiving raw data comprising images of license plates from a plurality of countries;

preparing, using a generic license plate detection (LPD), a base version of a ground truth based on the raw data, the generic (LPD) being trained based on the images of license plates from the plurality of countries; wherein preparing the base version of the ground truth further comprises: operating the generic (LPD) on the raw data to generate bounding boxes, classes, and country tags, the raw data further comprising existing information about bounding boxes, classes, and country tags; and validating, using the generic (LPD), the generated bounding boxes, classes, and country tags with the existing information from the raw data;

preparing input data for training a deep learning network; training the deep learning network with the prepared input data; and forming a newly trained generic (LPD) with the trained deep learning network, the newly trained generic (LPD) comprising a version of the generic (LPD) customized for one of the plurality of countries.

2. The computer-implemented method of claim 1 , further comprising:

testing the newly trained generic (LPD).

3. The computer-implemented method of claim 1 , wherein the raw data further comprises bounding boxes, and vehicle types.

4. The computer-implemented method of claim 1 , wherein preparing the input data further comprises:

gathering media data for the plurality of countries;

forming an image of a license plate from a video frame;

collecting the gathered media data separately for single row license plates and multiline license plates;

splitting the single row and the multiline sets of data into training and validation sets; and

saving the training and validation sets in a specific format.

5. The computer-implemented method of claim 1 , wherein the deep learning network receives an input image and outputs a bounding box corresponding to a license plate in the input image.

6. The computer-implemented method of claim 1 , wherein an output for the ground truth GT for license plate further comprises: initially autogenerated (GT) for license plates at images, and validation and suggestions for modifying of already existing (GT) for license plates.

7. The computer-implemented method of claim 1 , further comprising:

updating the generic (LPD) with the newly trained generic (LPD).

8. The computer-implemented method of claim 1 , further comprising:

forming a customized (LPD) for each country.

9. The computer-implemented method of claim 1 , wherein the deep neural network comprises a convolutional neural network (CNN).

10. A computing apparatus, the computing apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, configure the apparatus to: receive raw data comprising images of license plates from a plurality of countries; prepare, using a generic license plate detection (LPD), a base version of a ground truth based on the raw data, the generic (LPD) being trained based on the images of license plates from the plurality of countries;

wherein preparing the base version of the ground truth further comprises: operate the generic (LPD) on the raw data to generate bounding boxes, classes, and country tags, the raw data further comprising existing information about bounding boxes, classes, and country tags; and validate, using the generic (LPD), the generated bounding boxes, classes, and country tags with the existing information from the raw data; prepare input data for training a deep learning network; train the deep learning network with the prepared input data; and form a newly trained generic (LPD) with the trained deep learning network, the newly trained generic (LPD) comprising a version of the generic (LPD) customized for one of the plurality of countries.

11. The computing apparatus of claim 10 , wherein the instructions further configure the apparatus to:

test the newly trained generic (LPD).

12. The computing apparatus of claim 10 , wherein the raw data further comprises bounding boxes, and vehicle types.

13. The computing apparatus of claim 10 , wherein preparing the input data further comprises:

gathering media data for the plurality of countries;

form an image of a license plate from a video frame;

collect the gathered media data separately for single row license plates and multiline license plates;

split the single row and the multiline sets of data into training and validation sets; and

save the training and validation sets in a specific format.

14. The computing apparatus of claim 10 , wherein the deep learning network receives an input image and outputs a bounding box corresponding to a license plate in the input image.

15. The computing apparatus of claim 10 , wherein an output for the ground truth (GT) for license plate further comprises: initially autogenerated (GT) for license plates at images, and validation and suggestions for modifying of already exist (GT) for license plates.

16. The computing apparatus of claim 10 , wherein the instructions further configure the apparatus to:

update the generic (LPD) with the newly trained (LPD).

17. The computing apparatus of claim 10 , wherein the instructions further configure the apparatus to:

form a customized (LPD) for each country.

18. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:

receive raw data comprising images of license plates from a plurality of countries; prepare, using a generic license plate detection (LPD), a base version of a ground truth based on the raw data, the generic (LPD) being trained based on the images of license plates from the plurality of countries;

wherein preparing the base version of the ground truth further comprises: operating the generic (LPD) on the raw data to generate bounding boxes, classes, and country tags, the raw data further comprising existing information about bounding boxes, classes, and country tags; and validating, using the generic (LPD), the generated bounding boxes, classes, and country tags with the existing information from the raw data; prepare input data for training a deep learning network; train the deep learning network with the prepared input data; and form a newly trained generic (LPD) with the trained deep learning network, the newly trained generic (LPD) comprising a version of the generic (LPD) customized for one of the plurality of countries.

Assignments (2)
CHANGE OF NAME Recorded Jan 9, 2024
From: NORTEK SECURITY & CONTROL LLC
To: NICE NORTH AMERICA LLC
Reel/Frame 066242/0513 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 18, 2019
From: POPOV, ILYA; KHADLOYA, KRISHNA; KLYAN, SOFIYA
To: NORTEK SECURITY & CONTROL LLC
Reel/Frame 049508/0049 →