IP Library Granted Patent US 11,282,381
Granted Patent B2
US 11,282,381 · App. 16/906,250 · Granted Mar 22, 2022

Traffic speed enforcement based on license plate detection

Inventors: Hiroaki Nishimura (Paramus, NJ); Nikolaos Georgis (San Diego, CA); James Kuch (Wheaton, IL)
Assignee: SONY GROUP CORPORATION
G08G1/054G06T7/246G06T7/254G08G1/04G06T2207/20084G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,282,381
App. No.
16/906,250
Granted
Mar 22, 2022
Kind
B2
Abstract

An electronic apparatus and method for traffic speed enforcement based on license plate detection is provided. The electronic apparatus controls an image-capture device on a first vehicle to capture a sequence of images of a second vehicle and detects a license plate of the second vehicle in the captured sequence of images. The electronic apparatus extracts a set of license plate images from a set of images in the sequence of images and determines a first change in a size of a first license plate image with respect to that of a second license plate image. The electronic apparatus further determines a speed of the first vehicle and a traffic speed violation by the second vehicle based on the determined first change and the determined speed of the first vehicle. The electronic apparatus further controls a media device to output a notification indicative of the determined traffic speed violation.

Claims (63)

1. An electronic apparatus, comprising:

circuitry communicatively coupled to an image-capture device installed in a first vehicle, wherein the circuitry is configured to:

control the image-capture device to capture a sequence of images of a second vehicle;

detect a license plate of the second vehicle in each image of the captured sequence of images;

determine a time instant at which a speed of the first vehicle, in which the image-capture device is installed, is equal to a defined speed limit;

select a set of images from the captured sequence of images based on the determined time instant, wherein the selected set of images corresponds to the time instant at which the speed of the first vehicle is equal to the defined speed limit;

extract, from the selected set of images in the captured sequence of images, a set of license plate images corresponding to the detected license plate;

determine a first change in a size of a first license plate image of the extracted set of license plate images with respect to a size of a second license plate image of the extracted set of license plate images;

determine a traffic speed violation by the second vehicle based on the determined first change; and

control a media device to output a notification indicative of the determined traffic speed violation.

2. The electronic apparatus according to claim 1 , wherein the circuitry is further configured to:

input the captured sequence of images to a Deep Neural Network (DNN) trained on a license plate detection (LPD) task;

extract a sequence of LPD results as an output of the DNN for the input sequence of images, wherein the output of the DNN comprises bounding box coordinates of the license plate and an LPD confidence score indicating a likelihood of the license plate within the bounding box coordinates; and

detect the license plate of the second vehicle in each image of the captured sequence of images based on the extracted sequence of LPD results.

3. The electronic apparatus according to claim 1 , wherein

the defined speed limit is one of a legal maximum speed or a legal minimum speed at which the second vehicle is allowed to move on a road.

4. The electronic apparatus according to claim 1 , wherein the circuitry is further configured to determine a relative speed of the second vehicle with respect to the speed of the first vehicle based on the determined first change.

5. The electronic apparatus according to claim 4 , wherein

the circuitry is further configured to determine the traffic speed violation by the second vehicle based on a determination that the determined relative speed indicates that an absolute speed of the second vehicle is greater than the defined speed limit, and

the defined speed limit is a legal maximum speed at which the second vehicle is allowed to move on a road.

6. The electronic apparatus according to claim 4 , wherein

the circuitry is further configured to determine the traffic speed violation by the second vehicle based on a determination that the determined relative speed indicates that an absolute speed of the second vehicle is less than the defined speed limit, and

the defined speed limit is a legal minimum speed at which the second vehicle is allowed move on a road.

7. The electronic apparatus according to claim 1 , wherein the determined first change corresponds to one of an increase or a decrease in the size of the first license plate image with respect to the size of the second license plate image.

8. The electronic apparatus according to claim 1 , wherein the circuitry is further configured to:

determine a second change in a font-size of a license plate number on the first license plate image with respect to a font-size of the license plate number on the second license plate image; and

determine the traffic speed violation by the second vehicle based on the determined second change and the speed of the first vehicle.

9. The electronic apparatus according to claim 8 , wherein the determined second change corresponds to one of an increase or a decrease in the font-size of the license plate number on the first license plate image with respect to the font-size of the license plate number on the second license plate image.

10. The electronic apparatus according to claim 1 , wherein the circuitry is further configured to:

receive, via a location sensor, location information associated with the first vehicle; and

determine the speed of the first vehicle based on the received location information.

11. A method, comprising:

controlling an image-capture device installed in a first vehicle to capture a sequence of images of a second vehicle;

detecting a license plate of the second vehicle in each image of the captured sequence of images;

determining a time instant at which a speed of the first vehicle, in which the image-capture device is installed, is equal to a defined speed limit;

selecting a set of images from the captured sequence of images based on the determined time instant, wherein the selected set of images corresponds to the time instant at which the speed of the first vehicle is equal to the defined speed limit;

extracting, from the selected set of images in the captured sequence of images, a set of license plate images corresponding to the detected license plate;

determining a first change in a size of a first license plate image of the extracted set of license plate images with respect to a size of a second license plate image of the extracted set of license plate images;

determining a traffic speed violation by the second vehicle based on the determined first change; and

controlling a media device to output a notification indicative of the determined traffic speed violation.

12. The method according to claim 11 , further comprising:

inputting the captured sequence of images to a Deep Neural Network (DNN) trained on a license plate detection (LPD) task;

extracting a sequence of LPD results as an output of the DNN for the input sequence of images, wherein the output of the DNN comprises bounding box coordinates of the license plate and an LPD confidence score indicating a likelihood of the license plate within the bounding box coordinates; and

detecting the license plate of the second vehicle in each image of the captured sequence of images based on the extracted sequence of LPD results.

13. The method according to claim 11 , wherein

the defined speed limit is a legal maximum speed or a legal minimum speed at which the second vehicle is allowed to move on a road.

14. The method according to claim 11 , further comprising determining a relative speed of the second vehicle with respect to the speed of the first vehicle based on the determined first change.

15. The method according to claim 14 , further comprising determining the traffic speed violation by the second vehicle based on a determination that the determined relative speed indicates that an absolute speed of the second vehicle is greater than the defined speed limit,

wherein the defined speed limit is a legal maximum speed at which the second vehicle is allowed to move on a road.

16. The method according to claim 14 , further comprising determining the traffic speed violation by the second vehicle based on a determination that the determined relative speed indicates that an absolute speed of the second vehicle is less than the defined speed limit,

wherein the defined speed limit is a legal minimum speed at which the second vehicle is allowed move on a road.

17. The method according to claim 11 , further comprising:

determining a second change in a font-size of a license plate number on the first license plate image with respect to a font-size of the license plate number on the second license plate image; and

determining the traffic speed violation by the second vehicle based on the determined second change and the speed of the first vehicle.

18. A non-transitory computer-readable storage medium configured to store computer-executable instructions which, when executed by a processor of an electronic apparatus, causes the processor to perform operations, the operations comprising:

controlling an image-capture device installed in a first vehicle to capture a sequence of images of a second vehicle;

detecting a license plate of the second vehicle in each image of the captured sequence of images;

determining a time instant at which a speed of the first vehicle, in which the image-capture device is installed, is equal to a defined speed limit;

selecting a set of images from the captured sequence of images based on the determined time instant, wherein the selected set of images corresponds to the time instant at which the speed of the first vehicle is equal to the defined speed limit;

extracting, from the selected set of images in the captured sequence of images, a set of license plate images corresponding to the detected license plate;

determining a first change in a size of a first license plate image of the extracted set of license plate images with respect to a size of a second license plate image of the extracted set of license plate images;

determining a traffic speed violation by the second vehicle based on the determined first change; and

controlling a media device to output a notification indicative of the determined traffic speed violation.

Assignments (2)
CHANGE OF NAME Recorded Aug 20, 2021
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 057249/0679 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 22, 2020
From: NISHIMURA, HIROAKI; GEORGIS, NIKOLAOS; KUCH, JAMES
To: SONY CORPORATION
Reel/Frame 053001/0596 →
Continuity (1)
Related Publication 20210398421A1 · Dec 23, 2021