IP Library Granted Patent US 11,948,373
Granted Patent B2
US 11,948,373 · App. 18/064,883 · Granted Apr 2, 2024

Automatic license plate recognition

Inventors: Yi Xu (Belmont, CA); Yunchao Gong (Los Altos, CA); Suraj Arun Vathsa (Irvine, CA); Mayank Gupta (Foster City, CA); Naresh Nagabushan (San Mateo, CA)
Assignee: Verkada Inc.
G06V20/625G06V10/774G06V20/54G06V30/148
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,948,373
App. No.
18/064,883
Granted
Apr 2, 2024
Kind
B2
Abstract

Automatic license plate recognition occurs when a light sensor that continually captures video detects motion as a vehicle is driven through a gate. The light sensor detects the vehicle and license plate in the video stream captured by the light sensor. An algorithm associated with the video stream of the light sensor is trained to detect license plates. The light sensor starts executing the recognition algorithm when it detects motion. Recognition of characters in the license plate is based upon an aggregation of several captured video frames in which a license plate is detected.

Claims (117)

1. A method comprising:

analyzing a plurality of frames of a video of an object to detect a license plate associated with the object;

upon detecting the license plate in a frame of the plurality of frames, determining a subset of frames containing the license plate by analyzing neighboring frames associated with the frame to detect the license plate;

determining a plurality of characters in the license plate by, for each character in the plurality of characters:

based on a location of the character in the plurality of characters, determining corresponding characters in each frame in the subset of frames; and

determining, to be a part of the plurality of characters, a most frequent character among the character and the corresponding characters;

obtaining a plurality of bounding boxes indicating locations of the plurality of characters the locations represented by Cartesian coordinates;

obtaining a plurality of nodes based on the plurality of bounding boxes, wherein a respective location of each respective node indicates a respective location of a respective box in the plurality of bounding boxes;

determining a distance based on a first location of a first node of the plurality of nodes and a second location of a second node of the plurality of nodes;

determining whether the distance is below a threshold, wherein the threshold is computed based on a length of a long side of a box of the plurality of bounding boxes;

based on a determination that the distance is below the threshold, creating a first edge between the first and second nodes, thereby obtaining a first graph including the first node, the second node, and the first edge;

determining a leftmost node in the first graph based on first Cartesian coordinates representing the first location and second Cartesian coordinates representing the second location; and

ordering the plurality of characters into a sequence of characters based on the first graph and the leftmost node in the first graph.

2. The method of claim 1 , comprising:

performing an optical character recognition in each frame in the plurality of frames containing the license plate associated with the object to obtain a plurality of license plate characters associated with the object;

determining a number of characters in each frame in the plurality of frames containing the license plate associated with the object to obtain a plurality of numbers of characters;

determining a most frequent number of characters among the plurality of numbers of characters;

determining whether one or more frames in the plurality of frames containing the license plate associated with the object contain one or more numbers of characters different from the most frequent number of characters; and

upon determining that the one or more frames in the plurality of frames containing the license plate associated with the object contain the one or more numbers of characters different from the most frequent number of characters, discarding the one or more frames.

3. The method of claim 1 , further comprising:

determining whether a second graph disconnected from the first graph exists,

wherein the second graph includes a third node not connected to the first graph via a second edge,

and wherein the third node includes third Cartesian coordinates; and

upon determining that the second graph exists, determining an ordering of the first graph and the second graph based on the first Cartesian coordinates, the second Cartesian coordinates, and the third Cartesian coordinates, wherein ordering the plurality of characters comprises ordering the plurality of characters into the sequence of characters based on the ordering of the first graph and the second graph.

4. The method of claim 1 , further comprising:

determining whether a second graph disconnected from the first graph exists,

wherein the second graph includes a third node not connected to the first graph via a second edge;

receiving an indication of how to order the first graph and the second graph; and

upon determining that the second graph exists, based on the indication of how to order the first graph and the second graph, determining an ordering of the first graph and the second graph, wherein ordering the plurality of characters comprises ordering the plurality of characters into the sequence of characters based on the ordering of the first graph and the second graph.

5. The method of claim 1 , further comprising:

detecting a velocity of the object;

activating a camera,

wherein the camera is configured to record the video at an adjustable frame rate;

based on the velocity, adjusting the adjustable frame rate associated with the camera,

wherein a high velocity causes a first frame rate associated with the camera,

wherein a low velocity causes a second frame rate associated with the camera, and

wherein the first frame rate is higher than the second frame rate; and

recording the video of the object at the adjusted frame rate.

6. A system comprising:

at least one hardware processor; and

at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to:

analyze a plurality of frames of a video of an object to detect an alphanumeric identifier associated with the object;

upon detecting the alphanumeric identifier associated with the object in a frame of the plurality of frames, determine a subset of frames containing the alphanumeric identifier by analyzing neighboring frames associated with the frame to detect the alphanumeric identifier

determine a plurality of characters by:

based on a location of the character in the plurality of characters, determining corresponding characters in each frame in the subset of frames;

determine, to be a part of the plurality of characters, a most frequent character occurring among the character and the corresponding characters;

obtain a plurality of bounding boxes indicating locations of the plurality of characters, the locations represented by Cartesian coordinates;

obtain a plurality of nodes based on the plurality of bounding boxes, wherein a respective location of each respective node indicates a respective location of a respective box in the plurality of bounding boxes;

determine a distance based on a first location of a first node of the plurality of nodes and a second location of a second node of the plurality of nodes;

determine whether the distance is below a threshold, wherein the threshold is computed based on a length of a long side of a box of the plurality of bounding boxes;

based on a determination that the distance is below the threshold, create a first edge between the first and second nodes, thereby obtaining a first graph including the first node, the second node, and the first edge;

determine a leftmost node in the first graph based on first Cartesian coordinates representing the first location and second Cartesian coordinates representing the second location; and

order the plurality of characters into a sequence of characters based on the first graph and the leftmost node in the first graph.

7. The system of claim 6 , the instructions further comprising instructions that causes the at least one hardware processor to:

perform an optical character recognition in each frame in the plurality of frames containing the alphanumeric identifier associated with the object to obtain a plurality of alphanumeric identifier characters associated with the object; and

determine a number of characters associated with the alphanumeric identifier associated with the object based on each frame in the plurality of frames containing the alphanumeric identifier associated with the object.

8. The system of claim 6 ,

the instructions further comprising instructions that causes the at least one hardware processor to:

determine whether a second graph disconnected from the first graph exists,

wherein the second graph includes a third node not connected to the first graph via a second edge,

wherein the third node includes third Cartesian coordinates;

upon determining that the second graph exists, determine an ordering of the first graph and the second graph based on the first Cartesian coordinates, the second Cartesian coordinates, and the third Cartesian coordinates, wherein ordering the plurality of characters comprises ordering the plurality of characters into the sequence of characters based on the ordering of the first graph and the second graph.

9. The system of claim 6 , the instructions further comprising instructions that causes the at least one hardware processor to:

determine whether a second graph disconnected from the first graph exists,

wherein the second graph includes a third node not connected to the first graph via a second edge;

receive an indication of how to order the first graph and the second graph; and

upon determining that the second graph exists, based on the indication of how to order the first graph and the second graph, determine an ordering of the first graph and the second graph, wherein ordering the plurality of characters comprises ordering the plurality of characters into the sequence of characters based on the ordering of the first graph and the second graph.

10. The system of claim 6 , the instructions further comprising instructions that causes the at least one hardware processor to:

train a machine learning model to detect a target character having a predetermined proportion or a predetermined size; and

train the machine learning model to produce a bounding box associated with the target character.

11. The system of claim 6 , the instructions further comprising instructions that causes the at least one hardware processor to:

detect a velocity of the object;

activate a light sensor,

wherein the light sensor is configured to record the video at an adjustable frame rate;

based on the velocity, adjust the adjustable frame rate associated with the light sensor,

wherein a high velocity causes a first frame rate associated with the light sensor,

wherein a low velocity causes a second frame rate associated with the light sensor, and

wherein the first frame rate is higher than the second frame rate; and

record the video of the object at the adjusted frame rate.

12. At least one computer-readable storage medium, excluding transitory signals and carrying instructions, which, when executed by at least one data processor of a system, cause the system to:

analyze a plurality of frames of a video of an object to detect an alphanumeric identifier associated with the object;

upon detecting the alphanumeric identifier in a frame of the plurality of frames, determine a subset of frames containing the alphanumeric identifier by analyzing neighboring frames associated with the frame to detect the alphanumeric identifier;

determine a plurality of characters by performing, for each character in the plurality of characters:

based on a location of the character in the plurality of characters, determine corresponding characters in each frame in the subset of frames;

determine, to be a part of the plurality of characters, a most frequent character occurring among the character and the corresponding characters;

obtain a plurality of bounding boxes indicating locations of the plurality of characters, the locations represented by Cartesian coordinates;

obtain a plurality of nodes based on the plurality of bounding boxes, wherein a respective location of each respective node indicates a respective location of a respective box in the plurality of bounding boxes;

determine a distance based on a first location of a first node of the plurality of nodes and a second location of a second node of the plurality of nodes;

determine whether the distance between a first node and a second node of the plurality of nodes is below a threshold, wherein the threshold is computed based on a length of a long side of a box of the plurality of bounding boxes;

based on a determination that the distance is below the threshold, create a first edge between the first and second nodes, thereby obtaining a first graph including the first node, the second node, and the first edge;

determine a leftmost node in the first graph based on first Cartesian coordinates representing the first location and second Cartesian coordinates representing the second location; and

order the plurality of characters into a sequence of characters based on the first graph and the leftmost node in the first graph.

13. The at least one computer-readable storage medium of claim 12 , wherein the instructions further comprise instructions that causes the at least one data processor to:

determine whether a second graph disconnected from the first graph exists,

wherein the second graph includes a third node not connected to the first graph via a second edge,

wherein the third node includes third Cartesian coordinates;

upon determining that the second graph exists, determine an ordering of the first graph and the second graph based on the first Cartesian coordinates, the second Cartesian coordinates, and the third Cartesian coordinates, wherein ordering the plurality of characters comprises ordering the plurality of characters into the sequence of characters based on the ordering of the first graph and the second graph.

14. The at least one computer-readable storage medium of claim 12 , wherein the instructions further comprise instructions that causes the at least one data processor to:

determine whether a second graph disconnected from the first graph exists,

wherein the second graph includes a third node not connected to the first graph via a second edge;

receive an indication of how to order the first graph and the second graph; and

upon determining that the second graph exists, based on the indication of how to order the first graph and the second graph, determine an ordering of the first graph and the second graph, wherein ordering the plurality of characters comprises ordering the plurality of characters into the sequence of characters based on the ordering of the first graph and the second graph.

15. The at least one computer-readable storage medium of claim 12 , wherein the instructions further comprise instructions that causes the at least one data processor to:

train a machine learning model to detect a target character having a predetermined proportion or a predetermined size; and

train the machine learning model to produce a bounding box associated with the target character.

16. The at least one computer-readable storage medium of claim 12 , wherein the instructions further comprise instructions that causes the at least one data processor to:

detect a velocity of the object;

activate a light sensor,

wherein the light sensor is configured to record the video at an adjustable frame rate;

based on the velocity, adjust the adjustable frame rate associated with the light sensor,

wherein a high velocity causes a first frame rate associated with the light sensor,

wherein a low velocity causes a second frame rate associated with the light sensor, and

wherein the first frame rate is higher than the second frame rate; and

record the video of the object at the adjusted frame rate.

17. The at least one computer-readable medium of claim 12 , wherein the instructions further comprise instructions that causes the at least one data processor to:

perform an optical character recognition in each frame in the plurality of frames containing the alphanumeric identifier associated with the object to obtain a plurality of alphanumeric identifier characters associated with the object; and

determine a number of characters associated with the alphanumeric identifier associated with the object based on each frame in the plurality of frames containing the alphanumeric identifier associated with the object.

Assignments (2)
SECURITY INTEREST Recorded Oct 1, 2024
From: VERKADA INC.
To: FIRST-CITIZENS BANK & TRUST COMPANY, AS AGENT
Reel/Frame 068758/0910 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2022
From: XU, YI; GONG, YUNCHAO; VATHSA, SURAJ ARUN; GUPTA, MAYANK; NAGABUSHAN, NARESH
To: VERKADA INC.
Reel/Frame 062202/0906 →