Granted Patent B2

US 12,333,821 · App. 17/658,497 · Granted Jun 17, 2025

Inventors: Christopher Carson (Oakland, CA); Vaibhav Ghadiok (Mountain View, CA); Bo Shen (Fremont, CA)

Assignee: Hayden AI Technologies, Inc.

G06V20/58G06V20/625G08G1/0133G08G1/0141G08G1/0145

Abstract

Claims (82)

1. A method for alleviating traffic congestion within an enforcement zone, the method comprising:

capturing one or more videos of a vehicle using at least one video image sensor of an edge device,

using one or more processors of the edge device to pass video frames from the one or more videos to an object-detection neural network running on the edge device to detect at least one vehicle attribute of the vehicle from the video frames and pass the video frames to a license plate-recognition neural network running on the edge device to recognize a license plate number of the vehicle from the video frames;

determining, using the one or more processors of the edge device, that the vehicle is not exempt from a policy or rule concerning the enforcement zone based in part on the at least one vehicle attribute detected and the license plate number recognized;

determining, using one or more processors of the edge device, that the vehicle is at a first location within the enforcement zone during an enforcement period;

transmitting, from the edge device to a server, a first evidence package comprising the first location, a first timestamp marking when the vehicle was detected at the first location, the license plate number, and at least a portion of the one or more videos;

determining, using one or more processors of the edge device, that the vehicle is at a second location within the enforcement zone during the enforcement period;

transmitting, from the edge device to the server, a second evidence package comprising the second location, a second timestamp marking when the vehicle was detected at the second location, the license plate number, and at least another portion of the one or more videos; and

determining, at the server, that the vehicle was in motion within the enforcement zone during the enforcement period based on data and information from the first evidence package and the second evidence package.

2. The method of claim 1 , further comprising:

receiving, at the server, boundaries of the enforcement zone and the enforcement period as a result of user inputs applied to an interactive map user interface displayed on a computing device; and

transmitting, from the server to the edge device, coordinate data concerning the boundaries of the enforcement zone and the enforcement period to the edge device.

3. The method of claim 2 , wherein the enforcement zone is one lane of a roadway.

4. The method of claim 2 , wherein the enforcement zone comprises a partial segment of a roadway but not an entire segment of the roadway.

5. The method of claim 2 , wherein the enforcement zone comprises at least part of a carrier route of a carrier vehicle, and wherein the edge device is coupled to the carrier vehicle.

6. The method of claim 1 , wherein determining that the vehicle is at the first location within the enforcement zone further comprises:

calculating a separation distance representing a distance separating the edge device from a license plate of the vehicle using an image depth analysis algorithm;

adding the separation distance to a present location of the edge device obtained from a positioning unit of the edge device to yield the first location; and

comparing the first location against certain boundaries of the enforcement zone to determine that the first location is within the enforcement zone.

7. The method of claim 6 , wherein the separation distance is calculated using the image depth analysis algorithm based in part on a known size of the license plate in real-space, a known distance between a focal point of a video image sensor of the edge device and an image plane of the video image sensor, a projected size of the license plate on the image plane of the video image sensor, a known size of a calibration license plate in real space, a known calibration distance separating the calibration license plate from the focal point of the video image sensor, and a projected size of the calibration license plate on the image plane of the video image sensor.

8. The method of claim 1 , wherein the first evidence package further comprises at least one of a global positioning system (GPS) timestamp obtained by the edge device from one or more GPS satellites, a network time protocol (NTP) timestamp obtained by the edge device from a NTP server, and a local timestamp obtained from a clock running locally on the edge device.

9. The method of claim 8 , further comprising:

generating, using the one or more processors of the edge device, an interrupt at a rising edge of each pulse per second (PPS) signal of a GPS receiver of the edge device; and

executing a time synchronization function as a result of the interrupt, wherein the time synchronization function further comprises:

determining whether a local clock running on the edge device is synchronized with a time received from one or more GPS satellites or determining whether the clock running locally on the edge device is synchronized with a time received from the NTP server, and

synchronizing the clock running locally on the edge device to within one second of the time received from the one or more GPS satellites or the time received from the NTP server.

10. The method of claim 8 , wherein both the GPS timestamp and the NTP timestamp are transmitted to the server as part of the first evidence package if both the GPS timestamp and the NTP timestamp are available, and wherein only the local timestamp is transmitted to the server as part of the first evidence package if the edge device loses network connection with the NTP server and a GPS signal from the GPS satellite is unavailable.

11. The method of claim 1 , further comprising:

transmitting hashed signature packets from the edge device to the server at fixed time intervals;

determining, at the server, that the first timestamp received from the edge device as part of the first evidence package is inaccurate or the second timestamp received from the edge device as part of the second evidence package is inaccurate; and

calculating, at the server, a corrected timestamp to replace either the first timestamp or the second timestamp using the hashed signature packets.

12. The method of claim 1 , further comprising:

transmitting, from the edge device to the server, a first hashed signature packet along with the first evidence package; and

authenticating, at the server, the first evidence package using a first hashed signature from the first hashed signature packet.

13. The method of claim 12 , wherein the first hashed signature is generated at the edge device by applying a hash algorithm to a first alphanumeric string obtained by concatenating the first timestamp and data outputted by multiple electronic components running on at least one of the edge device and a carrier vehicle carrying the edge device, wherein the first hashed signature packet further comprises the first timestamp and the data outputted by the multiple electronic components, wherein the server authenticates the first evidence package by applying the hash algorithm to the first timestamp and the data outputted by the multiple electronic components to yield a first reconstructed hash signature, and wherein the server compares the first reconstructed hash signature against the first hashed signature received from the edge device to authenticate the first evidence package.

14. The method of claim 1 , further comprising:

receiving, at the server, a plurality of evidence packages including the first evidence package and the second evidence package;

grouping at least some of the evidence packages into a first location cluster, wherein the first evidence package is grouped into the first location cluster due to a geographic proximity of the first location to other vehicle locations within the other evidence packages included as part of the first location cluster; and

grouping at least some of the other evidence packages into a second location cluster, wherein the second evidence package is grouped into the second location cluster due to a geographic proximity of the second location to other vehicle locations within the other evidence packages included as part of the second location cluster.

15. The method of claim 1 , wherein the edge device is coupled to a carrier vehicle, wherein the one or more videos are captured while both the vehicle and the carrier vehicle are in motion.

16. A method for alleviating traffic congestion within an enforcement zone, the method comprising:

capturing a first video of a vehicle using at least one video image sensor of a first edge device coupled to a first carrier vehicle;

using one or more processors of the first edge device to pass video frames from the first video to a first object-detection neural network running on the first edge device to detect at least one vehicle attribute of the vehicle from the video frames and pass the video frames to a first license plate-recognition neural network running on the first edge device to recognize a license plate number of the vehicle from the video frames;

determining, using the one or more processors of the first edge device, that the vehicle is not exempt from a policy or rule concerning the enforcement zone based in part on the at least one vehicle attribute detected and the license plate number recognized;

determining, using the one or more processors of the first edge device, that the vehicle is at a first location within the enforcement zone during an enforcement period;

transmitting, from the first edge device to a server, a first evidence package comprising the first location, a first timestamp marking when the vehicle was detected at the first location, the license plate number, and at least a portion of the first video;

capturing a second video of the vehicle using at least one video image sensor of a second edge device coupled to a second carrier vehicle;

using one or more processors of the second edge device to pass video frames from the second video to a second object-detection neural network running on the second edge device to detect at least one vehicle attribute of the vehicle from the video frames and pass the video frames to a second license plate-recognition neural network running on the second edge device to recognize a license plate number of the vehicle from the video frames;

determining, using the one or more processors of the second edge device, that the vehicle is not exempt from a policy or rule concerning the enforcement zone based in part on the at least one vehicle attribute detected and the license plate number recognized;

determining, using the one or more processors of the second edge device, that the vehicle is at a second location within the enforcement zone during the enforcement period;

transmitting, from the second edge device to the server, a second evidence package comprising the second location, a second timestamp marking when the vehicle was detected at the second location, the license plate number, and at least a portion of the second video; and

determining, at the server, that the vehicle was in motion within the enforcement zone during the enforcement period based on data and information from the first evidence package and the second evidence package.

17. The method of claim 16 , wherein determining that the vehicle is at the first location within the enforcement zone further comprises:

calculating a separation distance representing a distance separating the first edge device from a license plate of the vehicle using an image depth analysis algorithm,

wherein the separation distance is calculated using the image depth analysis algorithm based in part on a known size of the license plate in real-space, a known distance between a focal point of a video image sensor of the first edge device and an image plane of the video image sensor, a projected size of the license plate on the image plane of the video image sensor, a known size of a calibration license plate in real space, a known calibration distance separating the calibration license plate from the focal point of the video image sensor, and a projected size of the calibration license plate on the image plane of the video image sensor;

adding the separation distance to a present location of the first edge device obtained from a positioning unit of the first edge device to yield the first location; and

comparing the first location against certain boundaries of the enforcement zone to determine that the first location is within the enforcement zone.

18. The method of claim 16 , further comprising:

transmitting hashed signature packets from the first edge device to the server and from the second edge device to the server at fixed time intervals;

determining, at the server, that the first timestamp received from the first edge device as part of the first evidence package is inaccurate or the second timestamp received from the second edge device as part of the second evidence package is inaccurate; and

calculating, at the server, a corrected timestamp to replace either the first timestamp or the second timestamp using the hashed signature packets.

19. A system for alleviating traffic congestion within an enforcement zone, comprising:

an edge device comprising video image sensors configured to capture one or more videos of a vehicle, wherein the edge device comprises one or more processors programmed to:

pass video frames from the one or more videos to an object-detection neural network running on the edge device to detect at least one vehicle attribute of the vehicle from the video frames and pass the video frames to a license plate-recognition neural network running on the edge device to recognize a license plate number of the vehicle from the video frames,

determine that the vehicle is not exempt from a policy or rule concerning the enforcement zone based in part on the at least one vehicle attribute detected and the license plate number recognized,

determine that the vehicle is at a first location within the enforcement zone during an enforcement period,

transmit, to a server, a first evidence package comprising the first location, a first timestamp marking when the vehicle was detected at the first location, the license plate number, and at least a portion of the one or more videos,

determine that the vehicle is at a second location within the enforcement zone during the enforcement period, and

transmit, to the server, a second evidence package comprising the second location, a second timestamp marking when the vehicle was detected at the second location, the license plate number, and at least another portion of the one or more videos; and

wherein the server is communicatively coupled to the edge device, wherein the server comprises one or more server processors programmed to determine that the vehicle was in motion within the enforcement zone during the enforcement period based on data and information from the first evidence package and the second evidence package.

20. A system for alleviating traffic congestion within an enforcement zone, comprising:

a first edge device comprising one or more video image sensors configured to capture a first video of a vehicle, wherein the first edge device comprises one or more processors programmed to:

pass video frames from the first video to a first object-detection neural network running on the first edge device to detect at least one vehicle attribute of the vehicle from the video frames and pass the video frames to a first license plate-recognition neural network running on the first edge device to recognize a license plate number of the vehicle from the video frames,

determine that the vehicle is not exempt from a policy or rule concerning the enforcement zone based in part on the at least one vehicle attribute detected and the license plate number recognized,

determine that the vehicle is at a first location within the enforcement zone during an enforcement period, and

transmit, to a server, a first evidence package comprising the first location, a first timestamp marking when the vehicle was detected at the first location, the license plate number, and at least a portion of the first video;

a second edge device comprising one or more video image sensors configured to capture a second video of the vehicle, wherein the second edge device comprises one or more processors programmed to:

pass video frames from the second video to a second object-detection neural network running on the second edge device to detect at least one vehicle attribute of the vehicle from the video frames and pass the video frames to a second license plate-recognition neural network running on the second edge device to recognize the license plate number of the vehicle from the video frames,

determine that the vehicle is not exempt from a policy or rule concerning the enforcement zone based in part on the at least one vehicle attribute detected and the license plate number recognized,

determine that the vehicle is at a second location within the enforcement zone during the enforcement period, and

transmit, to the server, a second evidence package comprising the second location, a second timestamp marking when the vehicle was detected at the second location, the license plate number, and at least a portion of the second video; and

wherein the server is communicatively coupled to the first edge device and the second edge device, wherein the server comprises one or more server processors programmed to determine that the vehicle was in motion within the enforcement zone during the enforcement period based on data and information from the first evidence package and the second evidence package.

Assignments (3)

Continuity (2)

References Cited (90)