IP Library Granted Patent US 12,266,261
Granted Patent B2
US 12,266,261 · App. 17/658,295 · Granted Apr 1, 2025

Systems and methods for managing traffic rules using multiple mapping layers with traffic management semantics

Inventors: Vaibhav Ghadiok (Mountain View, CA); Christopher Carson (Oakland, CA); Bo Shen (Fremont, CA)
Assignee: Hayden AI Technologies, Inc.
G08G1/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,266,261
App. No.
17/658,295
Granted
Apr 1, 2025
Kind
B2
Abstract

Disclosed herein are systems and methods for managing traffic rules. In one embodiment, a method of managing traffic rules can comprise generating or updating a semantic map layer based in part on positioning data obtained from one or more edge devices and videos captured by the one or more edge devices. The method can also comprise generating or updating a traffic enforcement layer on top of the semantic map layer. A plurality of traffic rules can be saved as part of the traffic enforcement layer. The method can further comprise generating or updating a traffic insight layer based in part on traffic violations or traffic conditions determined by the one or more edge devices or the server. The traffic insight layer can adjust or provide a suggestion to adjust at least one of the traffic rules based on an impact analysis conducted by the traffic insight layer concerning the traffic rule.

Claims (38)

1. A method of managing traffic rules related to traffic enforcement, comprising:

generating or updating a map layer, using one or more processors of a server, based in part on positioning data obtained from one or more edge devices and videos captured by the one or more edge devices;

generating or updating, using the one or more processors of the server, a traffic enforcement layer on top of the map layer, wherein a plurality of traffic rules are saved as part of the traffic enforcement layer, wherein the traffic enforcement layer is generated in part by converting raw traffic rule data into the plurality of traffic rules related to traffic enforcement; and

generating or updating, using the one or more processors of the server, a traffic insight layer, wherein the traffic insight layer is configured to adjust or provide a suggestion to adjust at least one of the traffic rules of the traffic enforcement layer based in part on traffic violations or traffic conditions determined by the one or more edge devices or the server.

2. The method of claim 1 , wherein generating or updating the traffic enforcement layer further comprises the server receiving at least some of the traffic rules via user inputs applied to an interactive map editor user interface.

3. The method of claim 2 , wherein generating or updating the traffic enforcement layer further comprises the server receiving at least some of the traffic rules in response to a user dragging and dropping a rule primitive comprising at least one of a rule type, a rule attribute, and a rule logic onto a roadway displayed on a map of the interactive map editor user interface.

4. The method of claim 3 , further comprising receiving at least some of the traffic rules in response to the user dragging and dropping at least one of the rule type, the rule attribute, and the rule logic onto a route point displayed over the roadway shown on the map.

5. The method of claim 2 , wherein updating the map layer further comprises receiving a semantic annotation via user inputs applied to the interactive map editor user interface.

6. The method of claim 1 , wherein the traffic insight layer is further configured to adjust or provide the suggestion to adjust one of the traffic rules based on a change in a traffic throughput or flow determined by the traffic insight layer, and wherein adjusting or providing the suggestion to adjust one of the traffic rules further comprises not enforcing or providing a suggestion to not enforce one of the traffic rules based on the change in the traffic throughput or flow.

7. The method of claim 1 , wherein each of the edge devices is coupled to a carrier vehicle and wherein at least part of the videos are captured while the carrier vehicle is in motion.

8. The method of claim 1 , wherein generating or updating the traffic insight layer further comprises generating a heatmap of traffic violations detected by the one or more edge devices.

9. The method of claim 1 , wherein the map layer is generated or updated by passing the videos captured by at least one of the edge devices to a neural network running on the edge device and annotating the map layer with object labels outputted by the neural network.

10. A system for managing traffic rules related to traffic enforcement, comprising:

one or more edge devices comprising video image sensors configured to capture videos of roadways and an environment surrounding the roadways; and

a server communicatively coupled to the one or more edge devices, wherein the server comprises one or more server processors programmed to:

generate or update a map layer based in part on positioning data obtained from the one or more edge devices and the videos captured by the one or more edge devices;

generate or update a traffic enforcement layer on top of the map layer, wherein a plurality of traffic rules are saved as part of the traffic enforcement layer, wherein the traffic enforcement layer is generated in part by converting raw traffic rule data into the plurality of traffic rules related to traffic enforcement; and

generate or update a traffic insight layer, wherein the traffic insight layer is configured to adjust or provide a suggestion to adjust at least one of the traffic rules of the traffic enforcement layer based in part on traffic violations or traffic conditions determined by the one or more edge devices or the server.

11. The system of claim 10 , wherein the one or more server processors are programmed to execute instructions to generate or update the traffic enforcement layer by receiving at least some of the traffic rules via user inputs applied to an interactive map editor user interface.

12. The system of claim 11 , wherein the one or more server processors are programmed to execute instructions to generate or update the traffic enforcement layer by receiving at least some of the traffic rules in response to a user dragging and dropping a rule primitive comprising at least one of a rule type, a rule attribute, and a rule logic onto a roadway displayed on a map of the interactive map editor user interface.

13. The system of claim 12 , wherein at least one of the rule type, the rule attribute, and the rule logic is configured to be dropped onto a route point displayed over a roadway shown on the map.

14. The system of claim 11 , wherein the one or more server processors are programmed to execute instructions to update the map layer by receiving a semantic annotation via user inputs applied to the interactive map editor user interface.

15. The system of claim 10 , wherein the one or more server processors are programmed to execute instructions to adjust or provide the suggestion to adjust one of the traffic rules based on a change in a traffic throughput or flow determined by the traffic insight layer, and wherein the one or more server processors are programmed to execute instructions to adjust or provide a suggestion to adjust one of the traffic rules by not enforcing or providing a suggestion to not enforce one of the traffic rules based on the change in the traffic throughput or flow.

16. The system of claim 10 , wherein each of the edge devices is coupled to a carrier vehicle and wherein at least part of the videos are captured while the carrier vehicle is in motion.

17. The system of claim 10 , wherein the one or more server processors are programmed to execute instructions to generate or update the traffic insight layer by generating a heatmap of traffic violations detected by the one or more edge devices.

18. The system of claim 10 , wherein the one or more server processors are programmed to execute instructions to generate or update the map layer by passing the videos captured by at least one of the edge devices to a neural network running on the edge device and annotating the map layer with object labels outputted by the neural network.

19. A non-transitory computer-readable medium comprising machine-executable instructions stored thereon, wherein the instructions comprise the steps of:

generating or updating a map layer based in part on positioning data obtained from one or more edge devices and videos captured by the one or more edge devices;

generating or updating a traffic enforcement layer on top of the map layer, wherein a plurality of traffic rules related to traffic enforcement are saved as part of the traffic enforcement layer, wherein the traffic enforcement layer is generated in part by converting raw traffic rule data into the plurality of traffic rules related to traffic enforcement; and

generating or updating a traffic insight layer, wherein the traffic insight layer is configured to adjust or provide a suggestion to adjust at least one of the traffic rules of the traffic enforcement layer based in part on traffic violations or traffic conditions determined by the one or more edge devices or a server.

20. The non-transitory computer-readable medium of claim 19 , wherein the instructions further comprise the steps of generating or updating the traffic enforcement layer by receiving at least some of the traffic rules via user inputs applied to an interactive map editor user interface.

21. The non-transitory computer-readable medium of claim 20 , wherein the instructions further comprise the steps of generating or updating the traffic enforcement layer by receiving at least some of the traffic rules in response to a user dragging and dropping a rule primitive comprising at least one of a rule type, a rule attribute, and a rule logic onto a roadway displayed on a map of the interface map editor user interface.

22. The non-transitory computer-readable medium of claim 21 , wherein the instructions further comprise the steps of receiving at least some of the traffic rules in response to the user dragging and dropping at least one of the rule type, the rule attribute, and the rule logic onto a route point displayed over a roadway shown on the map.

23. The non-transitory computer-readable medium of claim 20 , wherein the instructions further comprise the steps of updating the map layer by receiving a semantic annotation via user inputs applied to the interactive map editor user interface.

24. The non-transitory computer-readable medium of claim 19 , wherein the instructions further comprise the steps of adjusting or providing the suggestion to adjust one of the traffic rules based on a change in a traffic throughput or flow determined by the traffic insight layer, and wherein the instructions further comprise the steps of adjusting or providing a suggestion to adjust one of the traffic rules by not enforcing or providing a suggestion to not enforce one of the traffic rules based on the change in the traffic throughput or flow.

25. The non-transitory computer-readable medium of claim 19 , wherein each of the edge devices is coupled to a carrier vehicle and wherein at least part of the videos are captured while the carrier vehicle is in motion.

26. The non-transitory computer-readable medium of claim 19 , wherein the instructions further comprise the steps of generating or updating the traffic insight layer by generating a heatmap of traffic violations detected by the one or more edge devices.

27. The non-transitory computer-readable medium of claim 19 , wherein the instructions further comprise the steps of generating or updating the map layer by passing the videos captured by at least one of the edge devices to a neural network running on the edge device and annotating the map layer with object labels outputted by the neural network.

Assignments (2)
SECURITY INTEREST Recorded Oct 27, 2025
From: HAYDEN AI TECHNOLOGIES INC.
To: BANK OF MONTREAL
Reel/Frame 072691/0585 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 7, 2022
From: GHADIOK, VAIBHAV; CARSON, CHRISTOPHER; SHEN, BO
To: HAYDEN AI TECHNOLOGIES, INC.
Reel/Frame 059530/0357 →
Continuity (4)
Continuation 17390226 · Jul 30, 2021
Provisional Application 63142903 · Jan 28, 2021
Related Publication 20220238012A1 · Jul 28, 2022
Related Publication 20230290245A9 · Sep 14, 2023
References Cited (81)
US 9070289B2 · Saund et al. · 2015 [cited by applicant]
US 10296794B2 · Ratti · 2019 [cited by applicant]
US 11003919B1 · Ghadiok et al. · 2021 [cited by applicant]
US 11164014B1 · Ghadiok et al. · 2021 [cited by applicant]
US 11361558B2 · Seo · 2022 [cited by applicant]
US 11688182B2 · Ghadiok et al. · 2023 [cited by applicant]
US 20020072847A1 · Trajkovic et al. · 2002 [cited by applicant]
US 20100081200A1 · Rajala et al. · 2010 [cited by applicant]
US 20110276370A1 · Agrait · 2011 [cited by examiner]
US 20120148092A1 · Ni et al. · 2012 [cited by applicant]
US 20130266188A1 · Bulan et al. · 2013 [cited by applicant]
US 20140007762A1 · Gavish et al. · 2014 [cited by applicant]
US 20140036076A1 · Nerayoff et al. · 2014 [cited by applicant]
US 20140311456A1 · Richter et al. · 2014 [cited by applicant]
US 20180172454A1 · Ghadiok et al. · 2018 [cited by applicant]
US 20180174446A1 · Wang · 2018 [cited by applicant]
US 20180240336A1 · Kareev et al. · 2018 [cited by applicant]
US 20190137280A1 · Ghadiok et al. · 2019 [cited by applicant]
US 20190197369A1 · Law et al. · 2019 [cited by applicant]
US 20190385453A1 · Dorne · 2019 [cited by examiner]
US 20200063866A1 · Reinhart et al. · 2020 [cited by applicant]
US 20200177767A1 · Kelly et al. · 2020 [cited by applicant]
US 20200380270A1 · Cox et al. · 2020 [cited by applicant]
US 20210150895A1 · Huang · 2021 [cited by applicant]
US 20210166145A1 · Omari · 2021 [cited by examiner]
US 20210209941A1 · Maheshwari et al. · 2021 [cited by applicant]
US 20210237737A1 · Al-Nuaimi et al. · 2021 [cited by applicant]
US 20210241003A1 · Seo · 2021 [cited by applicant]
US 20210264339A1 · Monaci · 2021 [cited by examiner]
US 20220147745A1 · Ghadiok et al. · 2022 [cited by applicant]
CN 2277104 · 1998 [cited by applicant]
CN 106560861 · 2017 [cited by applicant]
CN 110197589 · 2019 [cited by applicant]
CN 110321823 · 2019 [cited by applicant]
CN 110717433 · 2020 [cited by applicant]
CN 111368687 · 2020 [cited by applicant]
CN 111492416 · 2020 [cited by applicant]
CN 111666853 · 2020 [cited by applicant]
JP 4805763 · 2008 [cited by applicant]
KR 100812397 · 2008 [cited by applicant]
WO WO2010081200 · 2010 [cited by applicant]
WO WO2014007762 · 2014 [cited by applicant]
WO WO2020063866 · 2020 [cited by applicant]
WO WO2020177767 · 2020 [cited by applicant]
WO WO2022099237 · 2022 [cited by applicant]
WO WO2022165459 · 2022 [cited by applicant]
“Consulting services in Computer Vision and AI,” accessed on May 8, 2023, online. [cited by applicant]
Clearlane, “Automated Bus Lane Enforcement System”, [cited by applicant]
Clearlane, “The Safe Fleet Automated Bus Lane Enforcement (ABLE)”, [cited by applicant]
Evanko, K. “Siemens Mobility launches first-ever mobile bus lane enforcement solution in New York,” [cited by applicant]
Franklin, R. “Traffic Signal Violation Detection using Artificial Intelligence and Deep Learning,” [cited by applicant]
Huval, B. et al. “An Empirical Evaluation of Deep Learning on Highway Driving,” [cited by applicant]
Liu, X. “Vehicle-Related Scene Understanding Using Deep Learning,” [cited by applicant]
Liu, Z. et al. “A ConvNet for the 2020s,” [cited by applicant]
Nehemiah, A. et al. “Deep Learning for Automated Driving with Matlab,” [cited by applicant]
Safe Fleet. “Whitepaper: Vendor Interoperability for ABLE,” [cited by applicant]
“Safety Vision Announces Smart Automated Bus Lane Enforcement (SABLETM) Solution,” [cited by applicant]
Selmoune, A. et al. “Influencing factors in congestion pricing acceptability: a literature review,” [cited by applicant]
Spencer, B. et al. “NYC extends Brooklyn bus lane enforcement,” [cited by applicant]
Sullivan, T. “Transit Bus Surveillance Solutions,” [cited by applicant]
Wu, C. et al. “Adjacent Lane Detection and Lateral Vehicle Distance Measurement Using Vision-Based Neuro-Fuzzy Approaches,” [cited by applicant]
Canizo, M. et al. “Multi-Head CNN-RNN for multi time series anomaly detection: An industrial case study,” [cited by applicant]
Bo, T. et al., “Common phase error estimation in coherent optical OFDM systems using best-fit bounding box,” [cited by applicant]
Bo, T. et al., “Common Phase Estimation in Coherent OFDM System Using Image Processing Technique,” [cited by applicant]
Bo, T. et al., “Image Processing Based Common Phase Estimation for Coherent Optical Orthogonal Frequency Division Multiplexing System,” [cited by applicant]
Chen, S. et al., “A Dense Feature Pyramid Network-Based Deep Learning Model for Road Marking Instance Segmentation Using MLS Point Clouds,” [cited by applicant]
Chhaya, S. et al., “Basic Geometric Shape and Primary Colour Detection Using Image Processing On Matlab,” [cited by applicant]
Fan, Y. et al., “A Coarse-to-Fine Framework for Multiple Pedestrian Crossing Detection,” [cited by applicant]
Github Repository, Our Camera, <https://github.com/Bellspringsteen/OurCamera> (last visited Sep. 25, 2023). [cited by applicant]
Glenn, J., “Adaptive Morphological Feature-Based Object Classifier for a Color Imaging System,” [cited by applicant]
Hsu, K., et al. “Augmented Multiple Instance Regression for Inferring Object Contours in Bounding Boxes,” [cited by applicant]
Novak, L., “Vehicle Detection and Pose Estimation for Autonomous Driving,” [cited by applicant]
Oh, J. et al., “Context-based abnormal object detection using the fully-connected conditional random fields,” [cited by applicant]
Paquet, E. et al., “Description of shape information for 2-D and 3-D objects,” [cited by applicant]
Sengupta, S., “Semantic Mapping of Road Scenes,” [cited by applicant]
Siemens Mobility Inc., “Ratification of Completed Procurement Actions,” New York City Transit and Siemens Mobility Inc., [cited by applicant]
Siemens Mobility Traffic Solutions, “Enforcement solutions for safe and efficient cities,” [cited by applicant]
Tonge, A. et al., “Traffic Rules Violation Detection using Deep Learning,” [cited by applicant]
Viorel, C., “Some Aspects Concerning Geometric Forms Automatically Find Images and Ordering Them Using Robot Studio Simulation,” [cited by applicant]
Zhao Z. et al., “Deep Reinforcement Learning Based Lane Detection and Localization,” [cited by applicant]
Zhou, C. et al., “Predicting the passenger demand on bus services for mobile users,” [cited by applicant]