IP Library Granted Patent US 11,823,389
Granted Patent B2
US 11,823,389 · App. 16/723,829 · Granted Nov 21, 2023

Road network mapping system and method

Inventor: Sanjay Chawla (Doha, QA)
Assignee: Qatar Foundation for Education, Science and Community Development
G06T7/12G01C21/3815G01C21/3852G06F16/29G06N3/084G06V10/764G06V10/82G06V20/182G06T2207/10032G06V20/194
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,823,389
App. No.
16/723,829
Granted
Nov 21, 2023
Kind
B2
Abstract

A method and system of constructing a network map from imagery comprising using an iterative search process guided by a CNN-based decision function to derive a network graph directly from the output of the CNN.

Claims (32)

1. A method of constructing a road network map from imagery, the method comprising:

using an iterative search process guided by a decision function implemented by using a convolutional neural network (CNN) to derive a road network graph, the road network graph including vertices which are annotated with spatial coordinates, including latitude and longitude coordinates, and edges corresponding to straight-line road segments, the road network graph being derived directly from an output generated by the CNN.

2. The method of claim 1 , wherein the iterative search process constructs the road network map by adding individual road segments one at a time.

3. The method of claim 1 , wherein the iterative search process proceeds by starting from a single location known to be on the road network graph and adds the vertices which are annotated with the spatial coordinates and the edges corresponding to the straight-line road segments along a shortest path that the iterative search process follows.

4. The method of claim 1 , wherein the CNN-based decision function implemented by the CNN is invoked at each step to determine a best action to take, the best action to take including either to add one of the edges corresponding to the straight-line road segments to the road network being mapped or step back to a previous one of the vertices annotated with the spatial coordinates.

5. The method of claim 1 , wherein the method further comprises the decision function deciding on a location of where to start a search algorithm walk action on the road network graph in a ground truth map G* and to pick a next one of the vertices annotated with spatial coordinates by:

applying a map-matching algorithm to find a path p* in the truth map G* that is most similar to a path p in a graph G ending at a vertex S top at the top of the vertices representing the current location of the search;

obtaining the path p in the graph G, by performing a random walk on the road network graph in the graph G starting from the vertex S top ;

stopping the random walk in the graph G after traversing a configurable number of the vertices or when there are none of the vertices adjacent to the current vertex S top one of the vertices that haven't already been traversed earlier in the search algorithm walk action; and

matching the path p in the graph G to the path p* in the map G* to which it is most similar.

6. The method of claim 5 , further comprising using the map-matching algorithm at an endpoint v of a last one of the edges in the path p*, starting the walk action in the map G* at the endpoint v and maintaining a set containing the edges E of the path p* in the map G* that have already been explored during the walk action in the map G*, the set of the edges E is initially empty, and on each training step, after deriving the path p*from the map-matching algorithm, adding each of the edges in path p* to the set of the edges E, then, when performing the walk action in the map G*, avoiding traversing the edges that are in the set of the edges E.

7. The method according to claim 1 , wherein the decision function implemented by using a convolutional neural network (CNN) decides on a next segment of the road network map to add given as an input a portion of the road network constructed so far to produce the road network directly from the CNN without generating an intermediate image representation of the road network.

8. The method of claim 1 , wherein the decision function repeatedly selects a stop action until the iterative search process returns to a vertex v 0 of the vertices and a stack of the vertices S becomes empty, wherein when the stack of the vertices S is empty, the construction of the road network map is complete.

9. The method of claim 1 , wherein the imagery comprises one or more aerial images.

10. The method of claim 1 , wherein the iterative search process is performed on the imagery in which pixels are classified as “road network” or “non-road network”.

11. The method of claim 1 , wherein the method comprises performing a training procedure for the decision function.

12. The method of claim 11 , wherein the training procedure comprises:

generating training examples by running the iterative search process with the CNN as the decision function during the training procedure.

13. The method of claim 11 , wherein the training procedure comprises:

performing a dynamic labelling process to generate the training examples.

14. The method of claim 11 , wherein given a region having a starting location at a vertex (v 0 ) of the vertices and a bounding box (B) defining an area in which to infer the road network, the training procedure begins by initialising an instance of a search algorithm of a partial graph (G) initially containing only the starting location at the vertex (v 0 ) generated by the search algorithm and a stack of the vertices (S), and wherein on each step of the training procedure, as during inference, the CNN decides on an action based on an output of the CNN, and updates the partial graph (G) and the vertex stack (S) based on the action.

15. The method of claim 14 , comprising updating the partial graph G and the stack of the vertices S based on the action decided by the CNN and not based on a prediction by an oracle as the decision function during the training procedure.

16. The method of claim 14 , wherein on each step of the training position, based on map G*, identify a set of angles R where there are unexplored elements of the road network from a vertex (S top ) at the top of the stack of vertices S,

convert angles R into a target output vector O*,

if angles R is empty, then output stop O stop *=1, else output walk o stop *−1, and for each of an angle θ∈R, set o i *=1, where i is a closest walkable angle to θ,

compute a loss between O and O* and apply back-propagation to update parameters of the CNN.

17. A system for constructing a road network map from imagery, the system comprising a processor and a non-transitory computer readable medium (CRM), the CRM storing executable instructions which, when executed by the processor, cause the processor to:

use an iterative search process guided by a decision function implemented by using a convolutional neural network (CNN) to derive a road network graph, the road network graph including vertices which are annotated with spatial coordinates, and edges corresponding to line road segments, the road network graph being derived directly from an output generated by the CNN.

18. An apparatus for constructing a road network map from imagery, the apparatus comprising:

a processor;

a non-transitory computer readable medium (CRM), the CRM storing executable instructions which, when executed by the processor, cause the processor to:

utilize a convolutional neural network (CNN)-based decision process to receive and process image data and to output a decision function implemented by using the CNN to derive a road network graph including vertices which are annotated with spatial coordinates, and edges corresponding to line road segments, the road network graph being derived directly from an output generated by the CNN.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 17, 2025
From: QATAR FOUNDATION FOR EDUCATION, SCIENCE & COMMUNITY DEVELOPMENT
To: HAMAD BIN KHALIFA UNIVERSITY
Reel/Frame 069936/0656 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2023
From: CHAWLA, SANJAY
To: QATAR FOUNDATION FOR EDUCATION, SCIENCE, AND COMMUNITY DEVELOPMENT
Reel/Frame 065405/0740 →
Continuity (2)
Provisional Application 62782615 · Dec 20, 2018
Related Publication 20200200548A1 · Jun 25, 2020