Granted Patent B2

US 12,670,613 · App. 17/910,684 · Granted Jun 30, 2026

Inventors: Subhash Challa (Victoria, AU); Nhat Vo (Victoria, AU); Louis Quinn (Victoria, AU); Duc Vo (Victoria, AU)

Assignee: SenSen Networks Group Pty Ltd.

G06T7/74G06V10/70G06V10/751G06V10/82G06V20/52G06V20/582G06V20/586

Abstract

Claims (67)

1 . A system for location determination, the system comprising:

a computing device comprising at least one processor and a memory accessible to the at least one processor;

wherein the memory comprises a library of reference background images and metadata for each reference background image, wherein the metadata comprises location information;

wherein the memory stores program code executable by the at least one processor to configure the at least one processor to:

receive an input image data from a remote computing device, wherein the input image data includes image data of at least one image captured by the remote computing device at a location to be determined;

process the received input image data using a background matching module to identify a matching reference background image;

determine location information corresponding to the input image data based on the metadata of the matching reference background image in the library; and

transmit the determined location information to the remote computing device; and

wherein the background matching module comprises a background feature extractor neural network trained to extract background descriptors in at least one captured image, wherein the background feature extractor neural network comprises an attention determination layer trained to determine attention weights for the background descriptors in the at least one captured image, wherein at least one of the background descriptors corresponding to a persistent background feature are given a high attention weight and at least another one of the background descriptors corresponding to a non-persistent background feature are given a low attention weight, wherein one or more permanent structures from the at least one captured image are identified by identifying the at least one of the background descriptors with the high attention weight corresponding to the persistent background feature, and the at least one processor is further configured to identify the matching reference background image by:

extracting the at least one of the background descriptors corresponding to the one or more permanent structures from the at least one captured image using the background feature extractor neural network;

selecting one or more candidate matching images from the library of background images based on the extracted at least one of the background descriptors; and

performing geometric matching between the at least one captured image and the candidate matching images to select the matching reference background image.

2 . The system of claim 1 , wherein the geometric matching comprises identifying common visual features in the at least one captured image and each of the one or more candidate matching images.

3 . The system of claim 1 , wherein the geometric matching is performed using a random sample consensus process.

4 . The system of claim 1 , wherein the memory stores program code executable by the at least one processor to further configure the at least one processor to:

receive GPS data corresponding to the input image from the remote computing device, wherein the GPS data comprises a low data quality indicator;

generate a GPS correction signal based on the determined location information;

transmit the GPS correction signal to the remote computing device.

5 . The system of claim 1 , wherein the memory stores program code executable by the at least one processor to further configure the at least one processor to:

receive GPS data corresponding to the input image from the remote computing device, wherein the GPS data comprises a low data quality indicator;

determine a subset of images from the library of background images based on the GPS data; and

select the one or more candidate matching images from the subset of images from the library of background images based on the extracted at least one of the background descriptors.

6 . A system for location determination in an urban area, the system comprising:

at least one camera, wherein the at least one camera is positioned to capture images of the urban area while the at least one camera is moving in the urban area;

a computing device moving with the at least one camera and in communication with the at least one camera to receive the captured images;

the computing device comprising at least one processor and a memory accessible to the at least one processor;

wherein the memory comprises a library of reference background images and metadata for each reference background image, wherein the metadata comprises location information;

wherein the memory stores program code executable by the at least one processor to configure the at least one processor to:

process a captured image using a background matching module to identify a matching reference background image; and

determine a location of the at least one camera and the computing device based on the metadata of the matching reference background image;

wherein the background matching module comprises a background feature extractor neural network trained to extract background descriptors in at least one captured image, wherein the background feature extractor neural network comprises an attention determination layer trained to determine attention weights for the background descriptors in the at least one captured image, wherein at least one of the background descriptors corresponding to a persistent background feature are given a high attention weight and at least another one of the background descriptors corresponding to a non-persistent background feature are given a low attention weight, wherein one or more permanent structures from the at least one captured image are identified by identifying the at least one of the background descriptors with the high attention weight corresponding to the persistent background feature, and processing the captured image using the background matching module comprises:

extracting the at least one of the background descriptors corresponding to the one or more permanent structures from the captured image using the background feature extractor neural network;

selecting one or more candidate matching images from the library of reference background images based on the extracted at least one of the background descriptors; and

performing geometric matching between the captured image and the candidate matching images to select the matching reference background image.

7 . The system of claim 6 , wherein the geometric matching is performed using a random sample consensus process; and

wherein the geometric matching comprises identifying common visual features in the at least one captured image and each of the one or more candidate matching images.

8 . The system of claim 6 , wherein the computing device is configured to determine the location in real-time.

9 . A computer implemented method for location determination, the method performed by a computing device comprising at least one processor in communication with a memory, the method comprising:

receiving an input image by the computing device from a remote computing device, wherein the input image corresponds to a location to be determined;

processing the received input image using a background matching module provided in the memory of the computing device to identify a matching reference background image from among a library of reference background images stored in the memory;

determining location information corresponding to the input image based on the metadata of the matching reference background image; and

transmitting the determined location information to the remote computing device;

wherein the background matching module comprises a background feature extractor neural network trained to identify background descriptors in at least one captured image, wherein the background feature extractor neural network comprises an attention determination layer trained to determine attention weights for the background descriptors in the at least one captured image, wherein at least one of the background descriptors corresponding to a persistent background feature are given a high attention weight and at least another one of the background descriptors corresponding to a non-persistent background feature are given a low attention weight, wherein one or more permanent structures from the at least one captured image are identified by identifying the at least one of the background descriptors with the high attention weight corresponding to the persistent background feature, and the method further comprises identifying the matching reference background image by:

extracting the at least one of the background descriptors corresponding to the one or more permanent structures from the at least one captured image using the background feature extractor neural network;

selecting one or more candidate matching images from the library of background images based on the extracted at least one of the background descriptors; and

performing geometric matching between the at least one captured image and the candidate matching images to select the matching reference background image.

10 . The method of claim 9 , wherein the geometric matching comprises identifying common visual features in the at least one captured image and each of the one or more candidate matching images.

11 . The method of claim 9 , wherein the geometric matching is performed using a random sample consensus process.

12 . The method of claim 9 , wherein the method further comprises:

receiving GPS data corresponding to the input image from the remote computing device, wherein the GPS data comprises a low data quality indicator;

generating a GPS correction signal based on the determined location information;

transmitting the GPS correction signal to the remote computing device;

wherein the GPS correction signal comprises information accessible by the remote computing device to determine a more accurate GPS location data.

13 . The method of claim 9 , wherein the method further comprises:

receiving GPS data corresponding to the input image from the remote computing device, wherein the GPS data comprises a low data quality indicator;

determining a subset of images from the library of background images based on the GPS data; and

selecting the one or more candidate matching images from the subset of images from the library of background images based on the extracted background descriptors.

14 . A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform operations comprising:

receiving an input image by the computing device from a remote computing device wherein the input image corresponds to a location to be determined;

processing the received input image using a background matching module provided in the memory of the computing device to identify a matching reference background image from among a library of reference background images stored in the memory;

determining location information corresponding to the input image based on the metadata of the matching reference background image; and

transmitting the determined location information to the remote computing device,

wherein the background matching module comprises:

a background feature extractor neural network trained to extract background descriptors in at least one captured image, wherein the background feature extractor neural network comprises an attention determination layer trained to determine attention weights for the background descriptors in the at least one captured image, wherein at least one of the background descriptors corresponding to a persistent background feature are given a high attention weight and at least another one of the background descriptors corresponding to a non-persistent background feature are given a low attention weight, wherein one or more permanent structures from the at least one captured image are identified by identifying the at least one of the background descriptors with the high attention weight corresponding to the persistent background feature, and the method further comprises identifying the matching reference background image by:

extracting the at least one of the background descriptors from the captured image;

selecting one or more candidate matching images from the library of background images based on the extracted at least one of the background descriptors; and

performing geometric matching between the at least one captured image and the candidate matching images to select the matching reference background image.

Assignments (1)

Priority Claims (2)

Continuity (2)

References Cited (36)