IP Library Granted Patent US 10,943,151
Granted Patent B2
US 10,943,151 · App. 16/147,470 · Granted Mar 9, 2021

Systems and methods for training and validating a computer vision model for geospatial imagery

Inventors: Joshua Eno (Denver, CO); Lindy Williams (Broomfield, CO); Wesley Boyer (Denver, CO); Kevin Brackney (Westminster, CO); Bergen Davell (Wheat Ridge, CO); Khoa Truong Pham (Centennial, CO)
Assignee: Verizon Patent and Licensing Inc.
G06K9/6263G06F3/0482G06K9/3241G06N20/00G06T17/05
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,943,151
App. No.
16/147,470
Granted
Mar 9, 2021
Kind
B2
Abstract

An exemplary geospatial image processing system generates, based on multiple detections of an object of interest detected by a computer vision model in multiple, correlated images of a geospatial location captured from different camera viewpoints, user interface content that includes a visual indication of the detected object of interest superimposed at an object position on a view of the geospatial location. The system provides the user interface content for display in a graphical user interface view of a user interface and provides, by way of the user interface, a user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest. The system may receive, a user validation of one or more of the multiple detections of the object of interest and may train the computer vision model based on the user validation. Corresponding methods and systems are also disclosed.

Claims (92)

1. A method comprising:

generating, by a geospatial image processing system, based on multiple detections of an object of interest detected by a computer vision model in multiple, correlated images of a geospatial location captured from different camera viewpoints, user interface content that includes a visual indication of the detected object of interest superimposed at an object position on a view of the geospatial location;

providing, by the geospatial image processing system, the user interface content for display in a graphical user interface view of a user interface; and

providing, by the geospatial image processing system, by way of the user interface, a user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest,

wherein

the multiple, correlated images of the geospatial location are a filtered set of images of the geospatial location in which each image in the filtered set of images includes the detected object of interest,

the filtered set of images of the geospatial location comprises multiple street-level images of the geospatial location in which the object of interest is detected, and

the user interface content includes a plurality of partial image portions extracted from the multiple street-level images of the geospatial location in which the object of interest is detected, the plurality of partial image portions depicting the multiple detections of the object of interest detected by the computer vision model from the different camera viewpoints.

2. The method of claim 1 , further comprising:

receiving, by the geospatial image processing system, by way of the user interface tool of the user interface, user input indicating a confirmation or a rejection of one or more of the multiple detections of the object of interest;

updating, by the geospatial image processing system, metadata associated with the filtered set of images based on the confirmation or the rejection; and

training, by the geospatial image processing system, the computer vision model based on the updated metadata.

3. The method of claim 1 , wherein:

the view of the geospatial location comprises a map view of the geospatial location;

the visual indication of the detected object of interest is superimposed at the object position on the map view of the geospatial location; and

the user interface content further includes visual indications of the different camera viewpoints, from which the multiple street-level images of the geospatial location were captured, superimposed at different camera positions on the map view.

4. The method of claim 1 , wherein:

the view of the geospatial location comprises a street view of the geospatial location; and

the visual indication of the detected object of interest is superimposed at the object position on the street view of the geospatial location.

5. The method of claim 1 , wherein:

the view of the geospatial location comprises a map view of the geospatial location; and

the visual indication of the detected object of interest is superimposed at the object position on the map view of the geospatial location.

6. The method of claim 1 , wherein the providing of the user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest comprises:

receiving, by way of the user interface, user input selecting a partial image portion included in the plurality of partial images portions; and

providing, in response to the user input the partial image portion, a menu of selectable options comprising

a confirmation option selectable by a user to confirm one of the multiple detections of the object of interest that is represented by the partial image portion, and

a rejection option selectable by the user to reject the one of the multiple detections of the object of interest that is represented by the partial image portion.

7. The method of claim 1 , wherein the providing of the user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest comprises:

receiving, by way of the user interface, user input selecting the visual indication of the detected object of interest; and

providing, in response to the user input selecting the visual indication of the detected object of interest, a menu of selectable options comprising

a confirmation option selectable by a user to confirm one or more of the multiple detections of the object of interest, and

a rejection option selectable by the user to reject one or more of the multiple detections of the object of interest.

8. A system comprising:

at least one physical computing device configured to:

generate, based on multiple detections of an object of interest detected by a computer vision model in multiple, correlated images of a geospatial location captured from different camera viewpoints, user interface content that includes a visual indication of the detected object of interest superimposed at an object position on a view of the geospatial location;

provide the user interface content for display in a graphical user interface view of a user interface; and

provide, by way of the user interface, a user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest,

wherein

the multiple correlated images of the geospatial location are a filtered set of images of the geospatial location in which each image in the filtered set of images includes the detected object of interest,

the filtered set of images of the geospatial location comprises multiple street-level images of the geospatial location in which the object of interest is detected, and

the user interface content includes a plurality of partial image portions extracted from the multiple street-level images of the geospatial location in which the object of interest is detected, the plurality of partial image portions depicting the multiple detections of the object of interest detected by the computer vision model from the different camera viewpoints.

9. The system of claim 8 , the at least one physical computing device further configured to:

receive, by way of the user interface tool of the user interface, user input indicating a confirmation or a rejection of one or more of the multiple detections of the object of interest;

update metadata associated with the filtered set of images based on the confirmation or the rejection; and

train the computer vision model based on the updated metadata.

10. The system of claim 8 , wherein:

the view of the geospatial location comprises a map view of the geospatial location;

the visual indication of the detected object of interest is superimposed at the object position on the map view of the geospatial location; and

the user interface content further includes visual indications of the different camera viewpoints, from which the multiple street-level images of the geospatial location were captured, superimposed at different camera positions on the map view.

11. The system of claim 8 , wherein:

the view of the geospatial location comprises a street view of the geospatial location; and

the visual indication of the detected object of interest is superimposed at the object position on the street view of the geospatial location.

12. The system of claim 8 , wherein:

the view of the geospatial location comprises a map view of the geospatial location; and

the visual indication of the detected object of interest is superimposed at the object position on the map view of the geospatial location.

13. The system of claim 8 , wherein the at least one physical computing device is configured to provide the user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest by:

receiving, by way of the user interface, user input selecting a partial image portion included in the plurality of partial images portions; and

providing, in response to the user input the partial image portion, a menu of selectable options comprising

a confirmation option selectable by a user to confirm one of the multiple detections of the object of interest that is represented by the partial image portion, and

a rejection option selectable by the user to reject the one of the multiple detections of the object of interest that is represented by the partial image portion.

14. The system of claim 8 , wherein the at least one physical computing device is configured to provide the user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest by:

receiving, by way of the user interface, user input selecting the visual indication of the detected object of interest; and

providing, in response to the user input selecting the visual indication of the detected object of interest, a menu of selectable options comprising

a confirmation option selectable by a user to confirm one or more of the multiple detections of the object of interest, and

a rejection option selectable by the user to reject one or more of the multiple detections of the object of interest.

15. A non-transitory computer-readable medium storing instructions executable by a processor of a computing device to:

generate, based on multiple detections of an object of interest detected by a computer vision model in multiple, correlated images of a geospatial location captured from different camera viewpoints, user interface content that includes a visual indication of the detected object of interest superimposed at an object position on a view of the geospatial location;

provide the user interface content for display in a graphical user interface view of a user interface; and

provide, by way of the user interface, a user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest,

wherein

the multiple, correlated images of the geospatial location are a filtered set of images of the geospatial location in which each image in the filtered set of images includes the detected object of interest,

the filtered set of images of the geospatial location comprises multiple street-level images of the geospatial location in which the object of interest is detected, and

the user interface content includes a plurality of partial image portions extracted from the multiple street-level images of the geospatial location in which the object of interest is detected, the plurality of partial image portions depicting the multiple detections of the object of interest detected by the computer vision model from the different camera viewpoints.

16. The non-transitory computer-readable medium of claim 15 , wherein the instructions are further executable by the processor of the computing device to:

receive, by way of the user interface tool of the user interface, user input indicating a confirmation or a rejection of one or more of the multiple detections of the object of interest;

update metadata associated with the filtered set of images based on the confirmation or the rejection; and

train the computer vision model based on the updated metadata.

17. The non-transitory computer-readable medium of claim 15 , wherein:

the view of the geospatial location comprises a map view of the geospatial location;

the visual indication of the detected object of interest is superimposed at the object position on the map view of the geospatial location; and

the user interface content further includes visual indications of the different camera viewpoints, from which the multiple street-level images of the geospatial location were captured, superimposed at different camera positions on the map view.

18. The non-transitory computer-readable medium of claim 15 , wherein:

the view of the geospatial location comprises a street view of the geospatial location; and

the visual indication of the detected object of interest is superimposed at the object position on the street view of the geospatial location.

19. The non-transitory computer-readable medium of claim 15 , wherein:

the view of the geospatial location comprises a map view of the geospatial location; and

the visual indication of the detected object of interest is superimposed at the object position on the map view of the geospatial location.

20. The non-transitory computer-readable medium of claim 15 , wherein the instructions are executable by the processor of the computing device to provide the user interface tool configured to facilitate user validation of one or more of the multiple detections of the object of interest by:

receiving, by way of the user interface, user input selecting the visual indication of the detected object of interest; and

providing, in response to the user input selecting the visual indication of the detected object of interest, a menu of selectable options comprising

a confirmation option selectable by a user to confirm one or more of the multiple detections of the object of interest, and

a rejection option selectable by the user to reject one or more of the multiple detections of the object of interest.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 8, 2019
From: VERIZON LOCATION TECHNOLOGY INC.
To: VERIZON PATENT AND LICENSING INC.
Reel/Frame 050652/0430 →
CHANGE OF NAME Recorded Oct 8, 2019
From: MAPQUEST, INC.
To: VERIZON LOCATION TECHNOLOGY INC.
Reel/Frame 050657/0121 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2018
From: ENO, JOSHUA; WILLIAMS, LINDY; BOYER, WESLEY; BRACKNEY, KEVIN; DAVELL, BERGEN; PHAM, KHOA TRUONG
To: MAPQUEST, INC.
Reel/Frame 047013/0279 →
Continuity (1)
Related Publication 20200104646A1 · Apr 2, 2020