IP Library Granted Patent US 11,079,242
Granted Patent B2
US 11,079,242 · App. 16/226,353 · Granted Aug 3, 2021

System and method for determining autonomous vehicle location using incremental image analysis

Inventors: Robert L. Cantrell (Herndon, VA); David C. Winkle (Bella Vista, AR); John J. O'Brien (Farmington, AR)
Assignee: Walmart Apollo, LLC
G01C21/32B64C39/024G05D1/0088G06F16/29G06F16/583G06F16/9537G06T7/74B64C2201/141G06T2207/30244G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,079,242
App. No.
16/226,353
Granted
Aug 3, 2021
Kind
B2
Abstract

Systems, methods, and computer-readable storage media for determining autonomous vehicle location using incremental image analysis. An exemplary method can include identifying an expected position of an autonomous vehicle which is moving, and identifying, an actual position of the autonomous vehicle. The identifying of the actual position occurs by obtaining images of the autonomous vehicle's surroundings, initiating an iterative image comparison of those images to previously stored images within a given geographic radius of the autonomous vehicle, and iteratively extending the radius (and the pictures being compared) until a match is found or until the maximum radius is reached.

Claims (62)

1. A method comprising:

identifying, at a central location and via a processor accessing a memory device, an expected position of an autonomous vehicle which is moving; and

identifying, via the processor, an actual position of the autonomous vehicle, by:

transmitting, from the processor at the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising:

surrounding images comprising surroundings of the autonomous vehicle; and

peer GPS (Global Positioning System) data for proximate peers;

receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images;

receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data;

determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination;

initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison:

extends a geographic radius extending from the expected position; and

compares stored images within the geographic radius to the surrounding images,

until the comparison of the stored images to the surrounding images within a given geographic radius yields the actual position with a predetermined certainty.

2. The method of claim 1 , wherein the surrounding images comprise a panoramic image with a 360° view around the autonomous vehicle.

3. The method of claim 1 , wherein the autonomous vehicle is an aerial drone, and the surrounding images comprise an image of ground beneath the aerial drone.

4. The method of claim 1 , wherein the proximate peers comprise a plurality of other autonomous vehicles, and the peer GPS data comprise GPS coordinates of the other autonomous vehicles.

5. The method of claim 1 , wherein the identifying of the actual position of the autonomous vehicle occurs in real-time, with the surrounding images are taken by the autonomous vehicle immediately upon receiving the request, and the initiation of the iterative image comparison begins immediately upon receiving the on-board location information.

6. The method of claim 1 , wherein each iteration of the iterative image comparison further comprises:

filtering landmarks from the stored images which are located outside the geographic radius for a given iteration, to yield filtered images; and

wherein the comparing of the stored images to the surrounding images uses the filtered images.

7. The method of claim 6 , wherein comparing the stored images to the surrounding images using the filtered images is more computationally efficient than comparing the stored images to an entirety of the surrounding images.

8. The method of claim 1 , wherein the on-board location information further comprises distance data corresponding to the surrounding images, the distance data identifying a distance between the autonomous vehicle and an object pictured within the surrounding images.

9. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:

identifying, at a central location, an expected position of an autonomous vehicle which is moving; and

identifying an actual position of the autonomous vehicle, by:

transmitting, from the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising:

surrounding images comprising surroundings of the autonomous vehicle; and

peer GPS (Global Positioning System) data for proximate peers;

receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images;

receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data;

determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination;

initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison:

extends a geographic radius extending from the expected position; and

compares stored images within the geographic radius to the surrounding images,

until the comparison of the stored images to the surrounding images within a given geographic radius yields the actual position with a predetermined certainty.

10. The system of claim 9 , wherein the surrounding images comprise a panoramic image with a 360° view around the autonomous vehicle.

11. The system of claim 9 , wherein the autonomous vehicle is an aerial drone, and the surrounding images comprise an image of ground beneath the aerial drone.

12. The system of claim 9 , wherein the proximate peers comprise a plurality of other autonomous vehicles, and the peer GPS data comprise GPS coordinates of the other autonomous vehicles.

13. The system of claim 9 , wherein the identifying of the actual position of the autonomous vehicle occurs in real-time, with the surrounding images are taken by the autonomous vehicle immediately upon receiving the request, and the initiation of the iterative image comparison begins immediately upon receiving the on-board location information.

14. The system of claim 9 , wherein each iteration of the iterative image comparison further comprises:

filtering landmarks from the stored images which are located outside the geographic radius for a given iteration, to yield filtered images; and

wherein the comparing of the stored images to the surrounding images uses the filtered images.

15. The system of claim 14 , wherein comparing the stored images to the surrounding images using the filtered images is more computationally efficient than comparing the stored images to an entirety of the surrounding images.

16. The system of claim 9 , wherein the on-board location information further comprises distance data corresponding to the surrounding images, the distance data identifying a distance between the autonomous vehicle and an object pictured within the surrounding images.

17. A non-transitory computer-readable storage medium having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

identifying, at a central location, an expected position of an autonomous vehicle which is moving; and

identifying an actual position of the autonomous vehicle, by:

transmitting, at the central location to the autonomous vehicle, a request for on-board location information, the on-board location information comprising:

surrounding images comprising surroundings of the autonomous vehicle; and

peer GPS (Global Positioning System) data for proximate peers;

receiving, at the central location from the autonomous vehicle and in response to the request, the surrounding images;

receiving, at the central location from the autonomous vehicle and in response to the request, the peer GPS data;

determining, based on the peer GPS data in the on-board location information, that the actual position of the autonomous vehicle is outside a threshold range of the expected position, to yield a determination;

initiating, based on the determination, an iterative image comparison, wherein each iteration of the iterative image comparison:

extends a geographic radius extending from the expected position; and

compares stored images within the geographic radius to the surrounding images,

until the comparison of the stored images to the surrounding images within a given geographic radius yields the actual position with a predetermined certainty.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the surrounding images comprise a panoramic image with a 360° view around the autonomous vehicle.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the autonomous vehicle is an aerial drone, and the surrounding images comprise an image of ground beneath the aerial drone.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the proximate peers comprise a plurality of other autonomous vehicles, and the peer GPS data comprise GPS coordinates of the other autonomous vehicles.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2021
From: CANTRELL, ROBERT L.; WINKLE, DAVID C.; O'BRIEN, JOHN J.
To: WALMART APOLLO, LLC
Reel/Frame 056602/0891 →
Continuity (2)
Provisional Application 62611773 · Dec 29, 2017
Related Publication 20190204093A1 · Jul 4, 2019