Granted Patent B2

US 11,501,524 · App. 16/712,821 · Granted Nov 15, 2022

Inventors: Aaron Matthew Rogan (Westminster, CO); Taehun Yoon (Superior, CO); Gregory Mark Mahowald (Broomfield, CO)

Assignee: Uber Technologies, Inc.

G06V20/00G01C21/343G01C21/3438G01C21/3644G01S19/48G06F3/016G06F16/587G06F16/5866G06F16/7837G06K9/6201G06K9/6215G06Q10/02G06Q10/047G06Q30/0209G06Q30/0235G06Q50/30G06T19/006G06T19/20G06V20/20G06V20/56H04B1/3827H04L67/52H04N5/23222H04W4/021H04W4/029H04W4/185G06T2200/24G06T2219/024

Abstract

Claims (87)

1. A computer-implemented method for generating augmented reality elements associated with a transportation service, the method comprising:

receiving a request from a client device of a user for navigation instructions to navigate to a pickup location associated with the transportation service;

in response to receiving the request, instructing the client device to capture an image;

determining a current location of the client device based on a comparison between a rendering of the captured image and a plurality of entries in a database of renderings, each respective entry including a respective rendering and a respective associated location;

generating navigation instructions from the current location of the client device to the pickup location;

generating for display an augmented reality element associated with the navigation instructions; and

instructing the client device to visually augment a video feed being captured by a camera of the client device with the augmented reality element.

2. The method of claim 1 , wherein determining the current location comprises:

determining, based on the comparing, that the captured image matches at least one of the respective renderings included in the entries; and

determining the current location of the client device to be the location associated with the matching rendering.

3. The method of claim 2 , further comprising:

extracting geolocation data from the rendering of the captured image;

determining a subset of the entries corresponding to the geolocation data; and

limiting the comparing of the rendering of the captured image to the subset of the entries.

4. The method of claim 2 , further comprising:

monitoring a location of a client device of a driver navigating to the pickup location;

comparing the locations of the client device of the user and the client device of the driver; and

responsive to determining that the client device of the user and the client device of the driver are within a threshold distance of each other, instructing the client device of the user and the client device of the driver to display mutual augmented reality elements associated with a transportation service.

5. The method of claim 1 , wherein generating for display an augmented reality element associated with the navigation instructions comprises:

identifying, in the video feed, a visual indicator of a landmark;

generating an augmented reality element identifying the visual indicator; and

instructing the client device to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

6. The method of claim 1 , wherein generating for display an augmented reality element associated with the navigation instructions comprises:

identifying, in the video feed, a visual indicator of the pickup location;

generating an augmented reality element identifying the visual indicator; and

instructing the client device to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

7. The method of claim 1 , wherein instructing the client device to visually augment the video feed with the augmented reality element comprises:

determining that a landmark associated with the navigation instructions is within a line of sight of a camera of the client device; and

generating for display an augmented reality element identifying the landmark and instructing a user of the client device to perform an action upon reaching the landmark.

8. The method of claim 1 , further comprising:

designating a specified number pickup locations for transportation service, the pickup locations associated with landmarks located within a threshold distance of each other;

receiving a plurality of requests for transportation service from a plurality of client devices having current locations within a threshold distance of the pickup locations;

determining that the plurality of requests exceeds the specified number of pickup locations; and

assigning the requests to the pickup locations in an order in which the requests were received, wherein a first number of requests corresponding to the number of pickup locations are assigned to each respective pickup location and wherein a second number of requests are placed in a transportation request queue.

9. The method of claim 8 , further comprising:

determining that an occupied pickup location has been vacated;

determining a next request in the transportation request queue; and

assigning the next request to the vacated pickup location.

10. A non-transitory computer-readable storage medium storing computer program instructions executable by a processor to perform operations comprising:

receiving a request from a client device of a user for navigation instructions to navigate to a pickup location associated with the transportation service;

in response to receiving the request, instructing the client device to capture an image;

determining a current location of the client device based on a comparison between a rendering of the captured image and a plurality of entries in a database of renderings, each respective entry including a respective rendering and a respective associated location;

generating navigation instructions from the current location of the client device to the pickup location;

generating for display an augmented reality element associated with the navigation instructions; and

instructing the client device to visually augment a video feed being captured by a camera of the client device with the augmented reality element.

11. The non-transitory computer-readable storage medium of claim 10 , wherein determining the current location comprises:

determining, based on the comparing, that the captured image matches at least one of the respective renderings included in the entries; and

determining the current location of the client device to be the location associated with the matching rendering.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the operations further comprise:

extracting geolocation data from the rendering of the captured image;

determining a subset of the entries corresponding to the geolocation data; and

limiting the comparing of the rendering of the captured image to the subset of the entries.

13. The non-transitory computer-readable storage medium of claim 11 , wherein the operations further comprise:

monitoring a location of a client device of a driver navigating to the pickup location;

comparing the locations of the client device of the user and the client device of the driver; and

responsive to determining that the client device of the user and the client device of the driver are within a threshold distance of each other, instructing the client device of the user and the client device of the driver to display mutual augmented reality elements associated with a transportation service.

14. The non-transitory computer-readable storage medium of claim 10 , wherein generating for display an augmented reality element associated with the navigation instructions comprises:

identifying, in the video feed, a visual indicator of a landmark;

generating an augmented reality element identifying the visual indicator; and

instructing the client device to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

15. The non-transitory computer-readable storage medium of claim 10 , wherein generating for display an augmented reality element associated with the navigation instructions comprises:

identifying, in the video feed, a visual indicator of the pickup location;

generating an augmented reality element identifying the visual indicator; and

instructing the client device to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

16. The non-transitory computer-readable storage medium of claim 10 wherein instructing the client device to visually augment the video feed with the augmented reality element comprises:

determining that a landmark associated with the navigation instructions is within a line of sight of a camera of the client device; and

generating for display an augmented reality element identifying the landmark and instructing a user of the client device to perform an action upon reaching the landmark.

17. A system comprising:

a processor for executing computer program instructions; and

a non-transitory computer-readable storage medium storing computer program instructions executable by the processor to perform operations for estimating a location of a client device, the operations comprising:

receiving a request from a client device of a user for navigation instructions to navigate to a pickup location associated with the transportation service;

in response to receiving the request, instructing the client device to capture an image;

determining a current location of the client device based on a comparison between a rendering of the captured image and a plurality of entries in a database of renderings, each respective entry including a respective rendering and a respective associated location;

generating navigation instructions from the current location of the client device to the pickup location;

generating for display an augmented reality element associated with the navigation instructions; and

instructing the client device to visually augment a video feed being captured by a camera of the client device with the augmented reality element.

18. The system of claim 17 , wherein determining the current location comprises:

determining, based on the comparing, that the captured image matches at least one of the candidate images; and

determining the current location of the client device to be the location associated with the matching rendering.

19. The system of claim 18 , wherein the operations further comprise:

monitoring a location of a client device of a driver navigating to the pickup location;

comparing the locations of the client device of the user and the client device of the driver; and

responsive to determining that the client device of the user and the client device of the driver are within a threshold distance of each other, instructing the client device of the user and the client device of the driver to display mutual augmented reality elements associated with a transportation service.

20. The system of claim 17 , wherein generating for display an augmented reality element associated with the navigation instructions comprises:

identifying, in the video feed, a visual indicator of a landmark;

generating an augmented reality element identifying the visual indicator; and

instructing the client device to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

Assignments (1)

Continuity (8)