Generating augmented reality images for display on a mobile device based on ground truth image rendering

1 . A computer-implemented method for generating augmented reality elements associated with a transportation service, the method comprising:

receiving sensor data from a client device of a user of the transportation service;

determining a current location of the client device of the user based on the sensor data;

monitoring a location of a client device of a driver navigating to a pickup location;

comparing the locations of the client device of the user and the client device of the driver; and

responsive to determining that the client device of the user and the client device of the driver are within a threshold distance of each other, instructing the client device of the user and the client device of the driver to display mutual augmented reality elements associated with the transportation service.

2 . The computer-implemented method of claim 1 , wherein determining a current location of the client device of the user based on the sensor data comprises comparing renderings associated with the received sensor data with candidate renderings associated with sensor data stored in a database in association with respective locations.

3 . The computer-implemented method of claim 2 , wherein the sensor data comprises image data automatically captured by a camera on the client device of the user.

4 . The computer-implemented method of claim 3 , further comprising:

extracting geolocation data from a rendering of the captured image data;

determining a subset of entries in the database corresponding to the geolocation data; and

limiting a comparison of the renderings of the captured image data to the subset of entries.

5 . The computer-implemented method of claim 1 , further comprising:

identifying, in a video feed displayed on the client device of the user, a visual indicator of a landmark;

generating an augmented reality element identifying the visual indicator; and

instructing the client device of the user to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

6 . The computer-implemented method of claim 1 , further comprising:

identifying, in a video feed displayed on the client device of the user, a visual indicator of the pickup location;

generating an augmented reality element identifying the visual indicator; and

instructing the client device of the user to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

7 . The computer-implemented method of claim 1 , further comprising:

determining that a landmark associated with navigation instructions is within a line of sight of a camera of the client device of the user; and

generating for display an augmented reality element identifying the landmark and instructing a user of the client device of the user to perform an action upon reaching the landmark.

8 . The computer-implemented method of claim 1 , further comprising:

designating a specified number of pickup locations for transportation service, the pickup locations associated with landmarks located within a threshold distance of each other;

receiving a plurality of requests for transportation service from a plurality of client devices having current locations within a threshold distance of the pickup locations;

determining that the plurality of requests exceeds the specified number of pickup locations; and

assigning the plurality of requests to the pickup locations in an order in which the requests were received, wherein a first number of requests corresponding to the number of pickup locations are assigned to the pickup locations and wherein a second number of requests are placed in a transportation request queue.

9 . The computer-implemented method of claim 8 , further comprising:

determining that an occupied pickup location has been vacated;

determining a next request in the transportation request queue; and

assigning the next request to the vacated pickup location.

10 . The computer-implemented method of claim 1 , the sensor data captured automatically upon occurrence of a capture condition.

11 . The computer-implemented method of claim 10 , the capture condition including at least one of a certain distance traveled or a certain amount of time passed since a last captured image, a certain distance has been traveled or a certain amount of time has passed from a reference point or since a previous location of the client device of the user has been determined, an endpoint of a first navigation instruction has been reached, or a direction of the client device of the user has changed.

12 . A non-transitory computer-readable medium comprising memory with instructions encoded thereon for generating augmented reality elements associated with a transportation service, the instructions, when executed by one or more processors, causing the one or more processors to perform operations comprising:

receiving sensor data from a client device of a user of the transportation service;

determining a current location of the client device of the user based on the sensor data;

monitoring a location of a client device of a driver navigating to a pickup location;

comparing the locations of the client device of the user and the client device of the driver; and

responsive to determining that the client device of the user and the client device of the driver are within a threshold distance of each other, instructing the client device of the user and the client device of the driver to display mutual augmented reality elements associated with the transportation service.

13 . The non-transitory computer-readable medium of claim 12 , wherein determining a current location of the client device of the user based on the sensor data comprises comparing renderings associated with the received sensor data with candidate renderings associated with sensor data stored in a database in association with respective locations.

14 . The non-transitory computer-readable medium of claim 13 , wherein the sensor data comprises image data automatically captured by a camera on the client device of the user.

15 . The non-transitory computer-readable medium of claim 12 , further comprising:

identifying, in a video feed displayed on the client device of the user, a visual indicator of a landmark;

generating an augmented reality element identifying the visual indicator; and

instructing the client device of the user to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

16 . The non-transitory computer-readable medium of claim 12 , further comprising:

identifying, in a video feed displayed on the client device of the user, a visual indicator of the pickup location;

generating an augmented reality element identifying the visual indicator; and

instructing the client device of the user to place the augmented reality element identifying the visual indicator on a portion of the video feed where the visual indicator is located.

17 . The non-transitory computer-readable medium of claim 12 , further comprising:

determining that a landmark associated with navigation instructions is within a line of sight of a camera of the client device of the user;

generating for display an augmented reality element identifying the landmark; and

instructing a user of the client device of the user to perform an action upon reaching the landmark.

18 . The non-transitory computer-readable medium of claim 12 , wherein the operations further comprise:

designating a specified number of pickup locations for transportation service, the pickup locations associated with landmarks located within a threshold distance of each other;

receiving a plurality of requests for transportation service from a plurality of client devices having current locations within a threshold distance of the pickup locations;

determining that the plurality of requests exceeds the specified number of pickup locations; and

assigning the plurality of requests to the pickup locations in an order in which the requests were received, wherein a first number of requests corresponding to the number of pickup locations are assigned to the pickup locations and wherein a second number of requests are placed in a transportation request queue.

19 . The non-transitory computer-readable medium of claim 18 , wherein the operations further comprise:

determining that an occupied pickup location has been vacated;

determining a next request in the transportation request queue; and

assigning the next request to the vacated pickup location.

20 . A computer system comprising non-transitory computer-readable medium comprising memory with instructions encoded thereon for generating augmented reality elements associated with a transportation service, the instructions, when executed by one or more processors, causing the one or more processors to perform operations comprising:

receiving sensor data from a client device of a user of the transportation service;

determining a current location of the client device of the user based on the sensor data;

monitoring a location of a client device of a driver navigating to a pickup location;

comparing the locations of the client device of the user and the client device of the driver; and

responsive to determining that the client device of the user and the client device of the driver are within a threshold distance of each other, instructing the client device of the user and the client device of the driver to display mutual augmented reality elements associated with the transportation service.