Granted Patent B2

US 10,347,046 · App. 15/625,229 · Granted Jul 9, 2019

Inventor: Brian Mullins (Altadena, CA)

Assignee: DAQRI, LLC

G06T19/003G01C21/3647G01C21/3661G02B27/0101G02B27/0172G06T19/006G06T19/20G02B2027/0178G06T2219/2016

Abstract

Claims (98)

1. A first augmented-reality (AR) device comprising:

an optical sensor;

a geographic location sensor;

an orientation sensor a display; and

one or more hardware processors comprising an AR transport application, the AR transport application configured to perform operations comprising:

determining, using the geographic location sensor and the orientation sensor, a first geographic location of the first AR device and an orientation of the first AR device;

generating, using the optical sensor, a picture at the first geographic location of the first AR device and associated with the orientation of the first AR device;

retrieving, from a server, transportation information from a second AR device in a vehicle, the server being configured to assign the second AR device to the first AR device;

generating transportation AR content based on the transportation information from the second AR device; and

displaying the transportation AR content in the display of the first AR device based on the first geographic location of the first AR device, the orientation of the first AR device, the transportation information, and the picture generated at the first geographic location of the first AR device and associated with the orientation of the first AR device,

wherein the transportation information includes:

transportation profile data identifying the vehicle and a user of the second AR device, and

transportation travel data identifying a speed, a travel direction, and an estimated travel time from a second geographic location of the second AR device to the first geographic location of the first AR device,

wherein the transportation AR content comprises vehicle AR content generated based on the transportation profile data, and travel path AR content generated based on the transportation travel data,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

displaying the vehicle AR content as a first overlay to the picture based on the second geographic location of the second AR device, the vehicle AR content identifying the vehicle and the user of the second AR device; and

displaying the travel path AR content as a second overlay to the picture based on the second geographic location, the speed, and the travel direction of the second AR device, the travel path AR content identifying a planned travel route of the vehicle on the picture.

2. The first AR device of claim 1 , wherein the transportation AR content comprises vehicle AR content generated based on the transportation profile data, and travel path AR content generated based on the transportation travel data, wherein the display includes a transparent display,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

displaying the vehicle AR content in the transparent display based on the second geographic location of the second AR device, the vehicle AR content including a three-dimensional model of a virtual vehicle representing a color and model of the vehicle and a picture of the user of the second AR device; and

displaying the travel path AR content in the transparent display based on the second geographic location, the speed, and the travel direction of the second AR device, the travel path AR content including a virtual path identifying a planned travel route of the vehicle.

3. The first AR device of claim 1 , wherein the transportation AR content comprises vehicle AR content identifying the vehicle and the user of the second AR device,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

scaling a size of the vehicle AR content based on a distance between the first AR device and the second AR device; and

displaying the scaled vehicle AR content in the display based on the second geographic location of the second AR device relative to the first geographic location and orientation of the first AR device.

4. The first AR device of claim 1 , wherein the transportation AR content comprises travel path AR content identifying a route, a speed, a travel direction of the second AR device, and an estimated travel time from a second geographic location of the second AR device to the first geographic location of the first AR device,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

accessing the route of the second AR device from the server;

updating the second geographic location of the second AR device;

updating the route based on the updated second geographic location of the second AR device;

updating the travel path AR content based on the updated route; and

displaying the updated travel path AR content in the display, the updated travel path AR content including a virtual path between the first and second AR devices based on the updated route.

5. The first AR device of claim 1 , wherein the operations further comprise:

receiving a request from the server, the request identifying an alternate geographic location for the first AR device;

determining alternate location AR content identifying the alternate geographic location based on the first geographic location and orientation of the first AR device; and

displaying the alternate location AR content in the display, the alternate location AR content appearing coupled to a corresponding physical object depicted in the picture.

6. The first AR device of claim 5 , wherein the operations further comprise:

generating pedestrian path AR content identifying a walking path from the first geographic location of the first AR device to the alternate geographic location for the first AR device; and

displaying the pedestrian path AR content in the display, the pedestrian path AR content appearing coupled to a sidewalk depicted in the picture.

7. The first AR device of claim 6 , wherein the operations further comprise:

generating travel path AR content identifying a planned travel path of the second AR device from the first geographic location of the second AR device to the alternate geographic location for the first AR device; and

displaying the travel path AR content in the display, the travel path AR content appearing coupled to a road depicted in the picture.

8. The first AR device of claim 1 , wherein the operations further comprise:

receiving an identification of a third geographic location in the display from a user of the first AR device, the third geographic location identified in the picture generated at the first geographic location of the first AR device and associated with the orientation of the first AR device;

generating a request for the alternate geographic location to the server based on the third geographic location;

generating travel path AR content identifying a planned travel path of the second AR device from a second geographic location of the second AR device to the third geographic location; and

displaying the travel path AR content in the display.

9. A method comprising:

determining, using a geographic location sensor and an orientation sensor of a first AR device, a first geographic location of the first AR device and an orientation of the first AR device;

generating, using an optical sensor of the first AR device, a picture at the first geographic location of the first AR device and associated with the orientation of the first AR device;

retrieving, from a server, transportation information from a second AR device in a vehicle, the server being configured to assign the second AR device to the first AR device;

generating transportation AR content based on the transportation information from the second AR device; and

displaying the transportation AR content in a display of the first AR device based on the first geographic location of the first AR device, the orientation of the first AR device, the transportation information, and the picture generated at the first geographic location of the first AR device and associated with the orientation of the first AR device,

wherein the transportation information includes:

transportation profile data identifying the vehicle and a user of the second AR device, and

transportation travel data identifying a speed, a travel direction, and an estimated travel time from a second geographic location of the second AR device to the first geographic location of the first AR device,

wherein the transportation AR content comprises vehicle AR content generated based on the transportation profile data, and travel path AR content generated based on the transportation travel data,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

displaying the vehicle AR content as a first overlay to the picture based on the second geographic location of the second AR device, the vehicle AR content identifying the vehicle and the user of the second AR device; and

displaying the travel path AR content as a second overlay to the picture based on the second geographic location, the speed, and the travel direction of the second AR device, the travel path AR content identifying a planned travel route of the vehicle on the picture.

10. The method of claim 9 , wherein the transportation AR content comprises vehicle AR content generated based on the transportation profile data, and travel path AR content generated based on the transportation travel data, wherein the display includes a transparent display,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

displaying the vehicle AR content in the transparent display based on the second geographic location of the second AR device, the vehicle AR content including a three-dimensional model of a virtual vehicle representing a color and model of the vehicle and a picture of the user of the second AR device; and

displaying the travel path AR content in the transparent display based on the second geographic location, the speed, and the travel direction of the second AR device, the travel path AR content including a virtual path identifying a planned travel route of the vehicle.

11. The method of claim 9 , wherein the transportation AR content comprises vehicle AR content identifying the vehicle and the user of the second AR device,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

scaling a size of the vehicle AR content based on a distance between the first AR device and the second AR device; and

displaying the scaled vehicle AR content in the display based on the second geographic location of the second AR device relative to the first geographic location and orientation of the first AR device.

12. The method of claim 9 , wherein the transportation AR content comprises travel path AR content identifying a route, a speed, a travel direction of the second AR device, and an estimated travel time from a second geographic location of the second AR device to the first geographic location of the first AR device,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

accessing the route of the second AR device from the server;

updating the second geographic location of the second AR device;

updating the route based on the updated second geographic location of the second AR device;

updating the travel path AR content based on the updated route; and

displaying the updated travel path AR content in the display, the updated travel path AR content including a virtual path between the first and second AR devices based on the updated route.

13. The method of claim 9 , wherein the operations further comprise:

receiving a request from the server, the request identifying an alternate geographic location for the first AR device;

determining alternate location AR content identifying the alternate geographic location based on the first geographic location and orientation of the first AR device; and

displaying the alternate location AR content in the display, the alternate location AR content appearing coupled to a corresponding physical object depicted in the picture.

14. The method of claim 13 , wherein the operations further comprise:

generating pedestrian path AR content identifying a walking path from the first geographic location of the first AR device to the alternate geographic location for the first AR device; and

displaying the pedestrian path AR content in the display, the pedestrian path AR content appearing coupled to a sidewalk depicted in the picture.

15. The method of claim 14 , wherein the operations further comprise:

generating travel path AR content identifying a planned travel path of the second AR device from the first geographic location of the second AR device to the alternate geographic location for the first AR device; and

displaying the travel path AR content in the display, the travel path AR content appearing coupled to a road depicted in the picture.

16. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

determining, using a geographic location sensor and an orientation sensor of a first AR device, a first geographic location of the first AR device and an orientation of the first AR device;

generating, using an optical sensor of the first AR device, a picture at the first geographic location of the first AR device and associated with the orientation of the first AR device;

retrieving, from a server, transportation information from a second AR device in a vehicle, the server being configured to assign the second AR device to the first AR device;

generating transportation AR content based on the transportation information from the second AR device; and

displaying the transportation AR content in a display of the first AR device based on the first geographic location of the first AR device, the orientation of the first AR device, the transportation information, and the picture generated at the first geographic location of the first AR device and associated with the orientation of the first AR device,

wherein the transportation information includes:

transportation profile data identifying the vehicle and a user of the second AR device, and

transportation travel data identifying a speed, a travel direction, and an estimated travel time from a second geographic location of the second AR device to the first geographic location of the first AR device,

wherein the transportation AR content comprises vehicle AR content generated based on the transportation profile data, and travel path AR content generated based on the transportation travel data,

wherein displaying the transportation AR content in the display of the first AR device further comprises:

displaying the vehicle AR content as a first overlay to the picture based on the second geographic location of the second AR device, the vehicle AR content identifying the vehicle and the user of the second AR device; and

displaying the travel path AR content as a second overlay to the picture based on the second geographic location, the speed, and the travel direction of the second AR device, the travel path AR content identifying a planned travel route of the vehicle on the picture.

Assignments (12)

Continuity (1)

Cited By (1)

US 12,332,060