Granted Patent B1

US 9,865,093 · App. 15/199,570 · Granted Jan 9, 2018

Inventor: Brian Mullins (Altadena, CA)

Assignee: DAQRI, LLC

G06T19/006G02B27/0172G06F3/012G06F3/1423G06F3/1454G06T19/20G02B2027/014G06T2219/024

Abstract

Claims (79)

1. A first display device comprising:

one or more hardware processors; and

a memory storing instructions that, when executed by the one or more hardware processors, configure the first display device to perform operations comprising:

determining a first device configuration of the first display device and a first context of a first AR application operating on the first display device;

connecting to a second display device;

determining a second device configuration of the second display device and a second context of a second AR application operating on the second display device;

generating a first collaboration configuration for the first display device and a second collaboration configuration for the second display device, the first and second collaboration configurations being generated based on the first device configuration, the first context, the second device configuration, and the second context, the second display device being configured to modify an operation of the second AR application based on the second collaboration configuration; and

modifying an operation of the first AR application in the first display device based on the first collaboration configuration.

2. The display device of claim 1 , wherein the operations further comprise:

identifying a common process executed by the first and second display devices based on a common operation identified in the first and second configurations and in the first and second contexts;

determining that the first display device is configured to perform the common process more efficiently than the second display device;

assigning the common process to the first display device in the first collaboration configuration; and

removing the common process from the second display device in the second collaboration configuration.

3. The display device of claim 1 , wherein the modifying of the operation of the first AR application in the first display device comprises:

disabling a first display of the first display device; and

causing a second display of the second display device to display content from the first AR application using the second collaboration configuration on the second AR application of the second display device.

4. The display device of claim 1 , wherein the operations further comprise:

determining that a first task of a first user of the first display device and a second task of a second user of the second display device are part of a general workflow;

generating first content and a first user interface for the first user of the first display device based on the first task, the first collaboration configuration including the first content and the first user interface; and

generating a second content and a second user interface for the second user of the first display device based on the second task, the second collaboration configuration including the second content and the second user interface.

5. The display device of claim 1 , wherein the first configuration includes hardware and software specifications of the first display device, and hardware and software resource capacities and availabilities.

6. The display device of claim 1 , wherein the first context includes a user-based context and an ambient-based context of the display device,

the user-based context being generated based on a user profile of a first user of the first display device, a user credential of the first user, and biometric data from the first user, the biometric data including at least one of a heart rate, a blood pressure, or brain activity,

the ambient-based context indicating at least one of a geographic location of the display device, an orientation and a position of the first display device, an ambient pressure, an ambient humidity level, or an ambient light level.

7. The display device of claim 1 , wherein the first collaboration configuration includes allocation instructions of operations and tasks from the first AR application and the second AR application between the first display device and the second display device.

8. The display device of claim 1 , further comprising:

a display; and

a plurality of sensors including a first set of sensors configured to measure first sensor data related to a user of the HMD, and a second set of sensors configured to measure second sensor data related to the HMD,

wherein the operations further comprise:

determining a third context including a user-based context based on the first set of sensors,

determining a fourth context including an ambient-based context based on the second set of sensors, and

generating AR content based on the third context and the fourth context, the first AR application being configured to display the AR content in the display.

9. The display device of claim 8 , wherein the operations further comprises:

modifying the AR content based on the first collaboration configuration; and

causing a display of the modified AR content in the display of the first display device.

10. The display device of claim 8 , wherein the operations further comprise:

determining a first authorization level of the first user based on the third context and the fourth context; and

determining a second authorization level of a second user of the second display device based on a fifth context including a user-based context of the second user and a sixth context including an ambient-based context of the second display device,

wherein the first and second collaboration configurations are based on the first authorization level of the first user and the second authorization level of the second user.

11. A method comprising:

determining a first device configuration of a first display device and a first context of a first AR application operating on the first display device;

connecting to a second display device;

determining a second device configuration of the second display device and a second context of a second AR application operating on the second display device;

generating a first collaboration configuration for the first display device and a second collaboration configuration for the second display device, the first and second collaboration configurations being generated based on the first device configuration, the first context, the second device configuration, and the second context, the second display device being configured to modify an operation of the second AR application based on the second collaboration configuration; and

modifying an operation of the first AR application in the first display device based on the first collaboration configuration.

12. The method of claim 11 , further comprising:

identifying a common process executed by the first and second display devices based on a common operation identified in the first and second configurations and in the first and second contexts;

determining that the first display device is configured to perform the common process more efficiently than the second display device;

assigning the common process to the first display device in the first collaboration configuration; and

removing the common process from the second display device in the second collaboration configuration.

13. The method of claim 11 , wherein the modifying of the operation of the first AR application in the first display device comprises:

disabling a first display of the first display device; and

causing a second display of the second display device to display content from the first AR application using the second collaboration configuration on the second AR application of the second display device.

14. The method of claim 11 , further comprising:

determining that a first task of a first user of the first display device and a second task of a second user of the second display device are part of a general workflow;

generating first content and a first user interface for the first user of the first display device based on the first task, the first collaboration configuration including the first content and the first user interface; and

generating a second content and a second user interface for the second user of the first display device based on the second task, the second collaboration configuration including the second content and the second user interface.

15. The method of claim 11 , wherein the first configuration includes hardware and software specifications of the first display device, and hardware and software resource capacities and availabilities.

16. The method of claim 11 , wherein the first context includes user-based context and ambient-based context of the first display device,

the user-based context being generated based on a user profile of a first user of the first display device, a user credential of the first user, and biometric data from the first user, the biometric data including at least one of a heart rate, a blood pressure, or brain activity,

the ambient-based context indicating at least one of a geographic location of the display device, an orientation and a position of the first display device, an ambient pressure, an ambient humidity level, or an ambient light level.

17. The method of claim 11 , wherein the first collaboration configuration includes allocation instructions of operations and tasks from the first AR application and the second AR application between the first display device and the second display device.

18. The method of claim 11 , wherein the first display device comprises a display, and a plurality of sensors including a first set of sensors configured to measure first sensor data related to a user of the HMD, and a second set of sensors configured to measure second sensor data related to the HMD,

wherein the method further comprises:

determining a third context including a user-based context based on the first set of sensors,

determining a fourth context including an ambient-based context based on the second set of sensors,

generating AR content based on the third context and the fourth context, the first AR application being configured to display the AR content in the display,

modifying the AR content based on the first collaboration configuration; and

causing a display of the modified AR content in the display of the first display device.

19. The method of claim 18 , further comprising:

determining a first authorization level of the first user based on the third context and the fourth context; and

determining a second authorization level of a second user of the second display device based on a fifth context including a user-based context of the second user and a sixth context including an ambient-based context of the second display device,

wherein the first and second collaboration configurations are based on the first authorization level of the first user and the second authorization level of the second user.

20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

determining a first device configuration of a first display device and a first context of a first AR application operating on the first display device;

connecting to a second display device;

determining a second device configuration of the second display device and a second context of a second AR application operating on the second display device;

generating a first collaboration configuration for the first display device and a second collaboration configuration for the second display device, the first and second collaboration configurations being generated based on the first device configuration, the first context, the second device configuration, and the second context, the second display device being configured to modify an operation of the second AR application based on the second collaboration configuration; and

modifying an operation of the first AR application in the first display device based on the first collaboration configuration.

Assignments (12)