Granted Patent B2

US 9,665,985 · App. 14/461,252 · Granted May 30, 2017

Inventors: Brian Mullins (Sierra Madre, CA); Matthew Kammerait (West Hollywood, CA); Christopher Broaddus (Mountain View, CA)

Assignee: DAQRI, LLC

G06T19/006G06F3/1454G09G3/001G09G2370/16

Abstract

Claims (62)

1. A system comprising:

one or more computer processors; and

one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the system to perform operations comprising:

receiving, from a first wearable device, data including a video feed captured by the first wearable device, the video feed depicting to a physical object viewed with the first wearable device;

comparing the data received from the first wearable device to a three dimensional (3D) model dataset of the physical object, yielding a comparison;

identifying, based on the comparison, an anomaly between the physical object viewed with the first wearable device and the 3D model dataset of the physical object;

identifying, based on the anomaly, a user designated as an expert related to the anomaly;

providing, to a second wearable device used by the user designated as the expert related to the anomaly, virtual objects displayed by the first wearable device, causing the second wearable device to present the virtual objects to the user designated as the expert;

receiving, from the second wearable device, data describing a manipulation of the virtual objects displayed by the second wearable device, the virtual objects displayed in the first wearable device in relation to a reference object viewed with the first wearable device;

generating a visualization of the manipulation of the virtual objects from the second wearable device; and

communicating the visualization of the manipulation of the virtual objects to the first wearable device, causing the first wearable device to present the visualization of the manipulation to a user of the first wearable device.

2. The system of claim 1 , the operations further comprising:

identifying the physical object from the video feed;

generating a 3D model of the virtual objects based on the identification of the physical object; and

communicating the 3D model of the virtual objects to the first wearable device, the first wearable device being configured to render the 3D model of the virtual objects in relation to the physical object.

3. The system of claim 1 , the operations further comprising recording the manipulation of the virtual objects from the second wearable device.

4. The system of claim 1 , wherein the manipulation of the virtual objects is determined by the second wearable device using a visual gesture module, the visual gesture module configured to determine a predefined area of a display of the second wearable device and a feature of the virtual objects, and to change a state of the feature in response to the feature being in the predefined area of the display of the second wearable device.

5. The system of claim 1 , wherein the manipulation of the virtual objects is determined by the second wearable device using a physical gesture module, the physical gesture module configured to determine a physical movement of a finger of the user of the second wearable device relative to the virtual objects displayed by the second wearable device.

6. The system of claim 1 , wherein the visualization includes an animation of the manipulation of the virtual objects.

7. The system of claim 1 , the operations further comprising:

receiving, from the first wearable device, a subsequent manipulation of the virtual objects; and

communicating a visualization of the subsequent manipulation of the virtual objects to the second wearable device, causing the second wearable device to present the subsequent manipulation to the user designated as the expert.

8. The system of claim 1 , the operations further comprising:

receiving video feeds from a plurality of wearable devices, the video feeds depicting the physical object;

generating a 3D model of the physical object based on the video feeds received from the plurality of wearable devices; and

updating the 3D model of physical object based on manipulations of the physical object identified from the video feeds.

9. A method comprising:

receiving, from a first wearable device, data including a video feed captured by the first wearable device, the video feed depicting a physical object viewed with the first wearable device;

comparing the data received from the first wearable device to a three dimensional (3D) model dataset of the physical object, yielding a comparison;

identifying, based on the comparison, an anomaly between the physical object viewed with the first wearable device and the 3D model dataset of the physical object;

identifying, based on the anomaly, a user designated as an expert related to the anomaly;

providing, to a second wearable device used by the user designated as the expert related to the anomaly, virtual objects displayed by the first wearable device, causing the second wearable device to present the virtual objects to the user designated as the expert;

receiving, from the second wearable device, data describing a manipulation of the virtual objects displayed by the second wearable device, the virtual objects displayed in the first wearable device in relation to a reference object viewed with the first wearable device;

generating a visualization of the manipulation of the virtual objects from the second wearable device; and

communicating the visualization of the manipulation of the virtual objects to the first wearable device, causing the first wearable device to present the visualization of the manipulation to a user of the first wearable device.

10. The method of claim 9 , further comprising:

identifying the physical object from the video feed;

generating a 3D model of the virtual objects based on the identification of the physical object; and

communicating the 3D model of the virtual objects to the first wearable device, the first wearable device being configured to render the 3D model of the virtual objects in relation to the physical object.

11. The method of claim 9 , further comprising:

recording the manipulation of the virtual objects from the second wearable device.

12. The method of claim 9 , wherein

the manipulation of the virtual objects is determined by the second wearable device using a visual gesture module, the visual gesture module configured to determine a predefined area of a display of the second wearable device and a feature of the virtual objects, and to change a state of the feature in response to the feature being in the predefined area of the display of the second wearable device.

13. The method of claim 9 , wherein

the manipulation of the virtual objects is determined by the second wearable device using a physical gesture module, the physical gesture module configured to determine a physical movement of a finger of the user of the second wearable device relative to the virtual objects displayed by the second wearable device.

14. The method of claim 9 , wherein the visualization includes an animation of the manipulation of the virtual objects.

15. The method of claim 9 , the method further comprising:

receiving, from the first wearable device, a subsequent manipulation of the virtual objects; and

communicating a visualization of the subsequent manipulation of the virtual objects to the second wearable device, causing the second wearable device to present the subsequent manipulation to the user designated as the expert.

16. The method of claim 9 , further comprising:

receiving video feeds from a plurality of wearable devices, the video feeds depicting the physical object;

generating a 3D model of the physical object based on the video feeds received from the plurality of wearable devices; and

updating the 3D model of the physical object based on manipulations of the physical object identified from the video feeds.

17. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving, from a first wearable device, data including a video feed captured by the first wearable device, the video feed depicting a physical object viewed with the first wearable device;

comparing the data received from the first wearable device to a three dimensional (3D) model dataset of the physical object, yielding a comparison;

identifying, based on the comparison, an anomaly between the physical object viewed with the first wearable device and the 3D model dataset of the physical object;

identifying, based on the anomaly, a user designated as an expert related to the anomaly;

providing, to a second wearable device used by the user designated as the expert related to the anomaly, virtual objects displayed by the first wearable device, causing the second wearable device to present the virtual objects to the user designated as the expert;

receiving, from the second wearable device, data describing a manipulation of the virtual objects displayed by the second wearable device, the virtual objects displayed in the first wearable device in relation to a reference object viewed with the first wearable device;

generating a visualization of the manipulation of the virtual objects from the second wearable device; and

communicating the visualization of the manipulation of the virtual objects to the first wearable device, causing the first wearable device to present the visualization of the manipulation to a user of the first wearable device.

Assignments (10)

Continuity (1)