Granted Patent B2

US 10,755,416 · App. 15/711,317 · Granted Aug 25, 2020

Inventor: Brian Mullins (Altadena, CA)

G06T7/149G06T7/70G06T19/006

Abstract

Claims (51)

1. A server comprising:

a storage device storing instructions; and

a hardware processor communicatively coupled to the storage device and configured by the instructions to:

receive video feeds, location information, and orientation information from a plurality of wearable devices;

generate a three-dimensional model of an object based on the video feeds, the location information, and the orientation information received from the plurality of wearable devices;

perform analytics from the video feeds to identify a manipulation on the object, to update the three-dimensional model of the object based on the manipulation on the object, and to generate a dynamic status of the object based on the manipulation on the object with respect to reference data related to the object, the reference data comprising a reference object, the dynamic status comprising identifying a completion status of the object based on a three-dimensional model comparison with respect to the reference object;

keep an inventory of components of the object;

generate a history of manipulations of the components;

receive, from a client, a request for a location of a component related to the manipulation on the object; and

identify and communicate the location of the component relative to the reference data to the client.

2. The server of claim 1 , wherein the manipulation of the object comprises a modification of an existing component on the object, an addition of a new component to the object, or a removal of an existing component on the object.

3. The server of claim 1 , wherein the hardware processor is further configured by the instructions to perform object recognition on the video feeds actively or against the reference data to identify a particular component on the object or within a scene or a periphery from a wearable device.

4. The server of claim 1 , wherein the dynamic status comprises an identification of a type of manipulation on the object, an identification of a tool used in the manipulation on the object, a location of the manipulation relative to the three-dimensional model of the object, and an identification of the wearable device associated with the manipulation on the object.

5. The server of claim 1 , wherein the hardware processor is further configured by the instructions to update the dynamic status related to the manipulation on the object.

6. The server of claim 1 , wherein the hardware processor is further configured by the instructions to enable communication between two or more wearable devices determined based on the manipulation of the object.

7. The server of claim 6 , wherein to enable communication between two or more wearable devices determined based on the manipulation of the object, the hardware processor is configured by the instructions to identify a non-compliance of a first manipulation on the object, to identify a first wearable device corresponding to the first manipulation on the object, to identify a second wearable device associated with a second manipulation dependent on the first manipulation, and to notify the second wearable device of the non-compliance of the first manipulation on the object from the first wearable device.

8. The server of claim 6 , wherein to enable communication between two or more wearable devices determined based on the manipulation of the object, the hardware processor is configured by the instructions to identify a first wearable device corresponding to a first manipulation on the object, to identify a second wearable device associated with a second manipulation similar to the first manipulation, and to enable communication between the second wearable device and the first wearable device in response to receiving a request for assistance on the first manipulation on the object from the first wearable device.

9. The server of claim 1 , wherein the hardware processor is configured by the instructions to:

receive a request from the plurality of wearable devices to offload at least one of a tracking process and an augmented reality rendering process, the augmented reality rendering process based on an augmented reality database;

generate offloaded processed data based on the request and the location and the orientation of the wearable device; and

stream the offloaded processed data to the plurality of wearable devices in response to the request, the plurality of wearable devices configured to generate a visualization of the offloaded processed data in a corresponding display of the plurality of wearable devices.

10. A method comprising:

receiving video feeds, location information, and orientation information from a plurality of wearable devices;

generating a three-dimensional model of an object based on the video feeds, the location information, and the orientation information received from the plurality of wearable devices;

performing analytics from the video feeds to identify a manipulation on the object, to update the three-dimensional model of the object based on the manipulation on the object, and to generate a dynamic status of the object based on the manipulation on the object with respect to reference data related to the object, the reference data comprising a reference object, the dynamic status comprising identifying a completion status of the object based on a three-dimensional model comparison with respect to the reference object;

keeping an inventory of components of the object;

generating a history of manipulations of the components;

receiving, from a client, a request for a location of a component related to the manipulation on the object; and

identifying and communicating the location of the component relative to the reference data to the client.

11. The method of claim 10 , wherein the manipulation of the object comprises a modification of an existing component on the object, an addition of a new component to the object, or a removal of an existing component on the object.

12. The method of claim 10 , further comprising performing object recognition on the video feeds actively or against the reference data to identify a particular component on the object or within a scene or a periphery from a wearable device.

13. The method of claim 10 , wherein the dynamic status comprises an identification of a type of manipulation on the object, an identification of a tool used in the manipulation on the object, a location of the manipulation relative to the three-dimensional model of the object, and an identification of the wearable device associated with the manipulation on the object.

14. The method of claim 10 , further comprising update the dynamic status related to the manipulation on the object.

15. The method of claim 10 , further comprising enabling communication between two or more wearable devices determined based on the manipulation of the object.

16. The method of claim 15 , wherein the enabling communication between two or more wearable devices determined based on the manipulation of the object comprises:

identifying a non-compliance of a first manipulation on the object;

identifying a first wearable device corresponding to the first manipulation on the object;

identifying a second wearable device associated with a second manipulation dependent on the first manipulation; and

notifying the second wearable device of the non-compliance of the first manipulation on the object from the first wearable device.

17. The method of claim 15 , wherein the enabling communication between two or more wearable devices determined based on the manipulation of the object comprises:

identifying a first wearable device corresponding to a first manipulation on the object;

identifying a second wearable device associated with a second manipulation similar to the first manipulation; and

enabling communication between the second wearable device and the first wearable device in response to receiving a request for assistance on the first manipulation on the object from the first wearable device.

18. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving video feeds, location information, and orientation information from a plurality of wearable devices;

generating a three-dimensional model of an object based on the video feeds, the location information, and the orientation information received from the plurality of wearable devices;

performing analytics from the video feeds to identify a manipulation on the object, to update the three-dimensional model of the object based on the manipulation on the object, and to generate a dynamic status of the object based on the manipulation on the object with respect to reference data related to the object, the reference data comprising a reference object, the dynamic status comprising identifying a completion status of the object based on a three-dimensional model comparison with respect to the reference object;

keeping an inventory of components of the object;

generating a history of manipulations of the components;

receiving, from a client, a request for a location of a component related to the manipulation on the object; and

identifying and communicating the location of the component relative to the reference data to the client.

Assignments (12)

Continuity (2)