Granted Patent B2

US 10,679,337 · App. 16/247,194 · Granted Jun 9, 2020

Inventor: Brian Mullins (Altadena, CA)

Assignee: DAQRI, LLC

G06T7/0008G02B27/0093G02B27/017G02B27/0172G06T19/006G02B2027/014G02B2027/0138G06T2207/30164G06T2219/024

Abstract

Claims (65)

1. A computer-implemented method comprising:

detecting that a first machine within a first predefined distance of a first mobile device;

detecting that a first physical tool is within a second predefined distance of the first machine or a third predefined distance of the first mobile device;

determining whether the first physical tool is compatible with the first machine; and

generating, in response to the determining, augmented reality content configured to be displayed by the first mobile device, the augmented reality content comprising at least one of: a first confirmation indicating that the first physical tool is compatible with the first machine, a second confirmation indicating that the first physical tool is not compatible with the first machine, the second confirmation identifying a second physical tool that is compatible with the first machine.

2. The computer-implemented method of claim 1 , wherein the augmented reality content comprises at least one of: a three-dimensional object model of the first physical tool, and a three-dimensional object model of the second physical tool, wherein the method further comprises: displaying the augmented reality content on top of an image of the first physical tool in a display of the first mobile device.

3. The computer-implemented method of claim 1 , further comprising:

determining a task assigned to a user of the first mobile device by an augmented reality application at the first mobile device; and

accessing a compliance tool library, the compliance tool library comprising a database of tasks, corresponding physical tools that are compatible with the corresponding tasks, and corresponding machines,

wherein determining whether the first physical tool is compatible with the first machine further comprises: determining whether the first physical tool corresponds to the task for the first machine using the tool compliance library.

4. The computer-implemented method of claim 3 , further comprising:

in response to determining that the first physical tool does not correspond to the task for the first machine, determining that the second physical tool corresponds to the task for the first machine using the tool compliance library.

5. The computer-implemented method of claim 3 , wherein determining whether the first physical tool is compatible with the first machine further comprises:

identifying a type of first physical tool; and

determining whether the type of the first physical tool matches the task based on the tool compliance library, the tool compliance library indicating types of physical tools compatible with the first physical tool.

6. The computer-implemented method of claim 1 , further comprising:

identifying a location of the second physical tool,

wherein the augmented reality content further comprises information indicating the location of the second physical tool.

7. The computer-implemented method of claim 1 , further comprising:

identifying a location of the second physical tool, the location being associated with a second machine,

wherein the augmented reality content further comprises information indicating the location of the second machine.

8. The computer-implemented method of claim 1 , further comprising:

identifying a location of the second physical tool, the location being associated with a user of a second mobile device,

wherein the augmented reality content further comprises information indicating the location of the second mobile device.

9. The computer-implemented method of claim 1 , further comprising:

receiving, at a server, sensor data from a plurality of mobile devices having optical sensors and location sensors, the sensor data identifying a plurality of physical tools and their respective geographical locations within a predefined geographic region based on the optical and location sensors, the plurality of physical tools being detected at one or more mobile devices;

identifying one or more physical tools that are present or absent within the predefined geographic region based on a predefined catalog of the physical tools at the predefined geographic region; and

generating, at the server, a real-time inventory of the plurality of physical tools within the predefined geographic region based on the one or more physical tools that are present or absent within the predefined geographic region, the real-time inventory including the identification of the plurality of the physical tools and their respective geographical location within the predefined geographic region.

10. The computer-implemented method of claim 9 , further comprising:

generating a database identifying users, corresponding mobile devices, corresponding user tasks, corresponding physical tools detected at the corresponding mobile devices, corresponding geographic locations of the physical tools, and corresponding physical tool validation statuses, the physical tool validation statuses comprising at least one of a missing physical tool, an incorrect physical tool, and a valid physical tool, identifications of users with the missing physical tools, identifications of users with incorrect physical tools, or identification of users with valid physical tools.

11. A system comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the server to perform operations comprising:

detecting that a first machine within a first predefined distance of a first mobile device;

detecting that a first physical tool is within a second predefined distance of the first machine or a third predefined distance of the first mobile device;

determining whether the first physical tool is compatible with the first machine; and

generating, in response to the determining, augmented reality content configured to be displayed by the first mobile device, the augmented reality content comprising at least one of: a first confirmation indicating that the first physical tool is compatible with the first machine, a second confirmation indicating that the first physical tool is not compatible with the first machine, the second confirmation identifying a second physical tool that is compatible with the first machine.

12. The system of claim 11 , wherein the augmented reality content comprises at least one of: a three-dimensional object model of the first physical tool, and a three-dimensional object model of the second physical tool, wherein the method further comprises: displaying the augmented reality content on top of an image of the first physical tool in a display of the first mobile device.

13. The system of claim 11 , wherein the operations further comprise:

determining a task assigned to a user of the first mobile device by an augmented reality application at the first mobile device; and

accessing a compliance tool library, the compliance tool library comprising a database of tasks, corresponding physical tools that are compatible with the corresponding tasks, and corresponding machines,

wherein determining whether the first physical tool is compatible with the first machine further comprises: determining whether the first physical tool corresponds to the task for the first machine using the tool compliance library.

14. The system of claim 13 , wherein the operations further comprise:

in response to determining that the first physical tool does not correspond to the task for the first machine, determining that the second physical tool corresponds to the task for the first machine using the tool compliance library.

15. The system of claim 13 , wherein determining whether the first physical tool is compatible with the first machine further comprises:

identifying a type of first physical tool; and

determining whether the type of the first physical tool matches the task based on the tool compliance library, the tool compliance library indicating types of physical tools compatible with the first physical tool.

16. The system of claim 11 , wherein the operations further comprise:

identifying a location of the second physical tool,

wherein the augmented reality content further comprises information indicating the location of the second physical tool.

17. The system of claim 11 , wherein the operations further comprise:

identifying a location of the second physical tool, the location being associated with a second machine,

wherein the augmented reality content further comprises information indicating the location of the second machine.

18. The system of claim 11 , wherein the operations further comprise:

identifying a location of the second physical tool, the location being associated with a user of a second mobile device,

wherein the augmented reality content further comprises information indicating the location of the second mobile device.

19. The system of claim 11 , wherein the operations further comprise:

receiving, at the server, sensor data from a plurality of mobile devices having optical sensors and location sensors, the sensor data identifying a plurality of physical tools and their respective geographical locations within a predefined geographic region based on the optical and location sensors, the plurality of physical tools being detected at one or more mobile devices;

identifying one or more physical tools that are present or absent within the predefined geographic region based on a predefined catalog of the physical tools at the predefined geographic region; and

generating, at the server, a real-time inventory of the plurality of physical tools within the predefined geographic region based on the one or more physical tools that are present or absent within the predefined geographic region, the real-time inventory including the identification of the plurality of the physical tools and their respective geographical location within the predefined geographic region.

20. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform operations comprising:

detecting that a first machine within a first predefined distance of a first mobile device;

detecting that a first physical tool is within a second predefined distance of the first machine or a third predefined distance of the first mobile device;

determining whether the first physical tool is compatible with the first machine; and

generating, in response to the determining, augmented reality content configured to be displayed by the first mobile device, the augmented reality content comprising at least one of: a first confirmation indicating that the first physical tool is compatible with the first machine, a second confirmation indicating that the first physical tool is not compatible with the first machine, the second confirmation identifying a second physical tool that is compatible with the first machine.

Assignments (12)

Continuity (3)