Augmented reality assembly assistance and monitoring
Described are systems and methods for providing augmented reality information to workers to assist the workers in assembly of objects, such as aerial vehicles. An object or parts of an object may be determined by processing of image data corresponding to a field of view of a pair of augmented reality glasses worn by a worker to determine an object or a part corresponding to an object that is to be assembled by the worker. Based on the determined object and/or part, augmented reality information corresponding to an assembly task may be determined and visually presented to the worker to aid the worker in completion of the assembly task. The augmented reality information may be visually presented by the augmented reality glasses such that the worker can view the augmented reality information and the object or parts concurrently.
1. A system comprising:
a wearable frame;
a sensor device mounted to the wearable frame;
a display device mounted to the wearable frame; and
a computing device in communication with at least one of the sensor device or the display device,
wherein the computing device is configured to at least:
detect a scale included at a workstation, the scale providing a reference point to determine a relative size or a relative position of one or more parts at the workstation;
determine, based at least in part on a comparison of a first part at the workstation and the scale, an identity of the first part;
determine an identifier on the first part positioned at the workstation;
determine an assembly task to be performed by a worker located at the workstation and wearing the wearable frame, the assembly task determined based at least in part on the workstation, the worker, or the first part;
select augmented reality data corresponding to the assembly task to be performed; and
cause a display of the augmented reality data on the display device such that the augmented reality data is presented in a field of view of the worker.
2. The system of claim 1 , wherein the computing device is further configured to at least:
record video data representative of an assembly of the first part with a second part as part of the assembly task.
3. The system of claim 2 , wherein the computing device is further configured to at least:
process the video data to determine that the assembly of the first part with the second part was correctly completed.
4. The system of claim 1 , wherein the augmented reality data includes a visual animation of an assembly of the first part with a second part.
5. The system of claim 1 , wherein the augmented reality data includes a visual representation of a computer-aided drawing of at least a portion of an object being assembled.
6. A method for virtually illustrating information, comprising:
detecting a scale within a field of view of a worker wearing augmented reality (AR) glasses, the scale providing a reference to determine a relative size or a relative position of one or more parts within the field of view;
interpreting at least one identifier associated with a part positioned within the field of view of the worker wearing the AR glasses based at least in part on the scale;
determining augmented reality data regarding the part based at least in part on the at least one identifier using a first computer device, wherein the augmented reality data regarding the part is stored in at least one data store; and
causing a display of the augmented reality data by the AR glasses such that the augmented reality data is in the field of view of the worker while the worker is viewing the part.
7. The method of claim 6 , further comprising:
receiving a request from the worker for an assistance with respect to the part;
transmitting video data corresponding to the field of view of the worker from the AR glasses to at least one of the external computing devices;
receiving, from the at least one external computing device, annotation data corresponding to a visual input provided at the at least one external computing device; and
causing a display of the annotation data by the AR glasses such that the annotation data is in the field of view of the worker.
8. The method of claim 7 , further comprising:
receiving, from the at least one external computing device, audio data corresponding to audio provided by the at least one external computing device; and
causing an audible output such that the audio data is audibly presented to the worker.
9. The method of claim 7 , wherein the video data and the annotation data are transmitted and presented in real time or near real time.
10. The method of claim 6 , wherein the augmented reality data corresponds to an assembly task for assembly of the part with a work in progress object to be assembled.
11. The method of claim 10 , further comprising:
detecting an interaction by the worker indicating a completion of the assembly task;
determining a second assembly task to be completed by the worker; and
causing a display of second augmented reality data corresponding to the second assembly task by the AR glasses such that the second augmented reality data is presented in the field of view of the worker while the worker is viewing a second part.
12. The method of claim 11 , further comprising:
processing data from the AR glasses to verify that the assembly task was correctly completed.
13. The method of claim 6 , wherein the display of the augmented reality data includes a visual indicator that is positioned adjacent the part to indicate the part to the worker.
14. The method of claim 6 , further comprising:
causing a display, by the AR glasses, of a task progress indicator indicating a progress through a plurality of tasks to complete assembly of an object.
15. A non-transitory computer-readable medium having computer-executable instructions stored thereon,
wherein the computer-executable instructions, when executed, cause a computer system having at least one computer processor to perform a method comprising:
detecting a scale included at a workstation, the scale providing a reference to determine a relative size or a relative position of one or more parts at the workstation;
determining an object to be assembled at the workstation based at least in part on the scale;
determining an assembly task to be performed by a worker positioned at the workstation as part of an assembly of the object;
determining augmented reality data corresponding to the assembly task; and
presenting the augmented reality data such that the augmented reality data is visually presented in a field of view of the worker and visible by the worker while the worker views at least one part corresponding to the object.
16. The non-transitory computer-readable medium of claim 15 ,
wherein the computer-executable instructions, when executed, cause the computer system having the at least one computer processor to perform the method further comprising:
detecting an identifier corresponding to the at least one part positioned at the workstation; and
wherein the augmented reality data is presented to visually indicate the at least one part to the worker.
17. The non-transitory computer-readable medium of claim 15 ,
wherein the computer-executable instructions, when executed, cause the computer system having the at least one computer processor to perform the method further comprising:
determining that the assembly task has been completed;
determining a second assembly task to be performed by the worker positioned at the workstation as part of the assembly of the object;
determining second augmented reality data corresponding to the second assembly task; and
presenting the second augmented reality data such that the second augmented reality data is visually presented in the field of view of the worker and visible by the worker while the worker views at least one second part corresponding to the object.
18. The non-transitory computer-readable medium of claim 15 ,
wherein the computer-executable instructions, when executed, cause the computer system having the at least one computer processor to perform the method further comprising:
determining that the assembly task has been completed; and
processing an image of the field of view of the worker to verify that the assembly task has been properly completed.
19. The non-transitory computer-readable medium of claim 15 ,
wherein the computer-executable instructions, when executed, cause the computer system having the at least one computer processor to perform the method further comprising:
recording video data of the assembly task performed by the worker; and
determining a time duration required by the worker to perform the assembly task.
20. The non-transitory computer-readable medium of claim 19 ,
wherein the computer-executable instructions, when executed, cause the computer system having the at least one computer processor to perform the method further comprising:
determining a plurality of time durations required to perform the assembly task by a plurality of workers;
determining from the time duration and the plurality of time durations, an average time duration required to complete the assembly task; and
determining, based at least in part on the average time duration, that at least one of the augmented reality data corresponding to the assembly task or the assembly task is to be revised to decrease a time duration required to complete the assembly task.