Method and system for implementing augmented reality (AR)-based assistance within work environment
Novel tools and techniques are provided for implementing augmented reality (AR)-based assistance within a work environment. In various embodiments, a computing system might receive, from a camera having a field of view of a work environment, first images of at least part of the work environment, the first images overlapping with a field of view of a user wearing an AR headset; might analyze the received first images to identify objects; might query a database(s) to determine a task associated with a first object(s) among the identified objects; might generate an image overlay providing at least one of graphical icon-based, text-based, image-based, and/or highlighting-based instruction(s) each indicative of instructions presented to the user to implement the task associated with the first object(s); and might display, to the user's eyes through the AR headset, the generated first image overlay that overlaps with the field of view of the user's eyes.
1. An apparatus comprising:
at least one hardware processor; and
a non-transitory computer readable medium having stored thereon machine readable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to:
receive, from an image capture device, at least one image of a portion of an environment, the image overlapping with a field of view of a user wearing an augmented reality (“AR”) headset that includes the image capture device;
analyze the at least one image to identify a plurality of objects disposed in the environment;
determine a task associated with at least one object of the identified plurality of objects,
wherein the task includes a plurality of steps associated with a particular type of task, and
wherein the plurality of steps includes specific task-related step-by-step actions based on meeting of a specified percentage condition of usage that includes partial usage associated with the identified plurality of objects;
generate an image overlay that provides at least one instruction to be presented to the user to implement the plurality of steps; and
generate, to the user through the AR headset, a display of the generated image overlay.
2. The apparatus according to claim 1 , wherein the plurality of steps includes:
specific timings for the specific task-related step-by-step actions associated with the identified plurality of objects; and
specific sequences for the specific task-related step-by-step actions associated with the identified plurality of objects.
3. The apparatus according to claim 1 , wherein each object of the plurality of objects includes a radio frequency identifier (“RFID”) tag affixed thereto.
4. The apparatus according to claim 3 , wherein the machine readable instructions to analyze the at least one image to identify the plurality of objects disposed in the environment, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
identify each object of the plurality of objects based on the affixed RFID tag in conjunction with identifying each object of the plurality of objects based on analysis of the at least one image; and
filter, based on a gesture or a selection by the user, RFID tag information associated with an object outside of a central portion of the field of view of the user or outside of an area defined by the gesture or the selection by the user.
5. The apparatus according to claim 1 , wherein the AR headset includes at least one of a set of AR goggles, a pair of AR-enabled eyewear, an AR-enabled smartphone mounted in a headset, or an AR helmet.
6. The apparatus according to claim 1 , wherein the AR headset includes at least one of at least one earpiece or at least one speaker, and wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
generate at least one audio-based instruction corresponding to the at least one instruction to be presented to the user.
7. The apparatus according to claim 1 , wherein the at least one image represents at least one first image, and wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
receive, from a further image capture device that is disposed external to the AR headset, at least one second image of the portion of the environment, the at least one second image overlapping with the field of view of the user wearing the AR headset,
wherein the machine readable instructions to analyze the at least one first image to identify the plurality of objects disposed in the environment, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
analyze the at least one first image and the at least one second image to identify the plurality of objects disposed in the environment.
8. The apparatus according to claim 1 , wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
at least one of superimpose the at least one instruction over the at least one object, display the at least one instruction around the at least one object, or display the at least one instruction beside the at least one object.
9. The apparatus according to claim 1 , wherein the at least one instruction includes at least one of a graphical icon-based instruction, a text-based instruction, an image-based instruction, or a highlighting-based instruction.
10. The apparatus according to claim 1 , wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
illuminate, by a light source, at least one of the at least one object or the portion of the environment.
11. The apparatus according to claim 1 , wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
determine whether an object of the plurality of objects is not identifiable; and
based on a determination that the object of the plurality of objects is not identifiable, generate the image overlay to include the at least one instruction to prompt the user to specifically view the object that is not identifiable.
12. The apparatus according to claim 1 , wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
generate the image overlay to highlight an area in the environment in which the at least one object is to be fitted.
13. A method comprising:
receiving, by at least one hardware processor, at least one image of a portion of an environment, the image overlapping with a field of view of a user wearing an augmented reality (“AR”) headset;
analyzing, by the at least one hardware processor, the at least one image to identify a plurality of objects disposed in the environment;
determining, by the at least one hardware processor, a task associated with at least one object of the identified plurality of objects,
wherein the task includes a plurality of steps associated with a particular type of task, and
wherein the plurality of steps includes:
specific task-related step-by-step actions based on meeting of a specified percentage condition of usage that includes partial usage associated with the identified plurality of objects;
specific timings for the specific task-related step-by-step actions associated with the identified plurality of objects; and
specific sequences for the specific task-related step-by-step actions associated with the identified plurality of objects;
generating, by the at least one hardware processor, an image overlay that provides at least one instruction to be presented to the user to implement the plurality of steps; and
generating, by the at least one hardware processor, to the user through the AR headset, a display of the generated image overlay.
14. The method according to claim 13 , wherein the at least one image represents at least one first image, and wherein analyzing, by the at least one hardware processor, the at least one image to identify the plurality of objects disposed in the environment further comprises:
receiving, by the at least one hardware processor, from an image capture device that is disposed external to the AR headset, at least one second image of the portion of the environment, the at least one second image overlapping with the field of view of the user wearing the AR headset; and
analyzing, by the at least one hardware processor, the at least one first image and the at least one second image to identify the plurality of objects disposed in the environment.
15. The method according to claim 13 , further comprising:
determining, by the at least one hardware processor, whether an object of the plurality of objects is not identifiable; and
based on a determination that the object of the plurality of objects is not identifiable, generating, by the at least one hardware processor, the image overlay to include the at least one instruction to prompt the user to specifically view the object that is not identifiable.
16. A non-transitory computer readable medium having stored thereon machine readable instructions that, when executed by at least one hardware processor, cause the at least one hardware processor to:
receive at least one image of a portion of an environment, the image overlapping with a field of view of a user wearing an augmented reality (“AR”) headset;
analyze the at least one image to identify a plurality of objects disposed in the environment, wherein each object of the plurality of objects includes a radio frequency identifier (“RFID”) tag affixed thereto;
identify each object of the plurality of objects based on the affixed RFID tag in conjunction with identifying each object of the plurality of objects based on analysis of the at least one image;
filter, based on a gesture or a selection by the user, RFID tag information associated with an object outside of a central portion of the field of view of the user or outside of an area defined by the gesture or the selection by the user; and
determine a task associated with at least one object of the identified plurality of objects, wherein the task includes a plurality of steps associated with a particular type of task, and wherein the plurality of steps includes specific task-related step-by-step actions based on meeting of a specified percentage condition of usage that includes partial usage associated with the identified plurality of objects and specific timings for completion of the specific task-related step-by-step actions associated with the identified plurality of objects.
17. The non-transitory computer readable medium according to claim 16 , wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
generate an image overlay that provides at least one instruction to be presented to the user to implement the plurality of steps; and
generate to the user through the AR headset, a display of the generated image overlay.
18. The non-transitory computer readable medium according to claim 17 , wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
illuminate, by a light source, at least one of the at least one object or the portion of the environment.
19. The non-transitory computer readable medium according to claim 17 , wherein the machine readable instructions, when executed by the at least one hardware processor, further cause the at least one hardware processor to:
generate the image overlay to highlight an area in the environment in which the at least one object is to be fitted.
20. The non-transitory computer readable medium according to claim 16 , wherein the plurality of steps includes
specific sequences for the specific task-related step-by-step actions associated with the identified plurality of objects.