Remote expert system
A remote expert application identifies a manipulation of virtual objects displayed in a first wearable device. The virtual objects are rendered based a physical object viewed with a second wearable device. A manipulation of the virtual objects is received from the first wearable device. A visualization of the manipulation of the virtual objects is generated for a display of the second wearable device. The visualization of the manipulation of the virtual objects is communicated to the second wearable device.
1. A system comprising:
one or more computer processors; and
one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the system to perform operations comprising:
receiving recorded data captured by one or more display devices, wherein the recorded data is related to a physical environment viewed by the one or more display devices, and the recorded data includes a video feed captured by the one or more display devices, the video feed depicting one or more physical objects in the physical environment;
generating a display content for visualization on the one or more display devices, wherein the display content is based on the recorded data;
receiving a request from the one or more display devices and identifying a remote user display device based on the request, wherein identifying the remote user display device comprises:
comparing the recorded data received from the one or more display devices to a reference three dimensional model dataset of the one or more physical objects, yielding a comparison;
identifying, based on the comparison, an anomaly between the one or more physical objects depicted in the video feed and the three dimensional model dataset of the one or more physical objects; and
identifying, based on the anomaly, a remote user at the remote user display device designated as an expert related to the anomaly;
communicating the generated display content to the identified remote user display device;
recording information related to the generated display content at the identified remote user display device;
generating a visualization of the recorded information; and
communicating the visualization to the one or more display devices for display.
2. The system of claim 1 , wherein the display content includes a three dimensional model of one or more virtual objects based on the one or more physical objects in the physical environment.
3. The system of claim 2 , wherein the recorded information includes manipulations of the three dimensional model of the one or more virtual objects, and the manipulations are performed by a remote user at the remote user display device.
4. The system of claim 3 , wherein generating the visualization further comprises:
generating an animation based on the recorded information.
5. The system of claim 3 , wherein the manipulations of the three dimensional model are generated by the remote user using a visual gesture module, the visual gesture module configured to determine a predefined area of a display of the remote display device and a feature of the three dimensional model, and to change a state of the feature in response to the feature being in the predefined area of the display of the remote display device.
6. The system of claim 3 , wherein the manipulations of the three dimensional model are generated by the remote user using a physical gesture module, the physical gesture module configured to detect a physical movement of a finger of the remote user relative to the three dimensional model displayed by the remote display device.
7. The system of claim 2 , the operations further comprising:
identifying a manipulation of the one or more physical objects depicted in the video feed;
updating the three dimensional model of one or more virtual objects based on the identified manipulation; and
creating a dynamic status related to the one or more physical objects using reference data related to the one or more physical objects and the updated three dimensional model of one or more virtual objects.
8. The system of claim 1 , wherein generating the visualization further comprises:
generating a warning based on the recorded information.
9. A method comprising:
receiving recorded data captured by one or more display devices, wherein the recorded data is related to a physical environment viewed by the one or more display devices, wherein the recorded data is related to a physical environment viewed by the one or more display devices, and the recorded data includes a video feed captured by the one or more display devices, the video feed depicting one or more physical objects in the physical environment;
generating a display content for visualization on the one or more display devices, wherein the display content is based on the recorded data;
receiving a request from the one or more display devices and identifying a remote user display device based on the request, wherein identifying the remote user display device comprises:
comparing the recorded data received from the one or more display devices to a reference three dimensional model dataset of the one or more physical objects, yielding a comparison;
identifying, based on the comparison, an anomaly between the one or more physical objects depicted in the video feed and the three dimensional model dataset of the one or more physical objects; and
identifying, based on the anomaly, a remote user at the remote user display device designated as an expert related to the anomaly;
communicating the generated display content to the identified remote user display device;
recording information related to the generated display content at the identified remote user display device;
generating a visualization of the recorded information; and
communicating the visualization to the one or more display devices for display.
10. The method of claim 9 , wherein the recorded information includes manipulations of the three dimensional model of the one or more virtual objects, and the manipulations are performed by a remote user at the remote user display device.
11. The method of claim 10 , wherein generating the visualization further comprises:
generating an animation based on the recorded information.
12. The method of claim 11 , wherein the manipulations of the three dimensional model are generated by the remote user using a visual gesture module, the visual gesture module configured to determine a predefined area of a display of the remote display device and a feature of the three dimensional model, and to change a state of the feature in response to the feature being in the predefined area of the display of the remote display device.
13. The method of claim 11 , wherein the manipulations of the three dimensional model are generated by the remote user using a physical gesture module, the physical gesture module configured to detect a physical movement of a finger of the remote user relative to the three dimensional model displayed by the remote display device.
14. The method of claim 10 , further comprising:
identifying a manipulation of the one or more physical objects depicted in the video feed;
updating the three dimensional model of one or more virtual objects based on the identified manipulation;
creating a dynamic status related to the one or more physical objects using reference data related to the one or more physical objects and the updated three dimensional model of one or more virtual objects.
15. The method of claim 9 , wherein generating the visualization further comprises:
generating a warning based on the recorded information.
16. A non-transitory machine-readable medium comprising instructions that, when executed by one or more computer processors of a machine, cause the machine to perform operations comprising:
receiving recorded data captured by one or more display devices, wherein the recorded data is related to a physical environment viewed by the one or more display devices, wherein the recorded data is related to a physical environment viewed by the one or more display devices, and the recorded data includes a video feed captured by the one or more display devices, the video feed depicting one or more physical objects in the physical environment;
generating a display content for visualization on the one or more display devices, wherein the display content is based on the recorded data;
receiving a request from the one or more display devices and identifying a remote user display device based on the request, wherein identifying the remote user display device comprises:
comparing the recorded data received from the one or more display devices to a reference three dimensional model dataset of the one or more physical objects, yielding a comparison;
identifying, based on the comparison, an anomaly between the one or more physical objects depicted in the video feed and the three dimensional model dataset of the one or more physical objects; and
identifying, based on the anomaly, a remote user at the remote user display device designated as an expert related to the anomaly;
communicating the generated display content to the identified remote user display device;
recording information related to the generated display content at the identified remote user display device;
generating a visualization of the recorded information; and
communicating the visualization to the one or more display devices for display.