SYSTEMS AND METHODS FOR NAVIGATIONAL AND INFORMATIONAL ASSISTANCE FOR DIGITAL TWINS
A example system comprising one or more processors, and memory containing instructions to control the one or more processors to receive a 3D digital model representing a physical environment, receive a first user input, the first user input including a first verbal input to control navigation to a destination within the 3D model, translate the first verbal input into a first text query using a first machine learning model, analyze, by a second machine learning model, the first text query to determine a desired navigation, and provide one or more navigational commands to control navigation to a destination within a graphical user interface associated with the 3D model responsive to the verbal input.
1 . A system comprising:
one or more processors; and
memory containing instructions to control the one or more processors to:
receive a 3D digital model representing a physical environment;
receive a first user input, the first user input including an first verbal input to control navigation to a destination within the 3D model;
translate the first verbal input into a first text query using a first machine learning model;
analyze, by a second machine learning model, the first text query to determine a desired navigation; and
provide one or more navigational commands to control navigation to a destination within a graphical user interface associated with the 3D model responsive to the verbal input.
2 . The system of claim 1 , wherein the memory containing instructions to further control the one or more processors to:
generate a first verbal response describing the destination based on metadata associated with the destination and the 3D model; and
provide the first verbal response.
3 . The system of claim 2 , wherein the first verbal response includes a position of the destination relative to other locations within the physical environment.
4 . The system of claim 1 , wherein the first verbal response is generated based on context from the user.
5 . The system of claim 2 , wherein the first verbal input is received via a microphone and the first verbal response is generated to be provided by an audio speaker.
6 . The system of claim 2 , wherein the first verbal input is received via a microphone and the first verbal response is to be provided as text.
7 . The system of claim 2 , wherein the memory containing instructions to further control the one or more processors to:
generate a textual response based on some or all of the first verbal response; and
provide to the graphical user interface the textual response.
8 . The system of claim 1 , wherein the memory containing instructions to further control the one or more processors to:
generate a textual response based on some or all of the first verbal input response; and
provide to the graphical user interface the textual response.
9 . The system of claim 1 , wherein the memory containing instructions to further control the one or more processors to:
receive a second user input, the second first user input including a second verbal input to request information of an aspect of the physical environment;
translate the second verbal input into a second text query using the first machine learning model;
analyze, by the second machine learning model, the second text query to determine an inquiry result; and
provide a second verbal response based on the inquiry result.
10 . The system of claim 9 , wherein analyze the second text query to determine the inquiry result is based on data from external data sources.
11 . The system of claim 1 , wherein the first verbal input is received from a chat session.
12 . A non-transitory computer-readable medium comprising executable instructions, the executable instructions being executable by one or more processors to perform a method, the method comprising:
receiving a 3D digital model representing a physical environment;
receiving a first user input, the first user input including an first verbal input to control navigation to a destination within the 3D model;
translating the first verbal input into a first text query using a first machine learning model;
analyzing, by a second machine learning model, the first text query to determine a desired navigation; and
providing one or more navigational commands to control navigation to a destination within a graphical user interface associated with the 3D model responsive to the verbal input.
13 . The non-transitory computer-readable medium of claim 12 , wherein the method further comprises:
generating a first verbal response describing the destination based on metadata associated with the destination and the 3D model; and
providing the first verbal response.
14 . The non-transitory computer-readable medium of claim 13 , wherein the first verbal response includes a position of the destination relative to other locations within the physical environment.
15 . The non-transitory computer-readable medium of claim 12 , wherein the first verbal response is generated based on context from the user.
16 . The non-transitory computer-readable medium of claim 13 , wherein the first verbal input is received via a microphone and the first verbal response is generated to be provided by an audio speaker.
17 . The non-transitory computer-readable medium of claim 13 , wherein the first verbal input is received via a microphone and the first verbal response is to be provided as text.
18 . The non-transitory computer-readable medium of claim 12 , wherein the method further comprises:
generating a textual response based on some or all of the first verbal response; and
providing to the graphical user interface the textual response.
19 . The non-transitory computer-readable medium of claim 12 , wherein the method further comprises:
generating a textual response based on some or all of the first verbal input response; and
providing to the graphical user interface the textual response.
20 . The non-transitory computer-readable medium of claim 12 , wherein the method further comprises:
receiving a second user input, the second first user input including a second verbal input to request information of an aspect of the physical environment;
translating the second verbal input into a second text query using the first machine learning model;
analyzing, by the second machine learning model, the second text query to determine an inquiry result; and
providing a second verbal response based on the inquiry result.
21 . The non-transitory computer-readable medium of claim 20 , wherein the analyzing the second text query to determine the inquiry result is based on data from external data sources.
22 . The non-transitory computer-readable medium of claim 20 , wherein the first verbal input is received from a chat session.
23 . A method comprising:
receiving a 3D digital model representing a physical environment;
receiving a first user input, the first user input including an first verbal input to control navigation to a destination within the 3D model;
translating the first verbal input into a first text query using a first machine learning model;
analyzing, by a second machine learning model, the first text query to determine a desired navigation; and
providing one or more navigational commands to control navigation to a destination within a graphical user interface associated with the 3D model responsive to the verbal.