Providing technical support in an augmented reality environment
View Patent ↗Various embodiments of the invention provide methods, systems, and computer program products for providing support to a party who is using an augmented reality device and seeking to resolve an issue. One or more pieces of information are obtained through the augmented reality device and a cause of the issue is identified by utilizing one or more predictive models and the pieces of information as input to the models. Once the cause has been identified, a script is queried that includes instructions to be performed by the party to address the cause and media to execute on the augmented reality device to aid the party in performing the instructions. Accordingly, the instructions and media are communicated to the party through the augmented reality device so that the party may perform the instructions with the aid of the media to address the cause of the issue.
1. A method for providing support to a party who is using an augmented reality device and seeking to resolve an issue the party who is using the augmented reality device is seeking to resolve, the method comprising:
obtaining one or more pieces of information from the augmented reality device needed to identify a cause of the issue, the one or more pieces of information comprising at least one of (1) an image of a real world, physical object photographed by a camera coupled to the augmented reality device, (2) a video of the real world, physical object recorded by the camera coupled to the augmented reality device, (3) a physical sound recorded through a microphone on the augmented reality device, and (4) input provided by the party on an input element on the augmented reality device;
identifying the cause of the issue by utilizing one or more predictive models and at least one of the one or more pieces of information as input to the one or more predictive models, wherein the one or more predictive models are arranged in a hierarchical program structure of possible causes to provide an order in which the one or more predictive models are utilized for identifying the cause of the issue and identifying the cause of the issue involves systematically working through the possible causes by making use of the one or more predictive models along with the at least one of the one or more pieces of information as input to the one or more predictive models to determine a likelihood of each of one or more of the possible causes being the cause of the issue;
querying a script for the identified cause of the issue, the script comprising one or more instructions to be performed by the party and one or more pieces of media to execute on the augmented reality device to aid the party in performing the one or more instructions; and
communicating the one or more instructions and the one or more pieces of media to the party through the augmented reality device.
2. The method of claim 1 comprising communicating location information to the augmented reality device, the location information identifying at least one of where to place one of the one or more instructions and one of the one or more pieces of media on a display of the augmented reality device so that the at least one of the one of the one or more instructions and the one of the one or more pieces of media are superimposed over a field of view of the party in one or more positions with respect to at least one physical object viewable in the field of view of the party.
3. The method of claim 1 , wherein the one or more predictive models comprise one or more of a decision tree, a support vector machine, Bayesian network, clustering, reinforcement learning, and neural network.
4. The method of claim 1 further comprising requesting access to the augmented reality device so that certain aspects of the augmented reality device can be controlled to obtain the one or more pieces of information.
5. The method of claim 1 , wherein the one or more instructions are communicated to the party one at a time and the method further comprises:
determining whether an instruction from the one or more instructions communicated to the party was performed correctly by the party;
in response to determining the instruction was not performed correctly by the party, communicating the instruction to the party again so that the party can re-perform the instruction correctly; and
in response to determining the instruction was performed correctly by the party, communicating a next instruction from the one or more instructions to the party to perform.
6. The method of claim 1 , wherein the one or more pieces of media comprise at least one of (1) an image to be displayed on a display of the augmented reality device, (2) a video to be displayed on the display of the augmented reality device, and (3) a sound to be played over a speaker coupled with the augmented reality device.
7. A non-transitory, computer-readable medium comprising computer-executable instructions for providing support to a party who is using an augmented reality device and seeking to resolve an issue the party who is using the augmented reality device is seeking to resolve, that when executed, cause at least one computer processor to:
obtain one or more pieces of information from the augmented reality device needed to identify a cause of the issue, the one or more pieces of information comprising at least one of (1) an image of a real world, physical object photographed by a camera coupled to the augmented reality device, (2) a video of the real world, physical object recorded by the camera coupled to the augmented reality device, (3) a physical sound recorded through a microphone on the augmented reality device, and (4) input provided by the party on an input element on the augmented reality device;
identify the cause of the issue by utilizing one or more predictive models and at least one of the one or more pieces of information as input to the one or more predictive models, wherein the one or more predictive models are arranged in a hierarchical program structure of possible causes to provide an order in which the one or more predictive models are utilized for identifying the cause of the issue and the computer-executable instructions cause the at least one computer process to identify the cause of the issue by systematically working through the possible causes by making use of the one or more predictive models along with the at least one of the one or more pieces of information as input to the one or more predictive models to determine a likelihood of each of one or more of the possible causes being the cause of the issue;
query a script for the identified cause of the issue, the script comprising one or more instructions to be performed by the party and one or more pieces of media to execute on the augmented reality device to aid the party in performing the one or more instructions; and
communicate the one or more instructions and the one or more pieces of media to the party through the augmented reality device.
8. The non-transitory, computer-readable medium of claim 7 , wherein the computer-executable instructions cause the at least one computer processor to communicate location information to the augmented reality device, the location information identifying at least one of where to place one of the one or more instructions and one of the one or more pieces of media on a display of the augmented reality device so that the at least one of the one of the one or more instructions and the one of the one or more pieces of media are superimposed over a field of view of the party in one or more positions with respect to at least one physical object viewable in the field of view of the party.
9. The non-transitory, computer-readable medium of claim 7 , wherein the one or more predictive models comprise one or more of a decision tree, a support vector machine, Bayesian network, clustering, reinforcement learning, and neural network.
10. The non-transitory, computer-readable medium of claim 7 , wherein the computer-executable instructions cause the at least one computer processor to request access to the augmented reality device so that the at least one computer processor is able to control certain aspects of the augmented reality device to obtain the one or more pieces of information.
11. The non-transitory, computer-readable medium of claim 7 , wherein the one or more instructions are communicated to the party one at a time and the computer-executable instructions cause the at least one computer processor to:
determine whether an instruction from the one or more instructions communicated to the party was performed correctly by the party;
in response to determining the instruction was not performed correctly by the party, communicate the instruction to the party again so that the party can re-perform the instruction correctly; and
in response to determining the instruction was performed correctly by the party, communicate a next instruction from the one or more instructions to the party to perform.
12. The non-transitory, computer-readable medium of claim 7 , wherein the one or more pieces of media comprise at least one of (1) an image to be displayed on a display of the augmented reality device, (2) a video to be displayed on the display of the augmented reality device, and (3) a sound to be played over a speaker coupled with the augmented reality device.
13. A system for providing support to a party who is using an augmented reality device and seeking to resolve an issue the party who is using the augmented reality device is seeking to resolve, the system comprising:
at least one computer processor configured to:
obtain one or more pieces of information from the augmented reality device needed to identify a cause of the issue, the one or more pieces of information comprising at least one of (1) an image of a real world, physical object photographed by a camera coupled to the augmented reality device, (2) a video of the real world, physical object recorded by the camera coupled to the augmented reality device, (3) a physical sound recorded through a microphone on the augmented reality device, and (4) input provided by the party on an input element on the augmented reality device;
identify the cause of the issue by utilizing one or more predictive models and at least one of the one or more pieces of information as input to the one or more predictive models, wherein the one or more predictive models are arranged in a hierarchical program structure of possible causes to provide an order in which the one or more predictive models are utilized for identifying the cause of the issue and the at least one computer process is configured to identify the cause of the issue by systematically working through the possible causes by making use of the one or more predictive models along with the at least one of the one or more pieces of information as input to the one or more predictive models to determine a likelihood of each of one or more of the possible causes being the cause of the issue;
query a script for the identified cause of the issue, the script comprising one or more instructions to be performed by the party and one or more pieces of media to execute on the augmented reality device to aid the party in performing the one or more instructions; and
communicate the one or more instructions and the one or more pieces of media to the party through the augmented reality device.
14. The system of claim 13 , wherein the at least one computer processor is configured to communicate location information to the augmented reality device, the location information identifying at least one of where to place one of the one or more instructions and one of the one or more pieces of media on a display of the augmented reality device so that the at least one of the one of the one or more instructions and the one of the one or more pieces of media are superimposed over a field of view of the party in one or more positions with respect to at least one physical object viewable in the field of view of the party.
15. The system of claim 13 , wherein the one or more predictive models comprise one or more of a decision tree, a support vector machine, Bayesian network, clustering, reinforcement learning, and neural network.
16. The system of claim 13 , wherein the at least one computer processor is configured to request access to the augmented reality device so that the at least one computer processor is able to control certain aspects of the augmented reality device to obtain the one or more pieces of information.
17. The system of claim 13 , wherein the one or more instructions are communicated to the party one at a time and the at least one computer processor is configured to:
determine whether an instruction from the one or more instructions communicated to the party was performed correctly by the party;
in response to determining the instruction was not performed correctly by the party, communicate the instruction to the party again so that the party can re-perform the instruction correctly; and
in response to determining the instruction was performed correctly by the party, communicate a next instruction from the one or more instructions to the party to perform.
18. The system of claim 13 , wherein the one or more pieces of media comprise at least one of (1) an image to be displayed on a display of the augmented reality device, (2) a video to be displayed on the display of the augmented reality device, and (3) a sound to be played over a speaker coupled with the augmented reality device.