Contextual applications in a mixed reality environment
A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.
1. A system comprising:
one or more computer processors;
one or more computer memories; and
a set of instructions incorporated into the one or more computer memories, the set of instructions configuring the one or more computer processors to perform operations for implementing a digital mesh controller within a mixed reality (MR) environment on an MR-capable device of a user, the operations comprising:
identifying at least one real-world object in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device;
monitoring over time a movement and an orientation of the identified real-world object in the MR environment:
selecting the digital mesh controller from a memo based at least in part on the attributes of the identified object;
deforming the digital mesh controller to fit physical dimensions of the identified object;
moving the digital mesh with the identified object as the identified object moves through the environment; and
controlling an application associated with the selected digital mesh controller based on an interaction of a user with the identified object.
2. The system of claim 1 , wherein the controlling is based on a detection of a predetermined value of at least one of the following during the interaction: a position, an orientation and a motion of the identified object.
3. The system of claim 1 , wherein the application generates dynamic digital content in a display of the MR-capable device based on the controlling.
4. The system of claim 3 , wherein the digital content is displayed relative to a coordinate system of the digital mesh controller as the digital mesh controller moves through the environment.
5. The system of claim 3 , wherein the interaction includes an interaction between a movement of the identified object and the generated digital content.
6. The system of claim 1 , wherein the interaction includes human body movements of the user detected using the sensors of the :MR-capable device.
7. The system of claim 1 , further comprising receiving the digital mesh controller from an additional system over a network.
8. The system of claim 1 , wherein a visible digital texture is applied to the digital mesh controller and displayed in the MR-capable device.
9. A method comprising:
performing operations for implementing a digital mesh controller within a mixed reality (MR) environment on an MR-capable device of a user, the operations comprising:
identifying at least one real-world object in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device;
monitoring over time a movement and an orientation of the identified real-world object in the MR environment;
selecting the digital mesh controller from a memory based at least in part on the attributes of the identified object;
deforming the digital mesh controller to fit physical dimensions of the identified object;
moving the digital mesh with the identified object as the identified object moves through the environment; and
controlling an application associated with the selected digital mesh controller based on an interaction of a user with the identified object.
10. The method of claim 9 , wherein the controlling is based on a detection of a predetermined value of at least one of the following during the interaction: a position, an orientation and a motion of the identified object.
11. The method of claim 9 , wherein the application generates dynamic digital content n a display of the MR-capable device based on the controlling.
12. The method of claim 11 , wherein the digital content is displayed relative to a coordinate system of the digital mesh controller as the digital mesh controller moves through the environment.
13. The method of claim 11 , wherein the interaction includes an interaction between a movement of the identified object and the generated digital content.
14. The method of claim 9 , wherein the interaction includes human body movements of the user detected using the sensors of the MR-capable device.
15. The method of claim 9 , further comprising receiving the digital mesh controller from an additional system over a network.
16. The method of claim 9 , wherein a visible digital texture is applied to the digital mesh controller and displayed in the MR-capable device.
17. A non-transitory machine-readable storage medium storing a set of instructions that, when executed by at least one processor of one or more machines, cause the one or more machines to perform operations for implementing a digital mesh controller within a mixed reality (MR) environment on an MR-capable device of a user, the operations comprising:
identifying at least one real-world object in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device;
monitoring over time a movement and an orientation of the identified real-world object in the MR environment;
selecting the digital mesh controller from a memory based at least in part on the attributes of the identified object;
deforming the digital mesh controller to fit physical dimensions of the identified object;
moving the digital mesh with the identified object as the identified object moves through the environment; and
controlling an application associated with the selected digital mesh controller based on an interaction of a user with the identified object.
18. The non-transitory machine-readable storage medium of claim 17 , wherein the controlling is based on a detection of a predetermined value of at least one of the following during the interaction: a position, an orientation and a motion of the identified object.
19. The non-transitory machine-readable storage medium of claim 17 , wherein the application generates dynamic digital content in a display of the MR-capable device based on the controlling.
20. The non-transitory machine-readable storage medium of claim 19 , wherein the digital content is displayed relative to a coordinate system of the digital mesh controller as the digital mesh controller moves through the environment.