Contextual applications in a mixed reality environment
A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.
1. A system comprising:
one or more computer processors;
one or more computer memories; and
a set of instructions incorporated into the one or more computer memories, the set of instructions configuring the one or more computer processors to perform operations for implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user, the operations comprising:
receiving data describing surroundings of the environment in proximity to the MR-capable device from a plurality of sensors on the MR-capable device;
analyzing the received data, the analyzing including comparing the received data against a plurality of predefined contextual triggers associated with a plurality of contextual applications in order to determine a matching set of predefined contextual triggers associated with a matched contextual application, wherein each of the predefined contextual triggers comprises a specific configuration of data from a sensor of the plurality of sensors;
determining a reality layer associated with the device; and
executing the matched contextual application based on the determined reality layer, wherein the executing of the matched contextual application is based on permissions associated with the determined reality layer, wherein the permissions make available predetermined instructions in the contextual application which are associated with the determined reality layer, the executing of the available predetermined instructions causing the matched contextual application to perform operations that are specific to the determined reality layer.
2. The system of claim 1 , wherein the specific configuration of data pertains to one or more of the following: a time, a duration of time, a location of the MR device, a proximity to a plurality of real-world objects identified in the MR environment by the plurality of sensors, and an orientation of a plurality of real-world objects identified in the MR environment by the plurality of sensors.
3. The system of claim 1 , wherein the operations further include:
sending the received data and a description of the determined reality layer over a network to a server device;
performing the comparing on the server device;
receiving data from the server device over the network, the data describing a matched contextual application based on the comparing; and
performing the executing of the matched contextual application on the MR-capable device.
4. The system of claim 1 , wherein the operations further include determining a reality layer associated with a user of the MR-capable device.
5. The system of claim 1 , wherein the executing of the matched contextual application includes presenting at least one virtual object on a display device within the MR-capable device such that it appears in the mixed reality environment.
6. The system of claim 5 , the operations further comprising executing a function within the matched contextual application based on a determination that an interaction of the user with the at least one virtual object satisfies a second set of contextual triggers, the determination that the interaction of the user with the at least one virtual object satisfies the second set of contextual triggers being based on interaction rules described within the matched contextual application.
7. The system of claim 1 , wherein the operations further include:
analyzing the received data to identify at least one real-world object in the MR environment;
monitoring over time a movement and an orientation of the identified real-world object in the MR environment; and
controlling an aspect of operation during the executing of the matched contextual application based on an interaction of a user with the identified object, the interaction including manipulating the movement and the orientation of the real-world object.
8. A non-transitory computer-readable medium storing a set of instructions that, when executed by one or more processors, cause the one or more processors to perform operations for implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user, the operations comprising:
receiving data describing surroundings of the environment in proximity to the MR-capable device from a plurality of sensors on the MR-capable device;
analyzing the received data, the analyzing including comparing the received data against a plurality of predefined contextual triggers associated with a plurality of contextual applications in order to determine a matching set of predefined contextual triggers associated with a matched contextual application, wherein each of the predefined contextual triggers comprises a specific configuration of data from a sensor of the plurality of sensors;
determining a reality layer associated with the device; and
executing the matched contextual application based on the determined reality layer, wherein the executing of the matched contextual application is based on permissions associated with the determined reality layer, wherein the permissions make available predetermined instructions in the contextual application which are associated with the determined reality layer, the executing of the available predetermined instructions causing the matched contextual application to perform operations that are specific to the determined reality layer.
9. The non-transitory computer-readable medium of claim 8 , wherein the specific configuration of data pertains to one or more of the following: a time, a duration of time, a location of the MR device, a proximity to a plurality of real-world objects identified in the MR environment by the plurality of sensors, and an orientation of a plurality of real-world objects identified in the MR environment by the plurality of sensors.
10. The non-transitory computer-readable medium of claim 8 , wherein the operations further include:
sending the received data and a description of the determined reality layer over a network to a server device;
performing the comparing on the server device;
receiving data from the server device over the network, the data describing a matched contextual application based on the comparing; and
performing the executing of the matched contextual application on the MR-capable device.
11. The non-transitory computer-readable medium of claim 8 , wherein the operations further include determining a reality layer associated with a user of the MR-capable device.
12. The non-transitory computer-readable medium of claim 8 , wherein the executing of the matched contextual application includes presenting at least one virtual object on a display device within the MR-capable device such that it appears in the mixed reality environment.
13. The non-transitory computer-readable medium of claim 12 , the operations further comprising executing a function within the matched contextual application based on a determination that an interaction of the user with the at least one virtual object satisfies a second set of contextual triggers, the determination that the interaction of the user with the at least one virtual object satisfies the second set of contextual triggers being based on interaction rules described within the matched contextual application.
14. The non-transitory computer-readable medium of claim 8 , wherein the operations further include:
analyzing the received data to identify at least one real-world object in the MR environment;
monitoring over time a movement and an orientation of the identified real-world object in the MR environment; and
controlling an aspect of operation during the executing of the matched contextual application based on an interaction of a user with the identified object, the interaction including manipulating the movement and the orientation of the real-world object.
15. A method comprising:
performing, using one or more processors, operations for implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user, the operations comprising:
receiving data describing surroundings of the environment in proximity to the MR-capable device from a plurality of sensors on the MR-capable device;
analyzing the received data, the analyzing including comparing the received data against a plurality of predefined contextual triggers associated with a plurality of contextual applications in order to determine a matching set of predefined contextual triggers associated with a matched contextual application, wherein each of the predefined contextual triggers comprises a specific configuration of data from a sensor of the plurality of sensors;
determining a reality layer associated with the device; and
executing the matched contextual application based on the determined reality layer, wherein the executing of the matched contextual application is based on permissions associated with the determined reality layer, wherein the permissions make available predetermined instructions in the contextual application which are associated with the determined reality layer, the executing of the available predetermined instructions causing the matched contextual application to perform operations that are specific to the determined reality layer.
16. The method of claim 15 , wherein the operations further include determining a reality layer associated with a user of the MR-capable device.
17. The method of claim 15 , wherein the operations further include:
analyzing the received data to identify at least one real-world object in the MR environment;
monitoring over time a movement and an orientation of the identified real-world object in the MR environment; and
controlling an aspect of operation during the executing of the matched contextual application based on an interaction of a user with the identified object, the interaction including manipulating the movement and the orientation of the real-world object.