Interactivity with a mixed reality
Methods of interacting with a mixed reality are presented. A mobile device captures an image of a real-world object where the image has content information that can be used to control a mixed reality object through an offered command set. The mixed reality object can be real, virtual, or a mixture of both real and virtual.
1. A mixed-reality command server computer system comprising at least one processor, one programmed to:
receive, from a mobile device:
image data captured by an image sensor of the mobile device, the image data including an image of a real-world object; and
sensor data captured by a second sensor of the mobile device;
derive attributes associated with the real-world object with the image data;
determine that a virtual object is local to the mobile device based on at least one of the image data and the sensor data, wherein the virtual object is associated with a game;
derive a command set for the real-world object as a mixed-reality object based on the attributes and the sensor data, wherein:
the mixed-reality object comprises a virtual representation of the real-world object in the game; and
the command set includes at least one command corresponding to an in-game interaction with the virtual representation of the real-world object and involving the virtual object; and
cause the mobile device to present the command set to a user via a display of the mobile device within a mixed-reality view, such that when the mobile device is moved, the command set moves within the display of the mobile device to remain in proper relation to the image of the real-world object within the mixed reality view.
2. The mixed-reality command server of claim 1 , further programmed to:
receive, from the mobile device, a command from the at least one command of the command set, the command comprising a network address; and
execute a game function associated with the network address, the game function corresponding to the in-game interaction.
3. The mixed-reality command server of claim 2 , wherein the network address comprises at least one of a URL, a URI, a remote procedure call, an email address, an IP address, a web service API, and a network-callable command.
4. The mixed-reality command server of claim 1 , wherein the at least on command from the command set comprises at least one in-game function of the virtual representation of the real-world object, the in-game function comprising a virtual depiction of a real-world function of the real-world object.
5. The mixed-reality command server of claim 1 , wherein the server programmed to cause the mobile device to present the command set such that when the mobile device is moved, the command set moves within the display of the mobile device to remains in proper relation to the real-world object further comprises the server programmed to superimpose the command set over the image of the real-world object within the mixed reality view.
6. The mixed-reality command server of claim 5 , wherein the server programmed to superimpose the command set over the image of the real-world object further comprises the server programmed to superimpose the command set over an image of a corresponding real-world interface part of the real-world object.
7. The mixed-reality command server of claim 6 , wherein the command set is a non-visible command set.
8. The mixed-reality command server of claim 1 , wherein the image data includes a plurality of real-world objects and wherein the plurality of real-world objects includes the real-world object, the mixed-reality command server further programmed to cause the mobile device to present the virtual object within the mixed reality view based on at least one of the plurality of the real-world objects.
9. The mixed-reality command server of claim 8 , wherein the server programmed to present the virtual object within the mixed reality view based on at least one of the plurality of the real-world objects further comprises the server programmed to render the virtual object such that the virtual object is at least partially hidden by the at least one of the plurality of the real-world objects.
10. The mixed-reality command server of claim 1 , wherein the real-world object comprises a goal object in the game.
11. The mixed-reality command server of claim 1 , wherein the real-world object has an in-game value within the game.
12. The mixed-reality command server of claim 1 , wherein the sensor data comprises an environmental characteristic.
13. The mixed-reality command server of claim 1 , wherein the sensor data comprises information associated with a physical location.
14. The mixed-reality command server of claim 1 , wherein the sensor data comprises information associated with at least one of an orientation and an acceleration of the mobile device.
15. The mixed-reality command server of claim 1 , wherein the command comprises inputting data associated with a virtual weapon.
16. The mixed-reality command server of claim 1 , wherein the image data comprises a still image.
17. The mixed-reality command server of claim 1 , wherein the image data comprises a video image.
18. The mixed-reality command server of claim 1 , wherein the real-world object comprises at least one of an appliance, a kitchen appliance, a business office appliance, a clinic appliance, a printer, a set-top box, a television, a vending machine, a radio, and a stereo.
19. The mixed-reality command server of claim 1 , further programmed to:
identify a second command set for the virtual object based on at least one of the image data and the sensor data, the second command set include at least one second command associated with the virtual object in the game; and
cause the mobile device to present the command set to the user via the display of the mobile device within the mixed-reality view.
20. The mixed-reality command server of claim 1 , wherein the virtual object does not have a corresponding physical object in the real-world.