CONTROL OF A REAL WORLD OBJECT USER INTERFACE
Systems and methods described allow users to select and obtain significant information about objects in the real world, and further to employ gestures as a “real world” interface to manipulate information and to manipulate the selection of objects. In this way, users may be enabled to make better decisions when, e.g., traveling and shopping, and may further be enabled to obtain significant information about objects with which they are surrounded. The systems and methods may include a camera which monitors a user's hand movements or gestures to control a UI, particularly where a user is interacting with real-world objects. Gestures can move the focus of a UI from one real world object to another. The systems and methods may also include a projector to illuminate selected objects, or to display information about objects.
1 . A method of selecting a physical object using a camera, comprising:
a. receiving a first image from a camera;
b. analyzing the first image to detect a gesture performed by a user, the gesture indicating a physical object, the physical object also portrayed within the first image; and
c. causing the display of an indicator of the physical object, the display of an indicator further comprising:
i. causing the display of an indicator of the physical object on a mobile device user interface; or
ii. causing light to be projected on the physical object.
2 . The method of claim 1 , wherein the causing the display of an indicator of the physical object, further comprises causing text or an image to be projected onto a surface, the text or image including data about the physical object.
3 . The method of claim 1 , further comprising transmitting the first image to a server, and wherein the analyzing is performed on the server.
4 . The method of claim 1 , wherein the analyzing is at least partially performed on the mobile device, and further comprising:
a. displaying an indicator of the physical object if an identity of the physical object is available from the analysis on the mobile device; and
b. if the analysis on the mobile device is insufficient to determine an identity of the physical object, then further including:
i. transmitting the first image to a server for additional analysis;
ii. receiving a result of the additional analysis; and
iii. displaying data about the result on the mobile device.
5 . The method of claim 1 , wherein the receiving, analyzing, and displaying are performed on the mobile device.
6 . The method of claim 1 , wherein the indicator is data about the physical object.
7 . The method of claim 1 , further comprising activating a light source to cause light to be projected in a first direction on the physical object.
8 . The method of claim 7 , wherein the activating a light source further comprises:
a. activating a light source to project a second image, a portion of the second image in highlight, the highlight configured to be present on the physical object when the second image is projected; or
b. activating a collimated light source, the collimated light source having a direction, the direction towards the physical object.
9 . The method of claim 1 , wherein the detected gesture selects the physical object.
10 . The method of claim 7 , wherein the detected gesture further causes the light source to project light in a direction different from the first direction.
11 . The method of claim 10 , wherein the detected gesture further causes the light source to project light on a different physical object.
12 . The method of claim 1 , wherein the analyzing the image further comprises analyzing objects within a detection zone defined within a field of view of the camera.
13 . The method of claim 12 , wherein the detected gesture further causes the detection zone to change in size or position.
14 . The method of claim 1 , wherein the detected gesture causes the camera or the light source or both to reset to a respective default direction of the camera or the light source.
15 . The method of claim 1 , wherein the detected gesture causes light projected on the physical object to remain stationary while the camera or the light source or both is moved, or wherein the detected gesture causes light projected on the physical object to remain stationary for duration of time, whereby a user may alter gesture or limb position without causing movement of the projected light.
16 . The method of claim 1 , wherein the indicator is a menu for actions related to the physical object.
17 . The method of claim 7 , wherein the camera and light source are in a fixed positional relationship, and further comprising performing a step of calibration, the calibration including:
a. a first method including:
i. projecting a beam of light in a known direction from the light source; and
ii. detecting the location of the beam of light relative to an axis or plane of a lens of the camera; or
b. a second method including:
i. projecting an image including a target in a known direction from the light source; and
ii. detecting the location of the target relative to an axis or plane of a lens of the camera.
18 . The method of claim 1 , wherein the displaying is performed on a headmounted display.
19 . The method of claim 18 , wherein the headmounted display includes eyeglasses.
20 . The method of claim 1 , further comprising:
a. detecting a SKU, barcode or product packaging to identify a physical object;
b. transmitting the SKU, barcode or product packaging to a server; and
c. receiving and displaying information about the physical object based on the transmitted SKU, barcode or product packaging.
21 . A non-transitory computer readable medium, comprising instructions for causing a computing device to perform the method of claim 1 .